Imaging system, imaging method, moving body control device, and program

Abstract

[Problem] To provide a imaging system, a imaging method, a moving body control device, and a program that enable imaging according to a difference in an area for use and an area to be imaged in a facility such as a sport stadium. [Solution] A imaging system (10) comprising a moving body (20) for imaging, the moving body (20) being provided with a camera (231) for imaging and a moving body control device (420) which controls a movement at the time of imaging and imaging of the moving body. The moving body control device comprises an area-to-be-imaged setting unit (441) which sets the area to be imaged or the area size thereof by detecting the area to be imaged or receiving user input of the area to be imaged. The moving body control device further switches at least any one of a movement condition and an imaging condition of the moving body at the time of imaging according to the area size of the area to be imaged.

Description

Photography system, photography method, mobile body control device and program

[0001] The present invention relates to a photographing system, a photographing method, a mobile object control device, and a program.

[0002] Patent Literature 1 discloses various aspects of a system and method for capturing a moving scene using multiple cameras in a network (Abstract). To solve this problem, the system in Patent Literature 1 (Abstract) includes multiple unmanned aerial vehicles (UAVs). Each of the multiple UAVs includes an imaging device configured to capture multiple images. A first UAV of the multiple UAVs includes a first imaging device configured to capture a first series of images of one or more stationary and / or moving objects. The first UAV is configured to receive focus lens information, a current position, and a current orientation from the one or more imaging devices. The first UAV determines a target position and a target orientation for each of the one or more imaging devices. The first UAV communicates control information to one or more other UAVs to correct the current position and current orientation of each of the one or more imaging devices to the determined target position and target orientation.

[0003] Patent Document 1 mentions a stadium or other facility as an example of a moving scene to be photographed (

[0003] ). It also introduces a method for setting the target position and target orientation of each imaging device using triangulation (

[0032] , Figures 5A to 5D). Another example is the photographing of a golf player from various angles (

[0153] , Figures 6A to 6C).

[0004] Japanese Patent Application Laid-Open No. 2020-115642

[0005] As described above, Patent Document 1 (Abstract) discloses a method for capturing images using multiple unmanned aerial vehicles (UAVs) in a facility such as a stadium. The imaging device of each UAV is controlled using a target position and a target orientation.

[0006] For example, in the case of a soccer field, a normal game is played using the entire soccer field (full court). In the case of practice, while a full court may be used, there are also cases where only half of the soccer field (half court) is used. Furthermore, the size of each soccer field often varies. Similarly, in the case of a baseball field, a normal game is played using the entire baseball field. In the case of practice, while the entire baseball field may be used, there are also cases where only the infield or only the outfield is used. Furthermore, the size of each baseball field often varies. Patent Document 1 does not consider cases where the usable area or the area to be photographed of one stadium (such as a soccer field or baseball field) changes, or cases where the size of the area to be photographed of different stadiums changes.

[0007] Furthermore, futsal is a sport similar to soccer. A (full) futsal court is smaller than a (full) soccer court. Patent Document 1 does not fully consider the case where the size or shape of the usable area or the area to be photographed is different between different stadiums (soccer courts or futsal courts) where similar sports such as soccer and futsal are played.

[0008] The above-mentioned problems are not limited to cases where a camera is mounted on an airborne vehicle, but also apply to cases where a camera is mounted on a ground vehicle.

[0009] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a photography system, photography method, mobile object control device, and program that are capable of taking photographs according to differences in the usage area or target area of ​​a facility such as a stadium.

[0010] A photography system according to one aspect of the present invention comprises a photography moving body having a photography camera, and a mobile body control device that controls the movement of the moving body and photography during photography, wherein the mobile body control device comprises a photography target area setting unit that sets the photography target area or its area size by detecting the photography target area, which is all or part of a single stadium, or by receiving user input for the photography target area, and further characterized in that the mobile body control device switches at least one of the movement conditions or photography conditions of the moving body during photography depending on the area size of the photography target area.

[0011] According to the present invention, at least one of the moving conditions or photographing conditions of the moving object used by the moving object control device during photographing is switched according to the area size of the photographed area. Therefore, even if the area size differs for each stadium, photographing according to the area size is possible. Furthermore, even when photographing at the same stadium, it is possible to differentiate the moving conditions or photographing conditions when using the entire stadium from the moving conditions or photographing conditions when using only a part of the stadium. Therefore, photographing according to differences in the usable area or photographed area at the stadium is possible.

[0012] The area size may be the area of ​​the target area. Alternatively, if the target area is rectangular, the area size may be the length of the short side or the long side of the target area. Alternatively, the area size may be a combination of the area, the length of the short side, or the length of the long side.

[0013] The photographing target area setting unit may set the photographing target area or its area size by distinguishing whether it is all or part of the one stadium by detecting the photographing target area which is all or part of the one stadium or by receiving user input of the photographing target area. Furthermore, the photographing target area setting unit may, for the same stadium, make at least one of the movement conditions of the moving object or the photographing conditions different when the photographing target area is the entire stadium and when it is part of the stadium. This makes it possible to use movement conditions or photographing conditions appropriate for the photographing target area when it is part of a stadium (for example, when a game is played using a half court).

[0014] The mobile object control device may switch the target flight altitude or permitted flight altitude range during photography in accordance with the area size of the photography target area, thereby enabling precise control of the flight altitude for each area size of the photography target area in one stadium.

[0015] The mobile body control device may set multiple objects of interest for a viewer of the image acquired by the photographic camera. The mobile body control device may identify positions of the multiple objects of interest in the image acquired by the photographic camera. The mobile body control device may calculate a centroid from the positions of the multiple objects of interest. The mobile body control device may control the angle of view of the photographic camera so that the centroid is located at a predetermined position (e.g., the center) of the acquired image. This makes it possible to adjust the angle of view so that the multiple objects of interest are easily visible to a viewer of the image acquired by the photographic camera.

[0016] The mobile object control device may switch an allowable range of imaging directions or a target imaging direction during imaging in accordance with the area size of the imaging target area, thereby enabling precise control of the imaging direction for each area size of the imaging target area.

[0017] The mobile object control device may switch the allowable range of the zoom amount or the target zoom amount during shooting in accordance with the area size of the shooting target area, thereby enabling precise control of the zoom amount for each area size of the shooting target area.

[0018] The mobile object control device may switch the geofence area in the vertical direction or the horizontal direction during image capture according to the area size of the image capture target area, thereby enabling precise geofence setting for each area size of the image capture target area.

[0019] The mobile object control device may switch a range of permitted flight altitudes during photography in accordance with the area size of the photography target area, and allow manual operation of the altitude of the photography moving object within the range of permitted flight altitudes, thereby enabling manual operation of the flight altitude in accordance with the area size of the photography target area.

[0020] The mobile object control device may set a plurality of geofences divided into areas in which the mobile object can fly. The mobile object control device may switch the timing of switching the geofences when switching between areas in which the mobile object can fly, depending on the size of the area to be photographed. This makes it possible to precisely set the timing of switching the flight geofences for each size of the area to be photographed.

[0021] The plurality of flight modes for photography may be switchable between a bird's-eye photography mode for photographing the entire photography target area from a bird's-eye view and a partial photography mode for photographing a part of the photography target area. When switching the flight mode, the mobile object control device may set a joint geofence that combines the geofence used in the flight mode before switching and the geofence used in the flight mode after switching. This enables smooth flight when switching flight modes.

[0022] The imaging target area setting unit may detect a candidate imaging target area based on a surrounding image acquired by the moving object. The imaging target area setting unit may set the imaging target area or its area size by using the candidate imaging target area as is or by having the user select or confirm the candidate imaging target area. This is expected to completely match the imaging target area (or candidate imaging area) detected when setting the imaging target area with the imaging target area used during normal imaging. Therefore, for example, if the user checks the detection result of the imaging target area (or candidate imaging area) when setting the imaging target area, it is possible to increase the reliability of the imaging target area during normal imaging.

[0023] The photographing target area setting unit may include a candidate area database storing candidates for the photographing target area. The photographing target area setting unit may also extract the candidate photographing target area or its area size that matches a user's specified conditions from the candidate area database and set it as the photographing target area or its area size. This allows the user to easily set the photographing target area or its area size if the user has previously acquired information on the photographing candidate area that the user wants to use as the photographing target area.

[0024] The mobile object control device may change the target flight altitude, or the upper limit flight altitude, or the lower limit flight altitude based on the detected information of the external environment. This makes it possible to restrict the movement of the mobile object and ensure flight stability, for example, when an external environment that affects the flight stability of the mobile object occurs.

[0025] The detected information of the external environment may be wind speed. When the wind speed exceeds a wind speed threshold, the mobile object control device may lower the target flight altitude or the upper limit flight altitude. In this way, in an environment where strong winds are occurring, lowering the flight altitude of the mobile object makes it possible to improve flight stability.

[0026] The detected information of the external environment may be the reception strength of satellite signals, the number of satellites receiving the satellite signals (number of communication satellites), or the elevation angle of the satellites receiving the satellite signals. The mobile object control device may increase at least one of the target flight altitude, the upper limit flight altitude, or the lower limit flight altitude when the reception strength falls below a reception strength threshold, or when the number of satellites falls below a predetermined number, or when the elevation angle of the satellite falls below a predetermined angle. This makes it possible to change the flight state to, for example, increase the reception strength of satellite signals.

[0027] Another aspect of the present invention is a photography method that uses a photography system comprising: a photography moving body having a photography camera; and a mobile body control device that controls the movement of the moving body and photography during photography, wherein the mobile body control device has a photography target area setting unit that sets the photography target area or its area size by detecting the photography target area or by receiving user input for the photography target area, and further, the mobile body control device switches at least one of the movement conditions or photography conditions of the moving body during photography depending on the area size of the photography target area.

[0028] According to the present invention, at least one of the moving conditions or the photographing conditions of the moving body used by the moving body control device during photographing is switched according to the area size of the photographing area. Therefore, by varying the moving conditions or the photographing conditions for each area size of the photographing area, precise movement or photographing becomes possible.

[0029] A mobile body control device according to yet another aspect of the present invention controls the movement and photography of a mobile body for photography having a photography camera, and the mobile body control device is equipped with a photography target area setting unit that sets the photography target area or its area size by detecting the photography target area or by receiving user input for the photography target area, and further characterized in that the mobile body control device switches at least one of the movement conditions or photography conditions of the mobile body during photography depending on the area size of the photography target area.

[0030] A further aspect of the present invention is a program executed by a mobile body control device that controls the movement and photography of a mobile body for photography having a photography camera, and is characterized in that the mobile body control device executes the following steps: setting the area to be photographed or its area size by detecting the area to be photographed or by accepting user input of the area to be photographed; and switching at least one of the movement conditions or photography conditions of the mobile body during photography depending on the area size of the area to be photographed.

[0031] According to the present invention, it is possible to take photographs according to differences in the area to be used or the area to be photographed in a facility such as a stadium.

[0032] 7 is a diagram illustrating an overall configuration of an imaging system according to an embodiment of the present invention. FIG. 1 is a diagram illustrating a functional configuration of a drone according to the embodiment. FIG. 2 is an external perspective view showing a simplified outline of the drone according to the embodiment. FIG. 3 is a functional configuration diagram of a control device according to the embodiment. FIG. 4 is an external front view showing a simplified outline of the control device according to the embodiment. FIG. 5 is a diagram illustrating a functional configuration diagram of a server according to the embodiment. FIG. 6 is a flowchart illustrating an overall flow of aerial photography control according to the embodiment. FIG. 7 is a flowchart illustrating a setting of a photography target area according to the embodiment (details of S101 in FIG. 7). FIG. 8 is a diagram illustrating an example of a hovering operation of the drone when setting the photography target area according to the embodiment. FIG. 9 is a diagram illustrating an example of an operation screen when setting the photography target area according to the embodiment. FIG. 10 is a diagram illustrating an example of a relationship between the area of ​​the photography target area and flight conditions and photography conditions in the bird's-eye photography mode and the area flight mode according to the embodiment. FIG. 11 is a diagram illustrating an example of geofences in the vertical and horizontal directions in the bird's-eye photography mode and the area flight mode according to the embodiment. FIG. 12 is a diagram illustrating an example of initial target flight altitudes in the bird's-eye photography mode and the area flight mode according to the embodiment, and corresponding permitted flight areas. FIG. 13 is an explanatory diagram of flight mode transitions according to the embodiment. FIG. 14 is a diagram illustrating an example of a photography reference position of the drone used in the embodiment. Fig. 16(A) is a diagram showing each shooting reference position when the entire stadium is set as the shooting target area in the embodiment, and Fig. 16(B) is a diagram simply illustrating an example of a screen displayed on the display unit of the control device corresponding to Fig. 16(A). Fig. 17(A) is a diagram showing each shooting reference position when half of the stadium is set as the shooting target area in the embodiment, and Fig. 17(B) is a diagram simply illustrating an example of a screen displayed on the display unit of the control device corresponding to Fig. 17(A). Fig. 18(A) is a diagram simply illustrating an example of a screen displayed on the display unit of the control device when the angle of view adjustment process is not performed in the embodiment, and Fig. 18(B) is a diagram simply illustrating an example of a screen displayed on the display unit of the control device when the angle of view adjustment process is performed in the embodiment. These figures show geofences for the bird's-eye view shooting mode, the area flight mode, and the outside-area takeoff and landing mode in the embodiment.23(A) is a flowchart of flight stability control in the embodiment. 23(B) is a flowchart of setting a shooting target area in a first modified example (details of S101 in FIG. 7). 23(C) is a diagram showing an example of an operation screen when setting a shooting target area in the first modified example of FIG. 21. 23(A) is a diagram showing the positions of the short and long sides when the shooting target area is rectangular. 23(B) is a diagram showing an example of the relationship between the short sides of the shooting target area and flight conditions and shooting conditions in a second modified example. 23(C) is a diagram showing an example of the relationship between the long sides of the shooting target area and flight conditions and shooting conditions in a third modified example. 24(A) is a diagram showing the shooting reference position 6, etc. when the entire stadium of a standard size is set as the shooting target area in a fourth modified example. 24(B) is a diagram showing the shooting reference position 6, etc. when the entire stadium of a standard size is set as the shooting target area in a fourth modified example.

[0033] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiments described below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc.

[0034] <A. One Embodiment> [A-1. Configuration] (A-1-1. Overall Configuration) Fig. 1 is a diagram showing the overall configuration of an imaging system 10 (hereinafter also referred to as "system 10") according to one embodiment of the present invention. The system 10 uses a drone 20 to capture aerial images of a predetermined area (facility) where people are likely to be active, such as a competition being held at a stadium 90 (Fig. 9, etc.) or an event being held at an event venue. As shown in Fig. 1, in addition to the drone 20, the system 10 also includes a control device 30 that allows a pilot to operate the drone 20, and a server 40 that manages the flight and imaging of the drone 20.

[0035] In this embodiment, a "competition" refers to a competition of skills or abilities, and includes soccer and various other sports. It is not limited to matches, but may also include practice sessions. A stadium is a place where a competition is held, and may include not only the area inside the court defined by the lines shown in FIG. 9, but also the area outside the court. The present invention is not limited to competitions and events, and can be applied to an imaging system that captures an area to be photographed for any purpose.

[0036] The drone 20 and the control device 30 are connected to each other via wireless communication (which may include communication via a base station 52). The control device 30 and the server 40 are connected to each other via a communication network 50 such as the Internet. The drone 20 acquires satellite signals from an artificial satellite 54 to set its own position, etc. The configuration of the system 10 is not limited to that shown in FIG. 1 , and other configurations are also usable (details will be described later).

[0037] (A-1-2. Drone 20) (A-1-2-1. Overview of the Drone 20) Fig. 2 is a functional configuration diagram of the drone 20 of this embodiment. Fig. 3 is a simplified external perspective view of the drone 20 of this embodiment. As described above, the drone 20 takes aerial photographs of competitions taking place in the stadium 90 (Fig. 9), events taking place at event venues, etc. As shown in Fig. 2, the drone 20 has a drone sensor group 200, a communication unit 210, a flight mechanism 220, a photographing mechanism 230, and a drone control unit 240.

[0038] In this specification, the term "drone" refers to any flying object that has multiple rotors and the ability to autonomously control its attitude, regardless of the power source (electricity, prime mover, etc.), the control method (wireless or wired, fully autonomous flight type or partially manual flight type, etc.), and whether manned or unmanned. Drones are also sometimes called unmanned aerial vehicles (UAVs), flying objects, multicopters, RPASs (remote piloted aircraft systems), or UASs (unmanned aircraft systems), etc.

[0039] (A-1-2-2. Drone Sensor Group 200) The drone sensor group 200 includes various sensors arranged on the drone 20. Specifically, the drone sensor group 200 has a position measurement unit 201, a direction measurement unit 202, an altimeter 203, a speedometer 204, a gyro sensor 205, an obstacle sensor 206, an anemometer 207, etc. In addition to these, the drone sensor group 200 may also include various sensors that acquire information such as temperature, air pressure, and acceleration.

[0040] The position measurement unit 201 receives signals from the artificial satellites 54 ( FIG. 1 ) and measures the position (absolute position) of the aircraft based on the signals. The position measurement unit 201 measures its current position using, for example, a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS), although this is not particularly limited. For example, a Real Time Kinematic - Global Navigation Satellite System (RTK-GNSS) can also be used as a method for measuring the position. The position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably three-dimensional coordinate information including altitude information.

[0041] Furthermore, the base station 52, which provides information on the reference point of a fixed station used for relative positioning such as RTK, is wirelessly connected to the drone 20 and the control device 30, thereby enabling measurement of the position of the drone 20 with higher accuracy. When RTK measurement is performed using a virtual reference point method with a VRS (Virtual Reference Station), the base station 52 can be omitted, or the accuracy of the position coordinate estimation of the base station 52 or the drone 20 can be further improved.

[0042] The position measurement unit 201 of this embodiment can determine the reception strength of the received signal from the artificial satellite 54, the number of artificial satellites 54 receiving the satellite signal, or the elevation angle of the artificial satellite 54 receiving the satellite signal.

[0043] The orientation measurement unit 202 measures the orientation (heading direction) of the drone 20. The orientation measurement unit 202 is composed of a geomagnetic sensor that measures the heading direction (heading direction) of the drone 20 by measuring geomagnetism, a compass, etc.

[0044] The altimeter 203 measures the altitude above ground (hereinafter referred to as "altitude H") as the distance from the ground below (vertically downward) the drone 20. The measured value of altitude H acquired by the altimeter 203 is also referred to as the measured altitude Hd. The speedometer 204 detects the flight speed of the drone 20. The gyro sensor 205 detects the angular velocity of the drone 20. The obstacle sensor 206 has multiple control cameras 208 (Figure 3) and measures the position, speed vector, etc. of people located below the drone 20, etc., based on the acquired images.

[0045] The anemometer 207 detects the wind direction and wind speed around the drone 20. When the drone 20 itself is moving, the wind direction and wind speed may be corrected using the direction and speed of movement of the drone 20.

[0046] (A-1-2-3. Communication Unit 210) Communication unit 210 is capable of radio wave communication via communication network 50 (FIG. 1) and includes, for example, a radio wave communication module. Communication unit 210 is capable of communication with control device 30 and the like via communication network 50 (including wireless base station 52).

[0047] (A-1-2-4. Flight Mechanism 220) The flight mechanism 220 is a mechanism that causes the drone 20 to fly, and generates thrust in the airframe for lifting the drone 20 and moving it in a desired direction. As shown in Figures 2 and 3, the flight mechanism 220 has a plurality of rotors 221 and a plurality of rotor actuators 222. The rotor actuators 222 have, for example, electric motors.

[0048] The flight mechanism 220 may also be provided with a propeller guard (not shown) to prevent the propeller from interfering with obstacles. The number of rotors 221 constituting the flight mechanism 220 is not particularly limited, but may include, for example, one, two, four, six, or eight rotors. The rotor 221 may be composed of a single propeller, or may be composed of multiple propellers arranged coaxially. The number and shape of the blades of each propeller are not particularly limited.

[0049] (A-1-2-5. Filming mechanism 230) The filming mechanism 230 is a mechanism for capturing images of competitions at the stadium 90 (FIG. 9), events at the event venue, etc., and has a camera 231, a camera holding unit 232, and an image processing unit 233. As shown in FIG. 3, the camera 231 (imaging device) is disposed at the bottom of the main body of the drone 20, and outputs image data relating to peripheral images captured around the drone 20. The camera 231 is a video camera (color camera) that captures video. The video may include audio data acquired by a microphone (not shown). Additionally or alternatively, the camera 231 may also capture still images.

[0050] The orientation of the camera 231 (the attitude of the camera 231 relative to the main body of the drone 20) can be adjusted by a camera actuator (not shown) incorporated in the camera holding unit 232. Alternatively, the position of the camera 231 relative to the main body of the drone 20 may be fixed. The camera holding unit 232 may have a mechanism for suppressing transmission of shaking or vibration of the aircraft to the camera 231. The image processing unit 233 performs predetermined image processing on image data acquired by the camera 231. The image data acquired by the camera 231 can be transmitted to a storage unit of the drone 20 itself, the control device 30, the server 40, etc. Part or all of the image processing unit 233 may be positioned as part of the photography control unit 242 described below.

[0051] (A-1-2-6. Drone control unit 240) The drone control unit 240 controls the entire drone 20, including flying and photographing the drone 20. The drone control unit 240 includes an input / output unit, a calculation unit, and a storage unit, which are not shown. The drone control unit 240 includes a calculation device such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The drone control unit 240 includes a flight control unit 241 and a photography control unit 242.

[0052] The flight control unit 241 controls the flight of the drone 20 (attitude control and flight operations of the aircraft from takeoff to flight and landing) via the flight mechanism 220. The flight control unit 241 has a processing unit, also called a flight controller. The processing unit may have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit has access to memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps. The memory may include, for example, a separable medium such as an SD card or RAM, or an external storage device. Various data acquired from the drone sensor group 200 may be directly transmitted to and stored in the memory. For example, video or still image data captured by the camera 231 may be recorded in an internal memory or an external memory.

[0053] The processing unit includes a control module configured to control the airframe state of the drone 20. For example, the control module controls the flight mechanism 220 (thrust generating unit) of the drone 20 to adjust the spatial configuration, attitude angle, angular velocity, angular acceleration, angular jerk rate, and / or acceleration of the drone 20 having six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz). The control module can control one or more of the camera holder 232 and sensors.

[0054] The flight control unit 241 can control the flight of the drone 20 based on control signals from the control device 30 or based on a preset autonomous flight program. The flight control unit 241 can also control the flight of the drone 20 by controlling the flight mechanism 220 (thrust generation unit) based on various information such as the photographing area, flight permitted / prohibited areas, information on the corresponding flight geofences, map information including two-dimensional or three-dimensional map data, the current position information of the drone 20, attitude information (heading information), speed information, and acceleration information, as well as any combination of these.

[0055] In this specification, "target shooting area" (or target shooting field) refers to a two-dimensional location to be photographed (for example, the outline defining the stadium 90 (for example, the frame consisting of the touchlines and goal lines of a soccer field)). "Flight permitted / prohibited area" refers to a three-dimensional unit of space that permits or prohibits flight of the drone 20. "Geofence" refers to a virtual boundary line, and in particular refers to a fence that marks the boundary between a flight permitted area in which a mobile object such as the drone 20 is permitted to fly or move and a flight prohibited area. Therefore, if a mobile object such as the drone 20 comes into contact with a geofence, flight or movement is restricted to prevent the aircraft from flying outside the flight permitted area.

[0056] The photography control unit 242 controls photography by the drone 20 via the photography mechanism 230.

[0057] (A-1-3. Control Device 30) (A-1-3-1. Overview of Control Device 30) FIG. 4 is a functional configuration diagram of the control device 30 of this embodiment. FIG. 5 is a simplified front view of the exterior of the control device 30 of this embodiment. The control device 30 is a portable information terminal that controls the drone 20 through operation by the operator and displays information received from the drone 20 (e.g., position, altitude, remaining battery level, camera footage, etc.). In this embodiment, the flight status (altitude, attitude, etc.) of the drone 20 is remotely controlled by the control device 30, but the drone 20 may also control it autonomously. In that case, when the operator transmits a flight command to the drone 20 via the control device 30, the drone 20 performs autonomous flight. However, manual operation may be possible during basic operations such as takeoff and return, and in emergencies.

[0058] The control device 30 has an input / output unit 300 and a communication unit 310. The input / output unit 300 performs various inputs by a user such as a pilot, outputs to the user, and inputs and outputs signals between the drone 20 and the server 40. The input / output unit 300 includes an operation input unit 320 and a display unit 330. The operation input unit 320 and the display unit 330 are connected to each other so that they can communicate with each other via wire or wirelessly. The control device 30 also includes a calculation unit such as a CPU for executing information processing, and storage devices such as RAM and ROM. Furthermore, the control device 30 of this embodiment receives and displays work instructions, etc. from the server 40.

[0059] (A-1-3-2. Communication Unit 310) The communication unit 310 is disposed in the same housing as the operation input unit 320 or the display unit 330, and has a communication function for wirelessly communicating with the drone 20 using Wi-Fi, 2.4 GHz, or a frequency band of 5.6 to 5.8 GHz. The communication unit 310 also has a wireless communication function that enables communication with the server 40 via an internet line 50 using a communication standard such as LTE (Long Term Evolution). In the system 10 ( FIG. 1 ) of this embodiment, the drone 20 communicates with the server 40 via the control device 30. Therefore, this system configuration is suitable for cases where the drone 20 and the control device 30 are located within a distance where direct wireless communication is possible (for example, visual flight by a pilot, etc.), but is not limited thereto.

[0060] (A-1-3-3. Operation Input Unit 320) The operation input unit 320 accepts various inputs from a user such as a pilot, and inputs operational commands such as flight direction and takeoff / landing when the pilot pilots the drone 20. For example, the operation input unit 320 accepts input operations that instruct three-dimensional flight operations of the drone 20, including takeoff, landing, ascent, descent, rotation, forward movement, backward movement, and left / right movement. As shown in FIG. 4 , the operation input unit 320 of this embodiment includes a movement input unit 321, a drone attitude input unit 322, a camera attitude input unit 323, a camera zoom input unit 324, a flight mode switching unit 325, an imaging condition input unit 326, and a power input unit 327. As shown in FIG. 5 , the operation input unit 320 as hardware includes a left input stick 340L, a right input stick 340R, a left two-way switch button 341L, a right two-way switch button 341R, and a menu button 342.

[0061] The movement input unit 321 is an input unit that allows the pilot to move the drone 20 in the up / down, left / right, and forward / backward directions, and is composed of a right input stick 340R and a left input stick 340L. That is, when the right input stick 340R is moved upward (toward the rear), the drone 20 rises, and when the right input stick 340R is moved downward (toward the viewer), the drone 20 descends. When the right input stick 340R is moved to the right, the drone 20 moves right, and when the right input stick 340R is moved to the left, the drone 20 moves left. When the left input stick 340L is moved upward (toward the rear), the drone 20 moves forward, and when the left input stick 340L is moved downward (toward the viewer), the drone 20 moves backward.

[0062] The drone attitude input unit 322 is an input unit that allows the pilot to control the attitude of the drone 20, and is configured with the left input stick 340L. The attitude of the drone 20 referred to here includes yaw rotation. That is, when the left input stick 340L is moved to the right, the drone 20 turns right, and when the left input stick 340L is moved to the left, the drone 20 turns left.

[0063] The camera attitude input unit 323 is an input unit for manipulating the attitude of the photographing camera 231, and is composed of a right bidirectional switch button 341R and a right input stick 340R. That is, when the right side of the right bidirectional switch button 341R is pressed, the camera 231 moves to the right. When the left side of the right bidirectional switch button 341R is pressed, the camera 231 moves to the left. Furthermore, when the right input stick 340R is pressed downward while pressing the right bidirectional switch button 341R, the camera 231 moves downward. When the right input stick 340R is pressed upward while pressing the right bidirectional switch button 341R, the camera 231 moves upward.

[0064] The camera zoom input unit 323 is an input unit for operating the zoom of the photographing camera 231, and is configured with a left two-way switch button 341L. That is, when the right side of the left two-way switch button 341L is pressed, the camera 231 zooms in. When the left side of the left two-way switch button 341L is pressed, the camera 231 zooms out.

[0065] The flight mode switching unit 325 is an input unit for switching the flight mode of the drone 20, and is composed of a menu button 342 and a left input stick 340L. That is, the flight mode is switched by pressing the menu button 342 and selecting from the displayed menu using the left input stick 340L. The shooting condition input unit 326 is an input unit for selecting shooting conditions such as the shooting target area and target shooting position of the drone 20, and is composed of the menu button 342 and a left input stick 340L. That is, the shooting conditions such as the shooting target area and target shooting position are selected by pressing the menu button 342 and selecting from the displayed menu using the left input stick 340L. The power input unit 327 is a component for turning the power of the control device 30 on and off, and is composed of a mechanical switch or the like.

[0066] (A-1-3-4. Display Unit 330) The display unit 330 displays to the pilot status information of the drone 20 obtained from the drone 20 or the server 40. The display unit 330 may be configured as a touch panel or LCD monitor that is integrally built into the control device 30, or may be configured as a display device such as an LCD monitor, tablet terminal, or smartphone that is connected to the control device 30 by wire or wirelessly. The display unit 330 can display images related to various information such as the area to be photographed, permitted / prohibited flight areas, flight geofences, map information, current position information of the drone 20, attitude information (directional information), speed information, acceleration information, and remaining battery power.

[0067] (A-1-4. Server 40) (A-1-4-1. Overview of the Server 40) Fig. 6 is a functional configuration diagram of the server 40 of this embodiment. The server 40 manages or controls the flight and photography of the drone 20. As shown in Fig. 6, the server 40 has an input / output unit 400, a communication unit 410, a calculation unit 420, and a storage unit 430. The input / output unit 400 is a part for inputting or outputting various types of information (image output, audio output). The communication unit 410 has a modem or the like (not shown), and is capable of communicating with the drone 20, the control device 30, etc. via the communication network 50.

[0068] The calculation unit 420 includes a CPU and operates by executing a program stored in the storage unit 430. Some of the functions executed by the calculation unit 420 can be realized using a logic IC (Integrated Circuit). Some of the programs of the calculation unit 420 can also be configured using hardware (circuit components).

[0069] The storage unit 430 stores programs and data used by the calculation unit 420 and includes a RAM. The RAM can be a volatile memory such as a register, or a non-volatile memory such as a hard disk or flash memory. The storage unit 430 may also include a ROM in addition to the RAM.

[0070] The server 40 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0071] (A-1-4-2. Calculation unit 420) As shown in Fig. 6, the calculation unit 420 has a pre-setting unit 440, a flight control unit 450, and an imaging control unit 460. The pre-setting unit 440 performs pre-settings for the flight and imaging of the drone 20. The flight control unit 450 controls the flight of the drone 20. The imaging control unit 460 controls imaging by the drone 20.

[0072] As shown in FIG. 6 , the presetting unit 440 includes a photographing target area setting unit 441, a flight condition setting unit 442, a photographing condition setting unit 443, and an area size determination unit 444. The photographing target area setting unit 441 is a unit that sets the photographing target area, for example, by a user's operation of the menu button 342 and the left input stick 340L, or by detecting the photographing target area from an image captured by the drone 20. The flight condition setting unit 442 is a unit that sets flight conditions, for example, by a user's operation of the menu button 342 and the left input stick 340L, or based on preset rules. The photographing restriction setting unit 443 is a unit that sets photographing conditions, for example, by a user's operation of the menu button 342 and the left input stick 340L, or based on preset rules.

[0073] The flight control unit 450 controls the flight of the drone 20. The flight here includes, in addition to flight during shooting, a takeoff operation, movement from the takeoff point to a target shooting location (e.g., the stadium 90), movement from the target shooting location to a target landing point, and a landing operation.

[0074] The photography control unit 460 has a photography mode setting unit 461, a drone state acquisition unit 462, a photography direction restriction unit 463, a zoom restriction unit 464, a flight restriction unit 465, and an automatic photography control unit 466. The photography mode setting unit 461 is a unit that sets the photography mode. The drone state acquisition unit 462 is a unit that acquires the state of the drone 20. The photography direction restriction unit 463 is a unit that restricts the photography direction of the camera 231. The zoom restriction unit 464 is a unit that restricts the zoom amount of the camera 231. The flight restriction unit 465 is a unit that restricts the flight of the drone 20. The automatic photography control unit 466 is a unit that controls automatic photography by the camera 231.

[0075] [A-2. Control] (A-2-1. Overall Flow) Next, various types of control in this embodiment will be described. FIG. 7 is a flowchart showing the overall flow of aerial photography control in this embodiment. As described above, in this embodiment, aerial photography of a competition taking place in a stadium 90 (FIG. 9), an event taking place at an event venue, etc. is performed by drone 20. Control in this embodiment can be roughly divided, as shown in FIG. 7, into control for pre-setting before aerial photography (pre-setting control in step S10) and control when aerial photography is performed (aerial photography control in step S20).

[0076] (A-2-2. Presetting Control (S10 in FIG. 7)) (A-2-2-1. Selection of Image Capture Area) In step S101 in FIG. 7, the server 40 (image capture area setting unit 441 of the presetting unit 440) sets the image capture area based on operations by the user of the control device 30, etc. FIG. 8 is a flowchart (details of S101 in FIG. 7) for setting the image capture area in this embodiment. FIG. 9 is a diagram illustrating an example of the hovering operation of the drone 20 when setting the image capture area in this embodiment.

[0077] In step S1011 of Figure 8, the user places the drone 20 at a predetermined position on the stadium 90 (Figure 9) that the user wants to use as the target area for photography. For example, if the user wants to use a portion (half court) of the stadium 90 in Figure 9 as the target area for photography, the user places the drone 20 outside one of the touchlines 901 (the lower touchline 901 in Figure 9) midway between one of the goal lines 900 (the left goal line 900 in Figure 9) and the center line 902 of the stadium 90. In this case, the drone 20 is oriented so that the camera 231 faces the stadium 90.

[0078] In step S1012, when a command to capture a candidate photographing area (candidate photographing area) is input from the user to the control device 30, the server 40 causes the drone 20 to hover at a predetermined height and acquire surrounding images. The acquired surrounding images are sequentially transmitted to the server 40. In FIG. 9 , reference numeral 260 denotes a geofence of the drone 20.

[0079] In step S1013, the server 40 detects potential photography areas by processing the peripheral image from the drone 20. For example, the server 40 detects the potential photography areas by detecting the white lines (center line 902, one or both goal lines 900, both touch lines 901, etc.) that define the outline of the stadium 90. In step S1014, the server 40 causes the display unit 330 of the control device 30 to display a display indicating the potential photography areas.

[0080] FIG. 10 is a diagram showing an example of an operation screen used when setting a target photography area in this embodiment. The operation screen 350 is displayed on the display unit 330 after screen data generated by the server 40 is transmitted to the control device 30. The screen 350 in FIG. 10 displays a line 360 ​​(overall contour line 360) indicating the outline of the entire stadium 90, as well as a display 361 (candidate area display 361) indicating a candidate photography area. The screen 350 also displays multiple bidirectional arrows 362 for changing the range of the candidate area display 361, a guidance message 363, and an approval button 364 for setting the candidate photography area as the target photography area. In addition to these displays, a surrounding image may also be displayed. In other words, these displays may be displayed superimposed on the surrounding image.

[0081] If the detected candidate shooting area is appropriate (S1015 in FIG. 8: true), the user selects the approve button 364. This causes the server 40 to set the current candidate shooting area (in other words, the area corresponding to the candidate area display 361) as the target shooting area (S1016). On the other hand, if the detected candidate shooting area is not appropriate (S1015: false), the user operates the bidirectional arrow 362 to adjust the position of the candidate area display 361, and then selects the approve button 364 to set the target shooting area (S1017).

[0082] In step S1018, the server 40 determines the area size of the photographing target area based on the information of the photographing target area set in step S1016 or S1017. Here, the area size is the area X (m ２ Alternatively, if the imaging target area is rectangular, the area size may be defined by the length of the short side or the length of the long side of the imaging target area, or a combination thereof.

[0083] (A-2-2-2. Setting flight conditions and photographing conditions (S102 in Figure 7)) Returning to Figure 7, in step S102, the server 40 (flight condition setting unit 442 and photographing condition setting unit 443 of the pre-setting unit 440) sets flight conditions and photographing conditions according to the photographing target area set in step S101.

[0084] 11 is a diagram showing an example of the relationship between the area of ​​the image capture area and the flight conditions and image capture conditions in the bird's-eye view image capture mode and the area flight mode in this embodiment. As shown in Fig. 11, in this embodiment, the range of the geofence in the vertical direction, the initial value of the target flight altitude of the drone 20 (initial target altitude), and the initial value of the zoom amount of the camera 231 (initial target zoom amount) are set according to the area X of the image capture area determined by the area size determination unit 444. Of these, the range of the geofence in the vertical direction and the initial target altitude are flight conditions, and the initial target zoom amount is an image capture condition.

[0085] 12 is a diagram showing an example of a geofence 260a in the vertical and horizontal directions in the bird's-eye view photography mode and a geofence 260b in the vertical and horizontal directions in the area flight mode set in this embodiment. FIG. 13 is a diagram showing the initial value of the target flight altitude Htar1 in the bird's-eye view photography mode and the corresponding permitted flight area 92a, and the initial value of the target flight altitude Htar2 in the area flight mode and the corresponding permitted flight area 92b. Both FIGS. 12 and 13 show an example in which half of the stadium 90 is selected as the photography target area.

[0086] The area size determination unit 444 may determine the area X of the area to be photographed, for example, from the altitude Hd of the drone 20 when the peripheral image was acquired, the orientation of the camera 231, the zoom amount, etc. Alternatively, as will be described later, if a database related to the stadium 90 exists, the area size determination unit 444 may calculate the area X of the area to be photographed using basic data of the stadium 90 identified from the position of the drone 20 (this may include reading the area X of the area to be photographed from the database).

[0087] The flight geofence 260 (also referred to as "geofence 260") indicates a virtual boundary line, and in particular indicates a fence that is the boundary line between a flight-permitted area in which a mobile object such as the drone 20 is permitted to fly or move and a no-fly area. Therefore, when a mobile object such as the drone 20 comes into contact with the geofence 260, the flight or movement is restricted so that the aircraft does not fly outside the flight-permitted area.

[0088] As shown in the example of FIG. 12 , the geofence 260 is actually set as a three-dimensional area with height added to length and width (however, for control purposes, it may be managed as a two-dimensional area). The flight geofence, flight altitude (flight conditions), and zoom amount (photography conditions) set according to the area size (area) of the photography target area in accordance with the table shown in FIG. 11 may be displayed on the operation screen 350 as shown in FIG. 10 , allowing the user to change the flight conditions and photography conditions. At this time, multiple bidirectional arrows for changing the geofence 260, a guidance message, and an approval button for confirming the geofence may also be displayed. In addition to these displays, a surrounding image may also be displayed. In other words, these displays may be displayed in a format in which they are superimposed on the surrounding image.

[0089] (A-2-3. Control During Aerial Photography (S20 in FIG. 7)) (A-2-3-1. Overview of Control During Aerial Photography) As described above, the control during aerial photography (S20 in FIG. 7) is control performed when performing aerial photography. In the control during aerial photography, in step S201 of FIG. 7, the server 40 executes control (outbound movement control) to move the drone 20 from the flight position in the advance preparation flight mode to a shooting start position (for example, the initial value of the target flight altitude Htar1 in the bird's-eye view photography mode) of the shooting target area (such as the stadium 90). In the following step S202, the server 40 executes control (shooting target area shooting control (or aerial photography execution control)) to perform aerial photography of the shooting target area by the drone 20 in the permitted flight area defined by the geofence. Finally, in step S203, the server 40 executes control (return movement control) to move the drone 20 from the permitted flight area to the target landing point.

[0090] (A-2-3-2. Outward Movement Control (S201 in FIG. 7)) As described above, in outward movement control, the drone 20 is moved from the flight position in the advance preparation flight mode to the shooting start position (a predetermined position in the sky) of the shooting target area (such as the stadium 90). The drone 20 then hovers at the shooting start position. At this time, the altitude of the drone 20 is adjusted to be within the range of the geofence in the vertical direction set in step S102. In addition, the zoom amount of the camera 231 is adjusted to the initial target zoom amount set in step S102.

[0091] (A-2-3-3. Aerial Photography Execution Control (S202 in FIG. 7)) (A-2-3-3-1. Overview) As described above, in the aerial photography execution control of this embodiment, aerial photography is performed by the drone 20 in the photography target area. In the aerial photography of this embodiment, a mode selected by the user or the server 40 from among a plurality of photography flight modes is used. Photography flight modes include overhead photography mode and area flight mode (details will be described later with reference to FIG. 14, etc.). In the normal state when photography is being performed (overhead photography mode or area flight mode), the photography position of the drone 20 is selected by the user or the server 40 from a plurality of preset reference positions (photography reference positions). In addition, the photography direction and zoom amount of the camera 231 are selected corresponding to each photography reference position. These will be described later with reference to FIGS. 15 to 17.

[0092] (A-2-3-3-2. Flight Modes During Photography) Figure 14 is an explanatory diagram of flight mode transitions in this embodiment. As shown in Figure 14, the flight modes used in this embodiment include an outside-area takeoff and landing mode, a preparatory flight mode, an overhead photography mode, an inside-area entry mode, an area flight mode, an outside-area exit mode, and an inside-area takeoff and landing mode. The preparatory flight mode is a flight mode used in the pre-setting control (S10 in Figure 7), particularly in setting the photography target area (S101). The outside-area takeoff and landing mode is a flight mode used in steps S101 and S203 in Figure 7. Therefore, the flight modes during photography include the overhead photography mode, the inside-area entry mode, the area flight mode, the outside-area exit mode, and the inside-area takeoff and landing mode.

[0093] The overhead shooting mode is a flight mode used to capture an overhead image of the entire shooting target area from outside the shooting target area. The area flight mode is a flight mode used to capture an image of a portion of the shooting target area from inside the shooting target area. The area entry mode is a flight mode used when switching from the overhead shooting mode to the area flight mode, and moves the drone 20 from outside to inside the shooting target area. The area exit mode is a flight mode used when switching from the area flight mode to the overhead shooting mode, and moves the drone 20 from inside to outside the shooting target area. The area takeoff and landing mode is a flight mode used when landing the drone 20 within the shooting target area or taking off from the shooting target area. The area takeoff and landing mode can be used, for example, when some abnormality occurs in the drone 20 or when severe weather conditions (strong winds, etc.) occur.

[0094] It should be noted that each of the flight modes described above also corresponds to the image capture control of the camera 231. Therefore, it should be noted that each of the flight modes listed here may also be referred to as a control mode of the camera 231.

[0095] (A-2-3-3-3. Photographing reference position and associated flight conditions and photographing conditions) As described above, in the normal state (bird's-eye view photographing mode or area flight mode) when photographing is being performed, the photographing position of the drone 20 is selected from a plurality of preset reference positions (photographing reference positions) by the user or the server 40. In addition, the photographing direction and zoom amount of the camera 231 are basically selected in accordance with each photographing reference position.

[0096] FIG. 15 is a diagram showing an example of the imaging reference positions of the drone 20 used in this embodiment. In FIG. 15, the imaging reference positions are indicated by circled numbers 1 to 15. Hereinafter, these imaging reference positions will also be referred to as "imaging reference positions 1 to 15," "reference positions 1 to 15," or "positions 1 to 15." Furthermore, each arrow corresponding to each of positions 1 to 15 indicates the pan (horizontal) direction of the camera 231 depending on its orientation, and the tilt (vertical) direction of the camera 231 depending on its length. In other words, a longer arrow indicates that the camera 231 is oriented closer to the horizontal direction, and a shorter arrow indicates that the camera 231 is oriented closer to the vertical direction.

[0097] As is clear from Fig. 15, the reference positions in Fig. 15 are used when the entire stadium 90 is set as the area to be photographed. Note that the case where half of the stadium 90 is set as the area to be photographed will be described later with reference to Fig. 17.

[0098] 15, reference positions 1 to 5 are located outside the stadium 90 (or touch line 901) and are used in the bird's-eye view shooting mode. Reference positions 6 to 15 are located inside the stadium 90 (or inside the two goal lines 900 and two touch lines 901) and are used in the area flight mode. For example, reference position 1 (altitude zero) can be used as the takeoff and landing position for the drone 20.

[0099] Although the altitude of the drone 20 is not shown in Figure 15, for example, as shown in Figure 12, inside the target shooting area, the target flight altitude Htar may be set for each position within the range of the geofence in the vertical direction. Alternatively, as shown in Figure 13, the target flight altitude Htar may be constant inside the target shooting area. However, as will be described later with reference to Figures 16 and 17, in this embodiment, the target flight altitude Htar, target zoom amount, etc. are adjusted so that the shooting range is the same or similar for each corresponding reference position when the entire stadium 90 is set as the target shooting area and when half of the stadium 90 is set as the target shooting area.

[0100] These multiple reference positions 1 to 15 can be used as selectable positions for one drone 20. That is, the user or the server 40 can select any of the reference positions for one drone 20. Alternatively, these reference positions can also be used for simultaneous shooting by multiple drones 20.

[0101] (A-2-3-3-4. Handling When the Area Size of the Photographed Area Varies) FIG. 16(A) is a diagram showing each photographing reference position when the entire stadium 90 is set as the photographed area in this embodiment, and FIG. 16(B) is a diagram simply illustrating an example of a screen displayed on the display unit 330 of the control device 30 corresponding to FIG. 16(A). FIG. 17(A) is a diagram showing each photographing reference position when half of the stadium 90 is set as the photographed area in this embodiment, and FIG. 17(B) is a diagram simply illustrating an example of a screen displayed on the display unit 330 of the control device 30 corresponding to FIG. 17(A). Screen 370a in FIG. 16(B) is a screen displayed when photographing is performed at reference position 7 in FIG. 16(A). Screen 370b in FIG. 17(B) is a screen displayed when photographing is performed at reference position 7 in FIG. 17(A).

[0102] As can be seen from Figures 16(B) and 17(B), for the same (or corresponding) reference position 7, the target flight altitude Htar, target zoom amount, etc. are adjusted to obtain the same or similar shooting range. For example, the target flight altitude Htar is lower and / or the target zoom amount (magnification ratio) is smaller in the case of Figure 17(B) than in Figure 16(B). The pan direction (horizontal direction) and tilt direction (vertical direction) of the camera 231 are basically the same in both the cases of Figures 16(B) and 17(B). However, the pan direction and / or tilt direction may be changed to bring the shooting range closer together.

[0103] (A-2-3-3-5. Camera 231 Angle of View Adjustment Processing) As explained with reference to Figures 15 to 17(B), in this embodiment, the target flight altitude Htar of the drone 20, the shooting direction (pan and tilt) of the camera 231, and the target value of the zoom amount are basically set to initial values ​​for each of the shooting reference positions 1 to 15. In this embodiment, the user of the control device 30 (the viewer of the images shot by the drone 20) can select a shooting mode that performs processing to adjust the angle of view of the camera 231 (angle of view adjustment processing) so that the competition is easier to view.

[0104] Figure 18(A) is a diagram showing a simplified example of a screen displayed on the display unit 330 of the control device 30 when the angle of view adjustment process is not performed in this embodiment, and Figure 18(B) is a diagram showing a simplified example of a screen displayed on the display unit 330 of the control device 30 when the angle of view adjustment process is performed in this embodiment.

[0105] 18(A), athlete images 381a, 381b, and 381c are located in the lower left of screen 380a, and no one is present in the upper right of screen 380a. This may make it difficult to see the movement of each athlete image 381a, 381b, and 381c. Therefore, in the angle of view adjustment process, the angle of view of camera 231 is adjusted so that athlete images 381a, 381b, and 381c are displayed near the center of screen 380b.

[0106] The angle of view adjustment process is performed as follows. That is, the server 40 processes the image acquired by the drone 20 and extracts each player image from the image. The player images can be extracted using existing technologies such as image binarization, feature point extraction, and pattern matching. The number of player images to be extracted is not limited to three. For example, all player images present in the image may be extracted. The server 40 then instructs the drone 20 to adjust the angle of view of the camera 231 so that the centroid of each detected player image is located at the center of the entire image. In Figures 18(A) and 18(B), the centroids are indicated by the symbol 382. Note that the centroid 382 may be calculated not only from the positions of the player images, but also as the centroid of player images 381a, 381b, and 381c and ball image 383. Alternatively, the angle of view of the camera 231 may be adjusted so that the ball image 383 is located at the center of the entire image. The centroid 382 may be calculated as a two-dimensional centroid on the plane of the acquired image, or as a three-dimensional centroid in the actual stadium.

[0107] Furthermore, in addition to the players, substitute players, managers, coaches, spectators, etc. may be present on the stadium 90. Taking such cases into consideration, the server 40 may detect lines (such as the goal line 900, center line 902, and touchline 901) that define the photographic area from the images acquired by the drone 20, distinguish between players who are within the photographic area and other people who are outside the photographic area, and calculate the centroid 382 for only the players. Alternatively, to exclude spectators and the like from the calculation of the centroid, the server 40 may distinguish between players (players and substitute players) and other people by the color of their uniforms, and calculate the centroid 382 for only the players.

[0108] (A-2-3-3-6. When changing flight modes) As described above, in this embodiment, geofences are set in both the vertical and horizontal directions (see, for example, FIG. 12). Separate geofences are set for each flight mode, such as the bird's-eye view shooting mode and the area flight mode.

[0109] FIG. 19 is a diagram showing geofences 260a, 260b, and 260c for the bird's-eye view photography mode, area flight mode, and outside-area takeoff and landing mode, respectively, in this embodiment. The geofence 260a in the bird's-eye view photography mode is a relatively narrow area in a planar view, but has a relatively wide range in the height direction. The geofence 260b in the area flight mode is a relatively wide area in a planar view, but has a relatively narrow range in the height direction. Note that the lower limit altitude Hlow is also shown for the geofence 260b. Furthermore, the geofence 260c in the outside-area takeoff and landing mode is positioned to cover the takeoff and landing point of the drone 20 and the area above it, and, like the geofence 260a, is a relatively narrow area in a planar view.

[0110] As described above, when transitioning from the bird's-eye view photography mode to the area flight mode, the mode is the inside-area entry mode, and when transitioning from the area flight mode to the bird's-eye view photography mode, the mode is the outside-area exit mode ( FIG. 14 ). In this embodiment, the inside-area entry mode and the outside-area exit mode use a composite geofence that combines the geofences 260a and 260b of the bird's-eye view photography mode and the area flight mode, respectively. Furthermore, when transitioning from the outside-area takeoff and landing mode to the bird's-eye view photography mode, and conversely, when transitioning from the bird's-eye view photography mode to the outside-area takeoff and landing mode, a joint geofence that temporarily combines the geofences 260a and 260c is used. Note that, as shown in FIG. 14 , in this embodiment, an example has been described in which the outside-area takeoff and landing mode is transitioned to the outside-area takeoff and landing mode via the bird's-eye view photography mode when landing outside the area from the area flight mode. However, instead, the area flight mode may be transitioned directly to the outside-area takeoff and landing mode without transitioning via the bird's-eye view photography mode. During this mode transition, a combined geofence is used that combines the area flight mode geofence 260b and the outside area takeoff and landing mode geofence 260c.

[0111] The timing of switching the geofence when switching flight modes may be changed depending on the area size (dimensions) of the image capture area. For example, in the examples of Figures 16(A) and 17(A), when the drone 20 moves from reference position 1 to reference position 7, the geofence is smaller in the case of Figure 17(A) where the image capture area is smaller, making it easier for the drone 20 to collide with the geofence. Therefore, when the image capture area is small, the timing of appearing the composite geofence may be made earlier.

[0112] (A-2-3-3-7. Control for Ensuring Flight Stability) As described above, in this embodiment, a vertical geofence or target flight altitude Htar is set for each flight mode. Exceptionally, if an external environment occurs that affects the flight stability of the drone 20, the drone control unit 240 executes flight stability control to temporarily lower the altitude (target flight altitude Htar) of the drone 20. Examples of such external environments include strong winds around the drone 20, a decrease in the reception strength of signals (satellite signals) from satellites 54, a state in which the number of satellites 54 receiving satellite signals falls below a predetermined number, or a state in which the elevation angle of the satellites 54 receiving satellite signals is lower than a predetermined angle. The flight stability control may be executed by the flight control unit 450 instead of the drone control unit 240.

[0113] FIG. 20 is a flowchart of flight stability control in this embodiment. In step S301, the drone control unit 240 acquires the wind speed V of the anemometer 207 and the reception strength S of the satellite signal. In step S302, the drone control unit 240 determines whether the wind speed V is smaller than the wind speed threshold THv. If the wind speed V is smaller than the wind speed threshold THv (S302: True), strong winds are not occurring in the external environment that would affect the flight stability of the drone 20. Therefore, the process proceeds to step S304. On the other hand, if the wind speed V is not smaller than the wind speed threshold THv (S302: False), strong winds are occurring. Therefore, in step S303, the drone control unit 240 temporarily lowers the target flight altitude Htar of the drone 20 (in this case, the drone may land within the target shooting area). Instead of the target flight altitude Htar, the upper limit flight altitude of the geofence may be temporarily lowered.

[0114] In step S304, the drone control unit 240 determines whether the reception strength S is greater than the strength threshold THs. If the reception strength S is greater than the strength threshold THs (S303: true), there is no decrease in the reception strength S due to the external environment that affects the flight stability of the drone 20. Therefore, the current flight stability control is terminated, and the process returns to step S301. On the other hand, if the reception strength S is not greater than the strength threshold THs (S303: false), there is a decrease in the reception strength S. Therefore, in step S305, the drone control unit 240 temporarily increases the target flight altitude Htar of the drone 20 to increase the reception strength S. Instead of the target flight altitude Htar, the lower limit flight altitude (and / or upper limit flight altitude) of the geofence may be temporarily increased.

[0115] Here, if there are tall trees, buildings, etc. around the target imaging area, the satellite signal may be blocked, reducing the strength (dB) of the satellite signal carrier wave, or the number of satellites 54 from which satellite signals can be received may be reduced, or only satellites 54 with low elevation angles may be available as satellites 54 from which satellite signals can be received, which is undesirable from the perspective of ensuring flight stability. Therefore, in addition to the example shown in Figure 20 in which the target flight altitude Htar is temporarily increased when the satellite signal reception strength S is lower than the intensity threshold THs, instead of the satellite signal reception strength S, the target flight altitude Htar may also be temporarily increased when the number of satellites 54 from which satellite signals are received falls below a predetermined number or when the elevation angle of the satellite 54 from which satellite signals are received is lower than a predetermined angle.

[0116] [A-3. Effects of this embodiment] According to this embodiment, at least one of the flight conditions or imaging conditions of the drone 20 (mobile body) used by the calculation unit 420 (mobile body control device) of the server 40 during imaging is switched according to the area size (area X, etc.) of the imaging target area ( FIG. 11 ). Therefore, even if the area size differs for each stadium 90, imaging according to the area size is possible. Furthermore, even when imaging the same stadium 90, it is possible to differentiate the movement conditions or imaging conditions when using the entire stadium 90 from the movement conditions or imaging conditions when using only a portion of the stadium 90. Therefore, imaging according to differences in the area used or the imaging target area of ​​the stadium 90 is possible.

[0117] According to this embodiment, the imaging target area setting unit 441 sets the imaging target area or its area size by distinguishing whether it is the entire stadium 90 or a portion thereof (for example, by distinguishing between a full court and a half court) ( FIGS. 10 and 11 ). Furthermore, for the same stadium 90, the imaging target area setting unit 441 differentiates at least one of the flight conditions or imaging conditions of the drone 20 (mobile body) during imaging depending on whether the imaging target area is the entire stadium 90 or a portion thereof ( FIGS. 11 and 16(A) to 17(B)). This allows for the use of flight conditions or imaging conditions appropriate for a portion of a stadium 90 (for example, when a sport is played on a half court).

[0118] In this embodiment, the calculation unit 420 (mobile object control device) of the server 40 switches the target flight altitude Htar during shooting or the range of the geofence 260 in the vertical direction (permitted flight altitude area) according to the area X (area size) of the area to be shot ( FIG. 11 ). This makes it possible to precisely control the flight altitude H for each area X of the area to be shot in one stadium 90.

[0119] In this embodiment, the calculation unit 420 (mobile object control device) of the server 40 sets athlete images 381a, 381b, and 381c as multiple objects of interest for viewers of the image acquired by the filming camera 231 ( FIG. 18(B) ). The calculation unit 420 also identifies the positions of the multiple athlete images 381a, 381b, and 381c within the image acquired by the filming camera 231 and calculates a centroid 382 from the positions of the multiple athlete images. The calculation unit 420 controls the angle of view of the filming camera 231 so that the centroid 382 is located at the center of the acquired image ( FIG. 18(B) ). This makes it possible to adjust the angle of view so that the multiple athlete images 381a, 381b, and 381c are easily visible to viewers of the image acquired by the filming camera 231.

[0120] In this embodiment, the calculation unit 420 (mobile object control device) of the server 40 switches the allowable range of the imaging direction or the target imaging direction during imaging depending on the area X (area size) of the imaging target area (FIGS. 16A and 17A). This makes it possible to precisely control the imaging direction for each area X of the imaging target area.

[0121] In this embodiment, the calculation unit 420 (mobile body control device) of the server 40 switches the allowable range of zoom amount or the target zoom amount during shooting according to the area X (area size) of the area to be shot (FIGS. 11, 16A, and 17A). This makes it possible to precisely control the zoom amount for each area X of the area to be shot.

[0122] In this embodiment, the calculation unit 420 (mobile object control device) of the server 40 switches the area of ​​the geofence 260 in the vertical or horizontal direction during shooting according to the area X (area size) of the shooting target area (FIGS. 11 and 12). This makes it possible to precisely set the geofence 260 for each area X of the shooting target area.

[0123] In this embodiment, the calculation unit 420 (mobile object control device) of the server 40 switches the range of permitted flight altitudes during photography according to the area X (area size) of the photography target area, and allows manual operation within the range of permitted flight altitudes (FIGS. 11 and 12). This allows manual operation of the flight altitude according to the area X of the photography target area.

[0124] In this embodiment, the calculation unit 420 (mobile body control device) of the server 40 sets multiple flight modes for photography divided by the area in which the drone 20 (mobile body) can fly ( FIG. 14 ). The calculation unit 420 also sets geofences 260a, 260b, and 260c for each flight mode ( FIG. 19 ). The calculation unit 420 switches the geofence switching timing when switching flight modes depending on the area X (area size) of the area to be photographed. This makes it possible to precisely set the switching timing of the flight geofence for each area X of the area to be photographed.

[0125] In this embodiment, the multiple flight modes for photography can be switched between a bird's-eye view photography mode, which captures the entire photography target area from a bird's-eye view, and an area flight mode (partial photography mode), which captures a portion of the photography target area (FIG. 14). When switching flight modes, the calculation unit 420 (mobile object control device) of the server 40 sets a combined geofence that combines the geofence used in the flight mode before the switch and the geofence used in the flight mode after the switch (FIG. 19). This enables smooth flight when switching flight modes.

[0126] In this embodiment, the imaging target area setting unit 441 ( FIG. 6 ) detects candidate imaging target areas based on peripheral images acquired by the drone 20 (moving body) (S1013 in FIG. 8 ). The imaging target area setting unit 441 sets the imaging target area by having the user select or confirm the candidate imaging target area (S1015 and S1016 in FIG. 8 and FIG. 10 ). This is expected to ensure that the imaging target area (or candidate imaging area) detected when setting the imaging target area is completely consistent with the imaging target area used during normal imaging. Therefore, for example, if the user checks the detection result of the imaging target area (or candidate imaging area) when setting the imaging target area, the reliability of the imaging target area during normal imaging can be improved.

[0127] In this embodiment, the calculation unit 420 (mobile object control device) of the server 40 changes the target flight altitude Htat, the upper limit flight altitude, or the lower limit flight altitude based on the wind speed V and the reception strength S (detection information of the external environment) ( FIG. 20 ). As a result, if an external environment occurs that affects the flight stability of the drone 20, it is possible to restrict the movement of the drone 20 and ensure flight stability.

[0128] The detected information of the external environment is wind speed V. Furthermore, when wind speed V exceeds wind speed threshold THv (S302 in FIG. 20: false), calculation unit 420 (mobile object control device) of server 40 lowers the target flight altitude or upper limit flight altitude (S303). As a result, in an environment where strong winds are occurring, lowering the flight altitude of drone 20 can improve flight stability.

[0129] The detected information of the external environment is the satellite signal reception strength S. If the reception strength S is below the reception strength threshold THs (S304 in FIG. 20: false), the calculation unit 420 (mobile object control device) of the server 40 increases the target flight altitude or the lower limit flight altitude (and / or the upper limit flight altitude) (S305). This makes it possible to change the flight state so as to increase the satellite signal reception strength S.

[0130] <B. Modifications> The present invention is not limited to the above-described embodiment, and various configurations can be adopted based on the contents of this specification. For example, the following configurations can be adopted.

[0131] [B-1. Configuration] (B-1-1. Photography System 10) The photography system 10 of the above embodiment was intended to photograph sports (soccer, baseball, tennis, etc.) taking place in a stadium 90 (see FIG. 9, etc.). However, this is not limited to this, for example, if attention is paid to the fact that the movement conditions or photography conditions of the drone 20 during photography can be switched depending on the area size of the photography target area, or that not only the entire facility but also a part of it can be set as the photography target area. For example, the photography target is not limited to the sports mentioned above, and the photography system can also be applied to other events (exhibitions, concerts, ceremonies, etc.) that attract people.

[0132] In the above embodiment, the photography system 10 has the configuration shown in FIG. 1 . However, the present invention is not limited to this configuration, for example, by switching the movement conditions or photography conditions of the drone 20 during photography depending on the size of the photography target area, or by allowing not only the entire facility but also part of it to be set as the photography target area. For example, the drone 20 can be configured to wirelessly communicate directly with the internet line 50 using a communication method such as LTE without using the control device 30. Alternatively, a system redundancy configuration is possible in which multiple servers 40 are connected to one or multiple drones 20 via multiple internet lines 50.

[0133] The device described in the above embodiment may be realized as a single device, or may be realized by a plurality of devices (e.g., cloud server 40, drone 20, control device 30) that are partly or entirely connected via a communication network 50. For example, each functional unit and memory unit of the server 40 may be implemented in a different server 40, drone 20, or control device 30 that is connected to each other via the communication network 50.

[0134] (B-1-2. Drone 20) In the above embodiment, basic behavior control of the drone 20 during image capture was performed based on operation of the control device 30 ( FIG. 5 ). However, for example, if attention is paid to the fact that the movement conditions or image capture conditions of the drone 20 during image capture are switched according to the area size of the image capture target area, or that not only the entire facility but also part of it can be set as the image capture target area, the basic behavior control of the drone 20 is not limited to this, and may be automatic control (without manual operation) by the drone 20 itself or based on commands from the server 40.

[0135] In the above embodiment, the drone 20 is used as an example of a mobile body for aerial photography, but the present invention is not limited to this, as long as, for example, the movement conditions or photography conditions of the mobile body for photography can be switched according to the area size of the photography target area, or not only the entire facility but also a part of it can be set as the photography target area. The mobile body may be, for example, a camera system that can move along a wire placed in the air within the stadium 90. Alternatively, the mobile body may be a mobile camera system (including one that moves on rails) placed on the ground.

[0136] (B-1-3. Control Device 30) The control device 30 in the above embodiment has the configuration shown in Figures 4 and 5. However, this is not limited to this, for example, if attention is paid to the fact that the movement conditions or image capture conditions of the drone 20 during image capture are switched according to the area size of the image capture target area, or that not only the entire facility but also part of it can be set as the image capture target area. With regard to the operation input unit 320, for example, the number and arrangement of input sticks, and the number, shape, and arrangement of buttons can be changed as appropriate. Alternatively, the configuration realized by the operation input unit 320 in Figure 5 can be replaced with a touch panel.

[0137] The control input unit 320 may have a takeoff button and a landing button that instruct automatic takeoff and landing, a flight start button that instructs the aircraft to automatically fly to a specified position and hover there, a home button that performs a return operation to the starting position, a mode switch button that switches flight modes, etc.

[0138] [B-2. Control] The series of processes described in connection with the above embodiment may be realized using software, hardware, or a combination of software and hardware. A computer program for realizing each function of the server 40 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium on which such a computer program is stored may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. Furthermore, the above computer program may be distributed, for example, via a communication network 50 without using a recording medium.

[0139] The flowcharts used in the above embodiments do not necessarily have to be executed in the order shown in the drawings. Some processing steps may be executed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted.

[0140] In the presetting control (S10 in FIG. 7 ) of the above embodiment, the candidate shooting area is detected based on the surrounding image acquired by the drone 20 (S101 in FIG. 7 , FIGS. 8 and 10 ). However, this is not limited to this, considering that the movement conditions or shooting conditions of the drone 20 during shooting can be switched depending on the area size of the target shooting area, or that not only the entire facility but also part of it can be set as the target shooting area.

[0141] FIG. 21 is a flowchart (details of S101 in FIG. 7 ) for setting a target shooting area in a modified example. While FIG. 8 shows a target shooting area set based on a potential shooting area detected using images acquired by the drone 20, the modified example of FIG. 21 stores data on the target shooting area in advance in a database (candidate area database) along with various information (e.g., the address, name, full court / half court status, and location coordinates of the stadium 90). This candidate area database (candidate area DB) can be stored, for example, in the storage unit 430 of the server 40. The server 40 then accepts a user's input via the control device 30 to identify a target shooting area stored in the candidate area DB (step S1101). The server 40 then retrieves the target shooting area corresponding to the user's input from the candidate area DB and displays it on the display unit 330 of the control device 30 (step S1102). The server 40 then sets the target shooting area selected by the user from the candidate shooting areas (step S1103).

[0142] FIG. 22 is a simplified diagram illustrating an example of an operation screen used when setting a photographing target area in the modified example of FIG. 21 . The operation screen 390 is displayed on the display unit 330 after screen data generated by the server 40 is transmitted to the control device 30. The screen 390 of FIG. 22 displays an image 391 (full-court image 391) showing the entire playing field 90 (full court). Also displayed within the full-court image 391 are images 392a and 392b (half-court images 392a and 392b) showing half of the playing field 90 (half court). The user selects either the full-court image 391 or the half-court image 392a or 392b by touch operation or the like. In the example of FIG. 22 , the half-court image 392b is selected, and is therefore highlighted with a thicker line.

[0143] Although not shown in Fig. 22, the operation screen 390 can display a guidance message similar to the guidance message 363 in Fig. 10 and a button similar to the approval button 364. Therefore, the user can read the guidance message and understand the operations related to setting the shooting target area. Then, with either the full-court image 391 or the half-court image 392a or 392b selected by touch operation or the like, the shooting target area is set by pressing the approval button (not shown).

[0144] According to the modified examples of FIGS. 21 and 22 , the photographic target area setting unit 441 includes a candidate area DB that stores candidate photographic target areas. Furthermore, the photographic target area setting unit 441 extracts candidate photographic target areas that meet the user's specified conditions from the candidate area DB and sets them as the photographic target area (S1102 and S1103 in FIG. 21 ). This allows the user to easily set the photographic target area if they have previously acquired information about the candidate photographic area they want to use as the photographic target area. In this case, the area size determination unit 444 determines the area size by determining the area X of the photographic target area, or the length of the short side or the long side if the photographic target area is rectangular, based on information about the set photographic target area (e.g., the coordinates of the four corners of the court, etc.). Alternatively, the area X, etc. included in the candidate area DB may be used directly as the area size.

[0145] 23(A) is a diagram showing the positions of the short side W and long side L when the imaging target area is rectangular, and Fig. 23(B) is a diagram showing an example of the relationship between the short side W of the imaging target area and the flight conditions and imaging conditions in Modification 2. In Modification 2 of Fig. 23(B), the flight condition setting unit 442 and the imaging condition setting unit 443 set the range of the geofence in the vertical direction, the initial value of the target flight altitude of the drone 20 (initial target altitude), and the initial value of the zoom amount of the camera 231 (initial target zoom amount) according to the short side W of the imaging target area.

[0146] 23(C) is a diagram showing an example of the relationship between the long side L of the photographing target area and the flight conditions and photographing conditions in Modification 3. In Modification 3 of FIG. 23(C), the flight condition setting unit 442 and the photographing condition setting unit 443 set the range of the geofence in the vertical direction, the initial value of the target flight altitude of the drone 20 (initial target altitude), and the initial value of the zoom amount of the camera 231 (initial target zoom amount) according to the long side L of the photographing target area.

[0147] Figures 24(A) and 24(B) show an example of a method for setting flight conditions for the photographing reference position in the fourth modified example. That is, Figure 24(A) is a diagram showing the photographing reference position 6, etc., when the entire stadium 90a of the standard size is set as the photographing target area in the fourth modified example, and Figure 24(B) is a diagram showing the photographing reference position 6, etc., when the entire stadium 90b, which is larger than the standard size, is set as the photographing target area in the fourth modified example. The photographing reference positions 6 in Figures 24(A) and 24(B) each correspond to the photographing reference position 6 shown in Figure 15.

[0148] In Figure 24(A), in addition to the photographing reference position 6 corresponding to the entire standard-sized stadium 90a, a target photographing point Ptar1 and a permitted flight area 92c corresponding to the initial value of the target flight altitude Htar are shown. Also, in Figure 24(B), in addition to the photographing reference position 6 corresponding to the entire stadium 90b, which is larger than the standard size, a target photographing point Ptar2 and a permitted flight area 92d corresponding to the initial value of the target flight altitude Htar are shown. Additionally, in Figure 24(B), the photographing reference position 6 and target photographing point Ptar1 of Figure 24(A) are indicated by dashed lines, etc.

[0149] 24(A) and 24(B), the position coordinates of the photographing reference position 6 do not match between the stadiums 90a and 90b. In other words, the position coordinates of the photographing reference position 6 are changed according to the ratio of the area sizes of the stadiums 90a and 90b. The position coordinates of the photographing reference position are changed as follows.

[0150] That is, for the stadium 90a of the standard size in Fig. 24(A), for each shooting reference position (position 1 to position 15), a target shooting point Ptar (for example, Ptar1 in Fig. 24(A)) to be included in the shooting angle of view of the camera 231 and the length of the virtual vector extending from it are set in advance. The tip of the virtual vector becomes each shooting reference position.

[0151] The area size determination unit 444 determines the area size of the area to be photographed (for example, the area size of the stadium 90b in FIG. 24(B)). The area size here can be, for example, the area X of the area to be photographed, or the length of the short side W, or the length of the long side L, or a combination of these. The presetting unit 440 then calculates the ratio between the area size of the current stadium 90b and the area size of the standard size stadium 90a.

[0152] Next, the preset unit 440 adjusts the position of the target photographing point Ptar based on the calculated ratio (e.g., Ptar1 → Ptar2). The preset unit 440 also multiplies the length of the virtual vector of the reference size by the ratio to adjust or calculate the length of the virtual vector for the current stadium 90b. The tip of the adjusted virtual vector, which has the adjusted position of the target photographing point Ptar as its starting point, is then set as the photographing reference position (horizontal position and altitude). The zoom amount can also be changed according to the ratio.

[0153] In the above embodiment, the flight modes shown in FIG. 14 are given. However, this is not limiting, for example, if attention is paid to the fact that the movement conditions or shooting conditions of the drone 20 during shooting are switched depending on the area size of the shooting target area, or that not only the entire facility but also a part of it can be set as the shooting target area. For example, one or more of the flight modes shown in FIG. 14 may be omitted. Alternatively, another shooting mode may be used in addition to or instead of the flight modes shown in FIG. 14.

[0154] In the above embodiment, initial values ​​for the shooting direction, zoom amount, and the like are set in advance for each reference position (camera position) of the drone 20 (in other words, for each position coordinate) ( FIG. 15 ). However, other methods may be used, for example, by switching the movement conditions or shooting conditions of the drone 20 during shooting depending on the area size of the target shooting area, or by focusing on the ability to set not only the entire facility but also a portion of it as the target shooting area. For example, only the range of the geofence in the vertical direction may be set in advance, and the shooting direction, zoom amount, and the like may be freely adjustable by the user through manual operation.

[0155] In the above embodiment, the angle of view adjustment process can be performed using the centroid 382 of the athlete images 381a, 381b, and 381b ( FIG. 18B ). However, this is not limited to this, for example, if attention is paid to positioning an object of interest that is considered to be the center of interest for the user or viewer at a predetermined position (e.g., the center) of the screen in the image acquired by the drone 20. For example, the angle of view may be adjusted based on the position or centroid of a specific one or more athletes (e.g., a favorite member). Such an object of interest may be set not only by the administrator of the server 40, but also by the user (or viewer) of the control device 30. Furthermore, the angle of view adjustment process may be omitted, for example, if attention is paid to switching the movement conditions or shooting conditions of the drone 20 during shooting depending on the area size of the shooting target area, or to the fact that not only the entire facility but also part of it can be set as the shooting target area.

[0156] In the flight stability control of the above embodiment, the wind speed V and the satellite signal reception strength S are used to determine a situation in which the flight stability of the drone 20 may be reduced ( FIG. 20 ). However, this is not limited to this, as long as the focus is on determining a situation in which the flight stability of the drone 20 may be reduced. For example, only one of the wind speed V or the satellite signal reception strength S may be used. Alternatively, flight stability control may be performed using other external environmental factors (e.g., rainfall amount, snowfall amount). Furthermore, flight stability control may be omitted, for example, by focusing on the fact that the movement conditions or image capture conditions of the drone 20 during image capture are switched depending on the area size of the image capture target area, or that not only the entire facility but also part of it can be set as the image capture target area.

[0157] 10...Photography system 20...Drone (moving body for photographing) 90...Sports field 231...Photography camera 260, 260a, 260b, 260c...Geofence 381a, 381b, 381c...Athlete image (interest object) 382...Centroid 420...Calculation unit (moving body control device) 441...Photography target area setting unit S...Satellite signal reception strength THs...Reception strength threshold THv...Wind speed threshold V...Wind speed

Claims

1. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting system comprising, The aforementioned mobile control device includes a shooting target area setting unit that sets the shooting target area or its size by detecting a shooting target area which is all or part of a single stadium, or by receiving user input for the shooting target area. The aforementioned shooting target area setting unit is By detecting a target area that is all or part of the aforementioned stadium, or by accepting user input of the target area, the system distinguishes whether it is all or part of the aforementioned stadium and sets the target area or its size. For the same stadium, the conditions for the movement of the moving object or the conditions for the photograph are to be different depending on whether the area to be photographed is the entire stadium or only a part of the stadium. A photographic system characterized by the following features.

2. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting system comprising, The aforementioned mobile control device includes a shooting target area setting unit that sets the shooting target area or its size by detecting a shooting target area which is all or part of a single stadium, or by receiving user input for the shooting target area. Furthermore, the mobile body control device switches at least one of the movement conditions or shooting conditions of the mobile body during shooting according to the area size of the area to be photographed. Furthermore, the mobile control device is By setting multiple objects of interest for the viewer of the image acquired by the aforementioned camera, The positions of the multiple objects of interest are identified within the image acquired by the aforementioned camera. The centroid is calculated from the positions of the aforementioned multiple objects of interest, The field of view of the camera is controlled so that the centroid is located at a predetermined position in the acquired image. A photographic system characterized by the following features.

3. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting system comprising, The aforementioned mobile control device includes a shooting target area setting unit that sets the shooting target area or its size by detecting a shooting target area which is all or part of a single stadium, or by receiving user input for the shooting target area. Furthermore, the mobile device control switches the permissible range of shooting direction or the target shooting direction during shooting according to the area size of the area to be photographed. A photographic system characterized by the following features.

4. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting system comprising, The aforementioned mobile control device includes a shooting target area setting unit that sets the shooting target area or its size by detecting a shooting target area which is all or part of a single stadium, or by receiving user input for the shooting target area. Furthermore, the mobile device control device switches the area of ​​the geofence in the vertical or horizontal direction during shooting according to the area size of the area to be photographed. A photographic system characterized by the following features.

5. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting system comprising, The aforementioned mobile control device includes a shooting target area setting unit that sets the shooting target area or its size by detecting a shooting target area which is all or part of a single stadium, or by receiving user input for the shooting target area. Furthermore, the mobile device control system switches the range of permitted flight altitudes during shooting according to the area size of the area to be photographed, and allows manual control of the altitude of the mobile device for shooting within the range of permitted flight altitudes. A photographic system characterized by the following features.

6. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting system comprising, The aforementioned mobile control device includes a shooting target area setting unit that sets the shooting target area or its size by detecting a shooting target area which is all or part of a single stadium, or by receiving user input for the shooting target area. Furthermore, the mobile control device is A plurality of geofences are set up to divide the area in which the aforementioned mobile body can fly, The timing of switching the geofence when switching between flyable areas is adjusted according to the area size of the target area for photography. A photographic system characterized by the following features.

7. In the imaging system described in claim 6, As multiple flight modes for photography, An overhead shooting mode that captures the entire target area from above, A partial shooting mode that captures a portion of the aforementioned target area. It is possible to switch between these two modes. When switching flight modes, the mobile control device sets a combined geofence that combines the geofence used in the previous flight mode and the geofence used in the new flight mode. A photographic system characterized by the following features.

8. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting system comprising, The aforementioned mobile control device includes a shooting target area setting unit that sets the shooting target area or its size by detecting a shooting target area which is all or part of a single stadium, or by receiving user input for the shooting target area. Furthermore, the mobile body control device switches at least one of the movement conditions or shooting conditions of the mobile body during shooting according to the area size of the area to be photographed. Furthermore, the mobile control device changes the target flight altitude, or the upper or lower flight altitude limit, based on the detection information of the external environment. A photographic system characterized by the following features.

9. In the imaging system according to claim 8, The detected external environmental information is wind speed. The mobile control device lowers the target flight altitude or the upper limit flight altitude when the wind speed exceeds the wind speed threshold. A photographic system characterized by the following features.

10. In the imaging system according to claim 8, The external environment detection information includes the received signal strength of the satellite signal, the number of satellites receiving the satellite signal, or the elevation angle of the satellite receiving the satellite signal. The mobile device control unit raises at least one of the target flight altitude, the upper flight altitude, or the lower flight altitude if the received signal strength falls below a received signal strength threshold, the number of satellites falls below a predetermined number, or the elevation angle of the satellites falls below a predetermined angle. A photographic system characterized by the following features.

11. A mobile camera equipped with a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting method using a shooting system equipped with, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. The aforementioned shooting target area setting unit is By detecting a target area that is all or part of a single stadium, or by accepting user input of the target area, the system distinguishes whether it is all or part of the stadium and sets the target area or its size. For the same stadium, the conditions for the movement of the moving object or the conditions for the photograph are to be different depending on whether the area to be photographed is the entire stadium or only a part of the stadium. A photographic method characterized by the following:

12. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting method using a shooting system equipped with, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile body control device switches at least one of the movement conditions or shooting conditions of the mobile body during shooting according to the area size of the area to be photographed. Furthermore, the mobile control device is By setting multiple objects of interest for the viewer of the image acquired by the aforementioned camera, The positions of the multiple objects of interest are identified within the image acquired by the aforementioned camera. The centroid is calculated from the positions of the aforementioned multiple objects of interest, The field of view of the camera is controlled so that the centroid is located at a predetermined position in the acquired image. A photographic method characterized by the following:

13. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting method using a shooting system equipped with, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. The mobile device controls the permissible range of shooting directions or the target shooting direction during shooting, according to the area size of the area to be photographed. A photographic method characterized by the following:

14. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting method using a shooting system equipped with, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. The mobile device control device switches the area of ​​the geofence in the vertical or horizontal direction during shooting according to the area size of the area to be photographed. A photographic method characterized by the following:

15. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting method using a shooting system equipped with, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. The mobile device control system switches the range of permitted flight altitudes during shooting according to the area size of the area to be photographed, and allows manual control of the altitude of the mobile device for shooting within the range of permitted flight altitudes. A photographic method characterized by the following:

16. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting method using a shooting system equipped with, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile control device is A plurality of geofences are set up to divide the area in which the aforementioned mobile body can fly, The timing of switching the geofence when switching between flyable areas is adjusted according to the area size of the target area for photography. A photographic method characterized by the following:

17. A mobile camera having a camera for taking pictures, A mobile device control unit that controls the movement of the mobile device and the shooting process during shooting. A shooting method using a shooting system equipped with, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile body control device switches at least one of the movement conditions or shooting conditions of the mobile body during shooting according to the area size of the area to be photographed. Furthermore, the mobile control device changes the target flight altitude, or the upper or lower flight altitude limit, based on the detection information of the external environment. A photographic method characterized by the following:

18. A mobile device control system that controls the movement and shooting of a mobile camera equipped with a camera, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. The aforementioned shooting target area setting unit is By detecting a target area that is all or part of a single stadium, or by accepting user input of the target area, the system distinguishes whether it is all or part of the stadium and sets the target area or its size. For the same stadium, the conditions for the movement of the moving object or the conditions for the photograph are to be different depending on whether the area to be photographed is the entire stadium or only a part of the stadium. A mobile device control device characterized by the following features.

19. A mobile device control for controlling the movement and shooting of a mobile shooting body having a shooting camera, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile body control device switches at least one of the movement conditions or shooting conditions of the mobile body during shooting according to the area size of the area to be photographed. Furthermore, the mobile control device is By setting multiple objects of interest for the viewer of the image acquired by the aforementioned camera, The positions of the multiple objects of interest are identified within the image acquired by the aforementioned camera. The centroid is calculated from the positions of the aforementioned multiple objects of interest, The field of view of the camera is controlled so that the centroid is located at a predetermined position in the acquired image. A mobile device control device characterized by the following features.

20. A mobile device control for controlling the movement and shooting of a mobile shooting body having a shooting camera, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile device control switches the permissible range of shooting direction or the target shooting direction during shooting according to the area size of the area to be photographed. A mobile device control device characterized by the following features.

21. A mobile device control device that controls the movement and shooting of a mobile shooting device having a shooting camera, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile device control device switches the area of ​​the geofence in the vertical or horizontal direction during shooting according to the area size of the area to be photographed. A mobile device control device characterized by the following features.

22. A mobile device control for controlling the movement and shooting of a mobile shooting body having a shooting camera, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile device control system switches the range of permitted flight altitudes during shooting according to the area size of the area to be photographed, and allows manual control of the altitude of the mobile device for shooting within the range of permitted flight altitudes. A mobile device control device characterized by the following features.

23. A mobile device control for controlling the movement and shooting of a mobile shooting body having a shooting camera, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile control device is A plurality of geofences are set up to divide the area in which the aforementioned mobile body can fly, The timing of switching the geofence when switching between flyable areas is adjusted according to the area size of the target area for photography. A mobile device control device characterized by the following features.

24. A mobile device control for controlling the movement and shooting of a mobile shooting body having a shooting camera, The mobile device control unit includes a shooting target area setting unit that sets the shooting target area or its size by detecting the shooting target area or by receiving user input of the shooting target area. Furthermore, the mobile body control device switches at least one of the movement conditions or shooting conditions of the mobile body during shooting according to the area size of the area to be photographed. Furthermore, the mobile control device changes the target flight altitude, or the upper or lower flight altitude limit, based on the detection information of the external environment. A mobile device control device characterized by the following features.

25. A program executed by a mobile device control device that controls the movement and shooting of a mobile device equipped with a camera, In the aforementioned mobile body control device, The steps include: setting the target area or its size by detecting the target area or by accepting user input of the target area; A step of switching at least one of the movement conditions or shooting conditions of the moving object during shooting, according to the area size of the area to be photographed. Execute, In the step of setting the target area for shooting or the size of that area, The steps include: detecting a target area that is all or part of a single stadium, or receiving user input of the target area, to distinguish whether it is all or part of the single stadium, and setting the target area or its area size; For the same stadium, the steps include differentiating at least one of the movement conditions or shooting conditions of the moving object during shooting depending on whether the area to be photographed is the entire stadium or only a part of the stadium. A program characterized by executing [this].

26. A program executed by a mobile device control device that controls the movement and shooting of a mobile device having a camera for shooting, In the aforementioned mobile body control device, The steps include: setting the target area or its size by detecting the target area or by accepting user input of the target area; A step of switching at least one of the movement conditions or shooting conditions of the moving object during shooting, according to the area size of the area to be photographed. Execute, Furthermore, in the mobile control device, The steps include setting up multiple objects of interest for viewers of the images acquired by the aforementioned camera, The steps include identifying the positions of the multiple objects of interest within the image acquired by the aforementioned camera, A step of calculating the centroid from the positions of the aforementioned multiple objects of interest, The steps include controlling the field of view of the camera so that the centroid is located at a predetermined position in the acquired image, and A program characterized by executing [this].

27. ​​A program executed by a mobile device control device that controls the movement and shooting of a mobile device having a camera for shooting, In the aforementioned mobile body control device, The steps include: setting the target area or its size by detecting the target area or by accepting user input of the target area; A step of switching the permissible range of shooting direction or the target shooting direction during shooting according to the area size of the area to be photographed. A program characterized by executing [this].

28. A program executed by a mobile device control device that controls the movement and shooting of a mobile device having a camera for shooting, In the aforementioned mobile body control device, The steps include: setting the target area or its size by detecting the target area or by accepting user input of the target area; The steps include switching the area of ​​the geofence in the vertical or horizontal direction during shooting according to the area size of the area to be photographed, and A program characterized by executing [this].

29. A program executed by a mobile device control device that controls the movement and shooting of a mobile device having a camera for shooting, In the aforementioned mobile body control device, The steps include: setting the target area or its size by detecting the target area or by accepting user input of the target area; The steps include switching the range of permitted flight altitudes during filming according to the area size of the area to be filmed, and allowing manual control of the altitude of the filming mobile device within the range of permitted flight altitudes. A program characterized by executing [this].

30. A program executed by a mobile device control device that controls the movement and shooting of a mobile device having a camera for shooting, In the aforementioned mobile body control device, The steps include: setting the target area or its size by detecting the target area or by accepting user input of the target area; The steps include setting up multiple geofences that divide the area in which the mobile body can fly, The steps include switching the timing of the geofence switching when switching the flyable area according to the area size of the area to be photographed, and A program characterized by executing [this].

31. A program executed by a mobile device control device that controls the movement and shooting of a mobile device having a camera for shooting, In the aforementioned mobile body control device, The steps include: setting the target area or its size by detecting the target area or by accepting user input of the target area; A step of switching at least one of the movement conditions or shooting conditions of the moving object during shooting, according to the area size of the area to be photographed, The steps include changing the target flight altitude, or the upper or lower flight altitude limit, based on detection information of the external environment. A program characterized by executing [this].