Imaging system, imaging method, imaging control device and program

Abstract

[Problem] To provide an imaging system, an imaging method, an imaging control device and a program with which convenience or operability at the time of capturing an image of an imaging target can be improved. [Solution] An imaging system (10) comprises: a mobile camera (231) that is capable of adjusting an imaging direction, including a panning direction and / or a tilting direction, manually by a user or automatically; and an imaging control device (460) that controls image capture executed by of the camera. The imaging control device sets, according to the position of the mobile camera, a permissible range of an imaging direction or of a zoom amount, and controls the imaging direction or the zoom amount using the permissible range, or notifies the user of the permissible range of the imaging direction or the zoom amount.

Description

Photographing system, photographing method, photographing control device and program

[0001] The present invention relates to an imaging system, an imaging method, an imaging control device, and a program.

[0002] The objective of Patent Document 1 is to provide an unmanned helicopter that can obtain images that follow the direction of travel and movement of the aircraft while moving toward a target, and that can grasp the status of the aircraft and its surroundings even if data communication is interrupted (

[0009] , Abstract).To achieve this objective, Patent Document 1 (Abstract) provides an unmanned helicopter equipped with a camera mounted on the bottom of the aircraft, which has four modes: normal mode, neutral mode, pan-following mode, and emergency mode.

[0003] In normal mode, both the pan (horizontal) and tilt (vertical) directions are maintained at the shooting target. In neutral mode, the pan (horizontal) direction is maintained in the nose direction, and the tilt (vertical) direction is maintained at a predetermined angle diagonally downward. In pan-following mode, the pan (horizontal) direction is maintained at an arbitrary constant angle relative to the nose direction, and the tilt (vertical) direction can be controlled arbitrarily by remote control. In emergency mode, the pan (horizontal) direction is maintained in the nose direction, and the tilt (vertical) direction is maintained at a predetermined angle diagonally downward.

[0004] Japanese Patent Application Laid-Open No. 2006-264573

[0005] As described above, Patent Document 1 (Abstract) discloses four modes for use in an unmanned helicopter equipped with a camera mounted on the underside of the fuselage: normal mode, neutral mode, pan-following mode, and emergency mode. However, the four shooting modes disclosed in Patent Document 1 leave room for improvement in terms of convenience and operability when photographing a subject.

[0006] For example, when a user uses an unmanned aerial vehicle (aerial vehicle) equipped with a camera whose pan (horizontal) and tilt (vertical) directions are controlled by the user to film a competition being held at a stadium, the user may end up filming a direction they did not intend (e.g., outside the stadium). In the normal mode of Patent Document 1, both the pan (horizontal) and tilt (vertical) directions are maintained at the shooting target (point of interest), making it impossible to handle situations where the subject of the image changes frequently. Furthermore, in the pan tracking mode, the pan (horizontal) direction is maintained at an arbitrary fixed angle relative to the nose direction, and the tilt (vertical) direction can be controlled arbitrarily by remote control. However, when filming a competition, constantly pointing the unmanned aerial vehicle at the subject of the image may impair convenience or operability.

[0007] The above-mentioned problems are not limited to cases where a camera is mounted on an airborne vehicle, but also apply to cases where a camera is mounted on a ground vehicle.

[0008] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a photography system, photography method, photography control device, and program that can improve the convenience or operability when photographing a subject.

[0009] An imaging system according to one aspect of the present invention comprises a mobile camera whose imaging direction, including at least one of a pan direction and a tilt direction, can be adjusted manually or automatically by a user, and an imaging control device that controls imaging by the camera, wherein the imaging control device changes the allowable range of the imaging direction according to the camera position, which is either an absolute position indicating the latitude and longitude of the camera or a relative position of the mobile camera with respect to a two-dimensional or three-dimensional imaging target area or imaging target point, and limits the imaging direction using the allowable range, or notifies the user of the allowable range of the imaging direction.

[0010] According to the present invention, the shooting direction is restricted using the allowable shooting direction range changed according to the camera position (absolute position or relative position) of a mobile camera, or the allowable shooting direction range is notified to the user. This makes it easier to shoot using the allowable shooting direction range according to the camera position in a configuration in which the camera is mobile and the shooting direction is adjusted manually or automatically, thereby improving convenience and operability. Therefore, for example, it is possible to prevent the camera from being pointed in an undesirable shooting direction.

[0011] The imaging control device may set the allowable range of the imaging direction so that at least a part of the imaging target area is included in the angle of view of the camera, thereby preventing a user or the imaging control device from erroneously changing the imaging direction so as to point the camera outside the imaging target area.

[0012] The photography system may include an unmanned aerial vehicle that flies in the air. The camera may be mounted on the unmanned aerial vehicle. This prevents the camera from being pointed in an undesirable direction during aerial photography, thereby improving convenience and operability.

[0013] The imaging control device may limit the imaging direction by at least one of the attitude of the unmanned aerial vehicle and a camera actuator, thereby enabling the imaging direction to be limited in a simple manner.

[0014] The camera may be capable of manually or automatically adjusting the zoom amount in addition to the shooting direction. The shooting control device may set an allowable range of the zoom amount depending on the camera position, the tilt direction, or the pan direction. The shooting control device may also limit the zoom amount using the allowable range of the zoom amount, or notify the user of the allowable range of the zoom amount. This makes it possible to prevent an undesirable camera magnification from being used in a configuration in which a camera is mobile and the zoom amount is adjusted manually or automatically, thereby improving convenience and operability.

[0015] The photographing system may include a ground vehicle that moves on the ground. The camera may be provided on the ground vehicle. This prevents the camera from being pointed in an undesirable photographing direction during ground photographing, thereby improving convenience and operability.

[0016] In a manual shooting direction operation mode in which the shooting direction is manually operated, the shooting control device may limit the shooting direction operated by the user using the allowable range of the shooting direction set according to the camera position. This prevents the camera from being pointed in an undesirable shooting direction when the user manually operates the shooting direction, thereby improving convenience or operability.

[0017] In a zoom amount manual operation mode in which the zoom amount is manually operated, the photography control device may limit the zoom amount operated by the user using the allowable range of the zoom amount set according to the camera position, thereby preventing an undesirable camera magnification when the user manually operates the zoom amount, thereby improving convenience and operability.

[0018] The imaging control device may be capable of setting a dolly imaging mode, which allows only horizontal linear movement while maintaining a constant camera altitude and enables dolly imaging in which at least the pan direction is fixed at a target pan direction. Furthermore, the imaging control device may set an allowable linear movement range, which is a range within which horizontal linear movement of the camera is allowed in the dolly imaging mode. Furthermore, the imaging control device may restrict the camera from moving outside the allowable linear movement range when at least the pan direction is the target pan direction. This makes it possible to prevent the camera's imaging range from, for example, going outside the imaging target area while the pan direction remains in the target pan direction in the dolly imaging mode.

[0019] In the dolly shooting mode, when a predetermined operation is performed while the camera is at one end of the linear movement allowable range, the shooting control device may allow the camera to move in the linear movement direction beyond the linear movement allowable range. The predetermined operation may be a first manual pan direction operation for moving the pan direction closer to the center of the linear movement allowable range, or a manual zoom out operation for zooming out the camera. In this way, in the dolly shooting mode, when an operation (a first manual pan direction operation or a manual zoom out operation) is performed to keep the camera's shooting range, for example, within a stadium or an event venue, the camera is allowed to move in the linear movement direction beyond the linear movement allowable range. This improves operability in dolly shooting.

[0020] In the dolly shooting mode, when the camera moves in the linear movement direction beyond the linear movement allowable range, the shooting control device may perform a first automatic pan direction operation to bring the pan direction closer to the center of the linear movement allowable range, or an automatic zoom out operation to zoom out the camera. This makes it possible to prevent the shooting range of the camera from going outside the shooting target area in the dolly shooting mode, for example.

[0021] In the dolly shooting mode, when the camera is outside the allowable linear movement range and a second manual pan direction operation is performed to bring the pan direction closer to the target pan direction, the shooting control device may perform an automatic camera position operation to bring the camera closer to the allowable linear movement range. This makes it possible to prevent the camera's shooting range from going outside the target shooting area when the pan direction of the camera is brought closer to the target pan direction during the first automatic pan direction operation.

[0022] In the dolly shooting mode, when the camera returns from outside the allowable linear movement range toward the allowable linear movement range, the shooting control device may perform a second automatic pan direction operation to bring the pan direction closer to the target pan direction. This makes it possible to automatically adjust the pan direction to relax the restriction when the shooting direction, which has been restricted by exceeding the allowable horizontal movement range, is returned to the allowable linear movement range.

[0023] When the camera enters the target area, if the target area is not within the camera's shooting range, the shooting control device may automatically adjust the shooting direction, the camera position, or the zoom amount so that the target area is within the camera's shooting range. This makes it possible to smoothly start shooting the target area.

[0024] The photography system may further include a notification unit that notifies a user of the allowable range of the photography direction or the zoom amount. When the photography direction or the zoom amount is restricted, the notification unit may notify the user of the restriction by displaying an image on a display unit or the like, lighting or flashing a light, or using sound or vibration. This makes it possible to notify the user that the photography direction or the zoom amount is restricted.

[0025] The notification means may display information relating to the shooting direction or the allowable range of the zoom amount corresponding to the camera position, thereby making it easier for the user to operate the camera.

[0026] The photography control device may set the allowable range of the photography direction or the zoom amount based on at least any of the following information: a ratio of the photography target area to an area outside the photography target area in the image photographed by the camera, outer edge information of the photography target area input by the user, an area value or a ratio of the photography target area in the photographed image, or an area value or a ratio of the area outside the photography target area in the photographed image. This eliminates the need for the user to input the allowable ranges of the photography direction and the zoom amount for all points, making it possible to simplify the setting of the allowable ranges.

[0027] Another aspect of the present invention is a photography method using a photography system including a mobile camera whose photography direction, including at least one of a pan direction and a tilt direction, can be adjusted manually or automatically by a user, and a photography control device that controls photography by the camera, wherein the photography control device changes the allowable range of the photography direction according to the camera position, which is either an absolute position indicating the latitude and longitude of the camera or a relative position of the mobile camera with respect to a two-dimensional or three-dimensional photography target area or photography target point, and limits the photography direction using the allowable range, or notifies the user of the allowable range of the photography direction.

[0028] A further aspect of the present invention is a photography control device that controls photography by a mobile camera whose photography direction, including at least one of the pan direction and tilt direction, can be adjusted manually or automatically by a user, and the photography control device is characterized in that it changes the allowable range of the photography direction according to the camera position, which is either an absolute position indicating the latitude and longitude of the camera or a relative position of the mobile camera with respect to a two-dimensional or three-dimensional photography target area or photography target point, and limits the photography direction using the allowable range, or notifies the user of the allowable range of the photography direction.

[0029] A further aspect of the present invention provides a program executed by an imaging control device that controls imaging with a mobile camera whose imaging direction, including at least one of the pan direction and tilt direction, can be adjusted manually or automatically by a user, and is characterized in that the imaging control device executes the following steps: changing the allowable range of the imaging direction according to the camera position, which is either an absolute position indicating the latitude and longitude of the camera or a relative position of the mobile camera with respect to a two-dimensional or three-dimensional imaging target area or imaging target point; and limiting the imaging direction using the allowable range, or notifying the user of the allowable range of the imaging direction.

[0030] According to the present invention, it is possible to improve the convenience and operability when photographing a subject.

[0031] 1 is an overall configuration diagram of a photography system according to an embodiment of the present invention. FIG. 1 is a functional configuration diagram of a drone according to the embodiment. FIG. 2 is an external perspective view showing a simplified outline of the drone according to the embodiment. FIG. 3 is a functional configuration diagram of a control device according to the embodiment. FIG. 4 is an external front view showing a simplified outline of the control device according to the embodiment. FIG. 5 is a functional configuration diagram of a server according to the embodiment. FIG. 6 is a flowchart showing an overall flow of aerial photography control according to the embodiment. FIG. 7 is a diagram showing how the drone moves from a takeoff point to a photography target area and from the photography target area to a landing point according to the embodiment. FIG. 8 is a diagram showing an example of a photography target area candidate list used to select a photography target area according to the embodiment. FIG. 9 is a first explanatory diagram related to setting a flight geofence according to the embodiment. FIG. 10 is a second explanatory diagram related to setting a flight geofence according to the embodiment. FIG. 11 is a first explanatory diagram related to setting photography restrictions according to the embodiment. FIG. 12 is a second explanatory diagram related to setting photography restrictions according to the embodiment. FIG. 13 is a third explanatory diagram related to setting photography restrictions according to the embodiment. FIG. 14 is a fourth explanatory diagram related to setting photography restrictions according to the embodiment. FIG. 15 is a flowchart of operation restriction control in normal photography mode according to the embodiment. FIG. 16 is a flowchart of operation restriction control in first dolly photography mode according to the embodiment. FIG. 17 is a first explanatory diagram of operation restriction control in the first dolly photography mode according to the embodiment. FIG. 2 is a second explanatory diagram of operation restriction control in the first dolly shooting mode in the embodiment. FIG. 3 is a third explanatory diagram of operation restriction control in the first dolly shooting mode in the embodiment. FIG. 4 is a flowchart of operation restriction control in the second dolly shooting mode in the embodiment. FIG. 5 is an overall configuration diagram of an imaging system according to a first modified example of the present invention. FIG. 6 is an overall configuration diagram of an imaging system according to a second modified example of the present invention. FIG. 7 is an overall configuration diagram of an imaging system according to a third modified example of the present invention. FIG. 8 is an explanatory diagram regarding restrictions on the imaging direction in the first modified example of the present invention. FIG. 9 is an explanatory diagram regarding restrictions on the imaging direction in the second modified example of the present invention.

[0032] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted. Furthermore, the embodiments described below are merely examples, and other known elements or alternative means may be adopted depending on the application, purpose, scale, etc.

[0033] <A. One Embodiment> [A-1. Configuration] (A-1-1. Overall Configuration) Fig. 1 is a diagram showing the overall configuration of an imaging system 10 (hereinafter also referred to as "system 10") according to one embodiment of the present invention. The system 10 uses a drone 20 to capture aerial images of a predetermined area where people are likely to be active, such as a competition being held at a stadium 90 (Fig. 8) or an event being held at an event venue. As shown in Fig. 1, in addition to the drone 20, the system 10 also includes a control device 30 that enables a pilot to operate the drone 20, and a server 40 that manages the flight and imaging of the drone 20.

[0034] In this embodiment, a "competition" refers to a competition of skills or abilities, and includes soccer and various other sports. It is not limited to matches, but may also include practice sessions. A stadium is a place where a competition is held, and may include not only the area inside the court defined by the lines shown in FIG. 8 , but also the area outside the court. The present invention is not limited to competitions and events, and can be applied to an imaging system that captures an area to be photographed for any purpose.

[0035] The drone 20 and the control device 30 are connected to each other via wireless communication (which may include communication via a base station 52). The control device 30 and the server 40 are connected to each other via a communication network 50 such as the Internet. The drone 20 acquires satellite signals from an artificial satellite 54 to determine its own location. The configuration of the system 10 is not limited to that shown in FIG. 1; for example, the configurations shown in FIGS. 22 to 24 can also be used (details will be described later).

[0036] (A-1-2. Drone 20) (A-1-2-1. Overview of the Drone 20) Fig. 2 is a functional configuration diagram of the drone 20 of this embodiment. Fig. 3 is a simplified external perspective view of the drone 20 of this embodiment. As described above, the drone 20 takes aerial photographs of competitions taking place in the stadium 90 (Fig. 8), events taking place at event venues, etc. As shown in Fig. 2, the drone 20 has a drone sensor group 200, a communication unit 210, a flight mechanism 220, a photographing mechanism 230, and a drone control unit 240.

[0037] In this specification, the term "drone" refers to any flying object that has multiple rotors and the ability to autonomously control its attitude, regardless of the power source (electricity, prime mover, etc.), the control method (wireless or wired, fully autonomous flight type or partially manual flight type, etc.), and whether manned or unmanned. Drones are also sometimes called unmanned aerial vehicles (UAVs), flying objects, multicopters, RPASs (remote piloted aircraft systems), or UASs (unmanned aircraft systems), etc.

[0038] (A-1-2-2. Drone Sensor Group 200) The drone sensor group 200 includes various sensors arranged on the drone 20. Specifically, the drone sensor group 200 has a position measurement unit 201, a direction measurement unit 202, an altimeter 203, a speedometer 204, a gyro sensor 205, an obstacle sensor 206, etc. In addition to these, the drone sensor group 200 may also include various sensors that acquire information such as temperature, air pressure, wind speed, and acceleration.

[0039] The position measurement unit 201 receives signals from the artificial satellites 54 ( FIG. 1 ) and measures the position (absolute position) of the aircraft based on the signals. The position measurement unit 201 measures its current position using, for example, a Global Navigation Satellite System (GNSS) or a Global Positioning System (GPS), although this is not particularly limited. For example, a Real Time Kinematic - Global Navigation Satellite System (RTK-GNSS) can also be used as a method for measuring the position. The position information includes at least two-dimensional coordinate information (e.g., latitude and longitude) in a planar view, and preferably three-dimensional coordinate information including altitude information.

[0040] Furthermore, the base station 52, which provides information on the reference point of a fixed station used for relative positioning such as RTK, is wirelessly connected to the drone 20 and the control device 30, thereby enabling measurement of the position of the drone 20 with higher accuracy. When RTK measurement is performed using a virtual reference point method with a VRS (Virtual Reference Station), the base station 52 can be omitted, or the accuracy of the position coordinate estimation of the base station 52 or the drone 20 can be further improved.

[0041] The orientation measurement unit 202 measures the orientation (heading direction) of the drone 20. The orientation measurement unit 202 is composed of a geomagnetic sensor that measures the heading direction (heading direction) of the drone 20 by measuring geomagnetism, a compass, etc.

[0042] The altimeter 203 measures the altitude above ground (hereinafter referred to as "altitude H") as the distance from the ground below (vertically downward) the drone 20. The measured value of altitude H acquired by the altimeter 203 is also referred to as the measured altitude Hd. The speedometer 204 detects the flight speed of the drone 20. The gyro sensor 205 detects the angular velocity of the drone 20. The obstacle sensor 206 has multiple control cameras 207 (Figure 3) and measures the position, speed vector, etc. of people located below the drone 20, etc., based on the acquired images.

[0043] (A-1-2-3. Communication Unit 210) Communication unit 210 is capable of radio wave communication via communication network 50 (FIG. 1) and includes, for example, a radio wave communication module. Communication unit 210 is capable of communication with control device 30 and the like via communication network 50 (including wireless base station 52).

[0044] (A-1-2-4. Flight Mechanism 220) The flight mechanism 220 is a mechanism that causes the drone 20 to fly, and generates thrust in the airframe for lifting the drone 20 and moving it in a desired direction. As shown in Figures 2 and 3, the flight mechanism 220 has a plurality of rotors 221 and a plurality of rotor actuators 222. The rotor actuators 222 have, for example, electric motors.

[0045] The flight mechanism 220 may also be provided with a propeller guard (not shown) to prevent the propeller from interfering with obstacles. The number of rotors 221 constituting the flight mechanism 220 is not particularly limited, but may include, for example, one, two, four, six, or eight rotors. The rotor 221 may be composed of a single propeller, or may be composed of multiple propellers arranged coaxially. The number and shape of the blades of each propeller are not particularly limited.

[0046] (A-1-2-5. Filming mechanism 230) The filming mechanism 230 is a mechanism for capturing images of competitions at the stadium 90 (FIG. 8), events at the event venue, etc., and has a camera 231, a camera holding unit 232, and an image processing unit 233. As shown in FIG. 3, the camera 231 (imaging device) is disposed at the bottom of the main body of the drone 20, and outputs image data relating to peripheral images captured around the drone 20. The camera 231 is a video camera (color camera) that captures video. The video may include audio data acquired by a microphone (not shown). Additionally or alternatively, the camera 231 may also capture still images.

[0047] The orientation of the camera 231 (the attitude of the camera 231 relative to the main body of the drone 20) can be adjusted by a camera actuator (not shown) incorporated in the camera holding unit 232. Alternatively, the position of the camera 231 relative to the main body of the drone 20 may be fixed. The camera holding unit 132 may have a mechanism for suppressing transmission of shaking or vibration of the aircraft to the camera 231. The image processing unit 233 performs predetermined image processing on image data acquired by the camera 231. The image data acquired by the camera 231 can be transmitted to a storage unit of the drone 20 itself, the control device 30, the server 40, etc. Part or all of the image processing unit 233 may be positioned as part of the photography control unit 242 described below.

[0048] (A-1-2-6. Drone control unit 240) The drone control unit 240 controls the entire drone 20, including flying and photographing the drone 20. The drone control unit 240 includes an input / output unit, a calculation unit, and a storage unit, which are not shown. The drone control unit 240 includes a calculation device such as a CPU (Central Processing Unit) for executing information processing, and storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The drone control unit 240 includes a flight control unit 241 and a photography control unit 242.

[0049] The flight control unit 241 controls the flight of the drone 20 (control of the aircraft's attitude angle and flight operations from takeoff to flight and landing) via the flight mechanism 220. The flight control unit 241 has a processing unit, also called a flight controller. The processing unit may have one or more processors, such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit has access to memory. The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps. The memory may include, for example, a separable medium such as an SD card or RAM, or an external storage device. Various data acquired from the drone sensor group 200 may be directly transmitted to and stored in the memory. For example, video or still image data captured by the camera 231 may be recorded in an internal or external memory.

[0050] The processing unit includes a control module configured to control the airframe state of the drone 20. For example, the control module controls the flight mechanism 220 (thrust generating unit) of the drone 20 to adjust the spatial configuration, attitude angle, angular velocity, angular acceleration, angular jerk rate, and / or acceleration of the drone 20 having six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz). The control module can control one or more of the camera holder 232 and sensors.

[0051] The flight control unit 241 can control the flight of the drone 20 based on control signals from the control device 30 or based on a preset autonomous flight program. The flight control unit 241 can also control the flight of the drone 20 by controlling the flight mechanism 220 (thrust generation unit) based on various information such as the field to be photographed, flight permitted / prohibited areas, information on the corresponding flight geofences, map information including two-dimensional or three-dimensional map data, current position information of the drone 20, attitude information (heading information), speed information, and acceleration information, or any combination of these.

[0052] In this specification, "field to be photographed" (or area to be photographed) refers to a two-dimensional location to be photographed (for example, the outline defining the stadium 90 (for example, the frame consisting of the touchlines and goal lines of a soccer field)). "Flight permitted / prohibited area" refers to a three-dimensional unit of space in which flight of the drone 20 is permitted or prohibited. "Geofence" refers to a virtual boundary line, and in particular refers to a fence that marks the boundary between a flight permitted area in which a mobile object such as the drone 20 is permitted to fly or move and a flight prohibited area. Therefore, if a mobile object such as the drone 20 comes into contact with a geofence, flight or movement is restricted to prevent the aircraft from flying outside the flight permitted area.

[0053] The photography control unit 242 controls photography by the drone 20 via the photography mechanism 230.

[0054] (A-1-3. Control Device 30) (A-1-3-1. Overview of Control Device 30) FIG. 4 is a functional configuration diagram of the control device 30 of this embodiment. FIG. 5 is a simplified front view of the exterior of the control device 30 of this embodiment. The control device 30 is a portable information terminal that controls the drone 20 through operation by the operator and displays information received from the drone 20 (e.g., position, altitude, remaining battery level, camera footage, etc.). In this embodiment, the flight status (altitude, attitude, etc.) of the drone 20 is remotely controlled by the control device 30, but the drone 20 may also control it autonomously. In that case, when the operator transmits a flight command to the drone 20 via the control device 30, the drone 20 performs autonomous flight. However, manual operation may be possible during basic operations such as takeoff and return, and in emergencies.

[0055] The control device 30 has an input / output unit 300 and a communication unit 310. The input / output unit 300 performs various inputs by a user such as a pilot, outputs to the user, and inputs and outputs signals between the drone 20 and the server 40. The input / output unit 300 includes an operation input unit 320 and a display unit 330. The operation input unit 320 and the display unit 330 are connected to each other so that they can communicate with each other via wire or wirelessly. The control device 30 also includes a calculation unit such as a CPU for executing information processing, and storage devices such as RAM and ROM. Furthermore, the control device 30 of this embodiment receives and displays work instructions, etc. from the server 40.

[0056] (A-1-3-2. Communication Unit 310) The communication unit 310 is disposed in the same housing as the operation input unit 320 or the display unit 330, and has a communication function for wirelessly communicating with the drone 20 using Wi-Fi, 2.4 GHz, or a frequency band of 5.6 to 5.8 GHz. The communication unit 310 also has a wireless communication function that enables communication with the server 40 via an internet line 50 using a communication standard such as LTE (Long Term Evolution). In the system 10 ( FIG. 1 ) of this embodiment, the drone 20 communicates with the server 40 via the control device 30. Therefore, this system configuration is suitable for cases where the drone 20 and the control device 30 are located within a distance where direct wireless communication is possible (for example, visual flight by a pilot, etc.), but is not limited thereto.

[0057] (A-1-3-3. Operation Input Unit 320) The operation input unit 320 accepts various inputs from a user such as a pilot, and inputs operational commands such as flight direction and takeoff / landing when the pilot pilots the drone 20. For example, the operation input unit 320 accepts input operations that instruct three-dimensional flight operations of the drone 20, including takeoff, landing, ascent, descent, rotation, forward movement, backward movement, and left / right movement. As shown in FIG. 4 , the operation input unit 320 of this embodiment includes a movement input unit 321, a drone attitude input unit 322, a camera attitude input unit 323, a camera zoom input unit 324, a flight mode switching input unit 325, and a power input unit 326. As shown in FIG. 5 , the operation input unit 320 as hardware includes a left input stick 327L, a right input stick 327R, a left two-way switch button 328L, a right two-way switch button 328R, and a menu button 329.

[0058] The movement input unit 321 is an input unit that allows the pilot to move the drone 20 in the up / down, left / right, and forward / backward directions, and is composed of a right input stick 327R and a left input stick 327L. That is, when the right input stick 327R is moved upward (toward the rear), the drone 20 rises, and when the right input stick 327R is moved downward (toward the viewer), the drone 20 descends. When the right input stick 327R is moved to the right, the drone 20 moves right, and when the right input stick 327R is moved to the left, the drone 20 moves left. When the left input stick 327L is moved upward (toward the rear), the drone 20 moves forward, and when the left input stick 327L is moved downward (toward the viewer), the drone 20 moves backward.

[0059] The drone attitude input unit 322 is an input unit that allows the pilot to control the attitude of the drone 20, and is configured with the left input stick 327L. The attitude of the drone 20 referred to here includes yaw rotation. That is, when the left input stick 327L is moved to the right, the drone 20 turns right, and when the left input stick 327L is moved to the left, the drone 20 turns left.

[0060] The camera attitude input unit 323 is an input unit for manipulating the attitude of the photographing camera 231, and is composed of a right bidirectional switch button 328R and a right input stick 327R. That is, when the right side of the right bidirectional switch button 328R is pressed, the camera 231 moves to the right. When the left side of the right bidirectional switch button 328R is pressed, the camera 231 moves to the left. Furthermore, when the right input stick 327R is pressed downward while pressing the right bidirectional switch button 328R, the camera 231 moves downward. When the right input stick 327R is pressed upward while pressing the right bidirectional switch button 328R, the camera 231 moves upward.

[0061] The camera zoom input unit 323 is an input unit for operating the zoom of the photographing camera 231, and is configured with a left two-way switch button 328L. That is, when the right side of the left two-way switch button 328L is pressed, the camera 231 zooms in. When the left side of the left two-way switch button 328L is pressed, the camera 231 zooms out.

[0062] The flight mode switching unit 325 is an input unit for switching the flight mode of the drone 20, and is composed of a menu button 329 and a left input stick 327L. That is, the flight mode is switched by pressing the menu button 329 and selecting from the menu displayed using the left input stick 327L. The power input unit 326 is a part for turning the power of the control device 30 on and off, and is composed of a mechanical switch or the like.

[0063] (A-1-3-4. Display Unit 330) The display unit 330 displays to the pilot status information of the drone 20 obtained from the drone 20 or the server 40. The display unit 330 may be configured as a touch panel or LCD monitor that is integrally built into the control device 30, or may be configured as a display device such as an LCD monitor, tablet terminal, or smartphone that is connected to the control device 30 by wire or wirelessly. The display unit 330 can display images related to various information such as the field to be photographed, permitted / prohibited flight areas, flight geofence, map information, current position information of the drone 20, attitude information (directional information), speed information, acceleration information, and remaining battery power.

[0064] (A-1-4. Server 40) (A-1-4-1. Overview of the Server 40) Fig. 6 is a functional configuration diagram of the server 40 of this embodiment. The server 40 manages or controls the flight and photography of the drone 20. As shown in Fig. 6, the server 40 has an input / output unit 400, a communication unit 410, a calculation unit 420, and a storage unit 430. The input / output unit 400 is a part for inputting or outputting various types of information (image output, audio output). The communication unit 410 has a modem or the like (not shown), and is capable of communicating with the drone 20, the control device 30, etc. via the communication network 50.

[0065] The calculation unit 420 includes a CPU and operates by executing a program stored in the storage unit 430. Some of the functions executed by the calculation unit 420 can be realized using a logic IC (Integrated Circuit). Some of the programs of the calculation unit 420 can also be configured using hardware (circuit components).

[0066] The storage unit 430 stores programs and data used by the calculation unit 420 and includes a RAM. The RAM can be a volatile memory such as a register, or a non-volatile memory such as a hard disk or flash memory. The storage unit 430 may also include a ROM in addition to the RAM.

[0067] The server 40 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0068] (A-1-4-2. Calculation unit 420) As shown in Fig. 6, the calculation unit 420 has a pre-setting unit 440, a flight control unit 450, and an imaging control unit 460. The pre-setting unit 440 performs pre-settings for the flight and imaging of the drone 20. The flight control unit 450 controls the flight of the drone 20. The imaging control unit 460 controls imaging by the drone 20.

[0069] As shown in FIG. 6 , the presetting unit 440 includes a shooting target field selection unit 441, a geofence setting unit 442, and a shooting restriction setting unit 443. The shooting target field selection unit 441 is a unit for selecting a shooting target field, for example, by a user's operation of the menu button 329 and the left input stick 327L. The geofence setting unit 442 is a unit for setting a flight geofence 91 ( FIG. 18 ), for example, by a user's operation of the menu button 329 and the left input stick 327L. The shooting restriction setting unit 443 is a unit for selecting a limit range for the shooting direction and zoom amount, for example, by a user's operation of the menu button 329 and the left input stick 327L. The shooting restriction setting unit 443 includes a shooting direction restriction setting unit 444 and a zoom restriction setting unit 445. The shooting direction restriction setting unit 444 is a unit for restricting the shooting direction, and the zoom restriction setting unit 445 is a unit for restricting the zoom amount.

[0070] The flight control unit 450 controls the flight of the drone 20. The flight here includes, in addition to flight during shooting, a takeoff operation, movement from a takeoff point Pto ( FIG. 8 ) to a target shooting location (e.g., the stadium 90), movement from the target shooting location to a target landing point Pln, and a landing operation.

[0071] The photography control unit 460 has a photography mode setting unit 461, a drone state acquisition unit 462, a photography direction restriction unit 463, a zoom restriction unit 464, a flight restriction unit 465, and an automatic photography control unit 466. The photography mode setting unit 461 is a unit that sets the photography mode. The drone state acquisition unit 462 is a unit that acquires the state of the drone 20. The photography direction restriction unit 463 is a unit that restricts the photography direction of the camera 231. The zoom restriction unit 464 is a unit that restricts the zoom amount of the camera 231. The flight restriction unit 465 is a unit that restricts the flight of the drone 20. The automatic photography control unit 466 is a unit that controls automatic photography by the camera 231.

[0072] [A-2. Control] (A-2-1. Overall Flow) Next, various types of control in this embodiment will be described. FIG. 7 is a flowchart showing the overall flow of aerial photography control in this embodiment. FIG. 8 is a diagram showing how the drone 20 moves from the takeoff point Pto to the photography target field (e.g., the stadium 90) and from the photography target field to the target landing point Pln in this embodiment. As described above, in this embodiment, the drone 20 photographs a competition taking place in the stadium 90 (FIG. 8), an event taking place at an event venue, etc. from the air. As shown in FIG. 7, the control in this embodiment can be roughly divided into control for pre-setting before aerial photography (pre-setting control in step S10) and control when aerial photography is performed (aerial photography control in step S20).

[0073] (A-2-2. Pre-setting control (S10 in FIG. 7)) (A-2-2-1. Selection of a field to be photographed) In step S101 in FIG. 7, the server 40 (photography control unit 460) selects a field to be photographed based on input from the user of the control device 30. FIG. 9 is a diagram showing an example of a field list 60 used to select a field to be photographed in this embodiment. The field list 60 includes information such as a field name, a field identification number (field ID), and an address. The user selects a field to be photographed from the field list 60.

[0074] (A-2-2-2. Setting of the flight geofence 91) In step S102 of FIG. 7, the server 40 (geofence setting unit 442) sets the flight geofence 91 (FIG. 18) based on input from the user of the control device 30. FIG. 10 is a first explanatory diagram regarding the setting of the flight geofence in this embodiment. FIG. 10 shows a screen 61a as an example of a screen used when setting the flight geofence. The screen 61a includes an image 600 of the stadium 90 (hereinafter also referred to as the "stadium image 600"), geofence images 601a and 601b (hereinafter also referred to as the "geofence images 601a and 601b"), and an obstacle image 602 (hereinafter also referred to as the "obstacle image 602").

[0075] The geofence image 601a is a geofence image registered in the storage unit 430 of the server 40, and the geofence image 601b is a geofence image modified by the user. Specifically, the geofence image 601b is enlarged on the left and bottom sides of the geofence image 601a (see arrow 603a). The geofence image 601b is also enlarged on the right side of the geofence image 601a (see arrow 603b). However, because an obstacle image 602 is present in the lower right of the stadium image 600 (and because an obstacle is present in the lower right of the actual stadium 90), enlargement of the lower right side of the geofence image 601b is restricted. The arrows 603a and 603b are intended to indicate how the geofence image 601a changes to the geofence image 601b through user manipulation, and are not actually displayed on the screen 61a.

[0076] A flight geofence (hereinafter also referred to as a "geofence") indicates a virtual boundary line, and in particular indicates a fence that marks the boundary line between a permitted flight area where a mobile object such as a drone 20 is permitted to fly or move and a prohibited flight area. Therefore, when a mobile object such as a drone 20 comes into contact with a geofence, the flight or movement is restricted so that the aircraft does not fly outside the permitted flight area.

[0077] In the example of Figure 10, the geofence is shown as a two-dimensional area with length and width, but in reality it is set as a three-dimensional area with height added to length and width (however, for control purposes it may be managed as a two-dimensional area).

[0078] 11 is a second explanatory diagram for setting a flight geofence in this embodiment. Fig. 11 shows a screen 61b as an example of a screen for setting a flight geofence. The screen 61b includes a stadium image 600 and geofence images 601c, 601d, and 601e.

[0079] Geofence image 601c is a geofence image registered in the storage unit 430 of the server 40, while geofence images 601d and 601e are geofence images modified or added by the user. Specifically, geofence image 601d is obtained by enlarging the right, upper, and lower sides of geofence image 601c (see arrow 603c). Geofence image 601e is obtained by copying geofence image 601c and adding it to the left side of the stadium image 600 (see arrow 603d). Note that arrows 603c and 603d are intended to indicate how geofence image 601c changes to geofence images 601d and 601e through user operation, and are not actually displayed on the screen 61b.

[0080] 10 and 11, the geofence settings are changed, but it is also possible for the user to simply check the geofence (geofence image 601a or 601c) registered in the storage unit 430 of the server 40 and use it as is. Alternatively, it is also possible to omit checking and changing the geofence (after performing step S101, immediately proceed to step S103, and check or change the geofence by a separate operation as necessary) on the premise that the geofence registered in the storage unit 430 of the server 40 is used as is.

[0081] (A-2-2-3. Setting Photography Restrictions) In step S103 of FIG. 7 , the server 40 (photography restriction setting unit 443) sets photography restrictions based on input from the user of the control device 30 and the position (camera position) of the drone 20 (camera 231). Photography restrictions in this embodiment set allowable ranges (hereinafter also referred to as "restriction ranges") for each camera position for both the photography direction and zoom amount of the camera 231. The restriction range may be a numerical range restriction set using three pieces of angle information (angles around the X, Y, and Z axes) that define the camera orientation for each position coordinate of the camera 231, or a restriction range may be set using a three-dimensional vector for the camera direction for each position coordinate of the camera 231. As another method, as in Visual SLAM (Simultaneous Localization and Mapping) or spatial coordinate surveying using images, a photography-prohibited area may first be set on a map (in spatial coordinates), and then restrictions may be set on the photography direction and movement if the spatial coordinates of the photography-prohibited area fall within the angle of view during photography. When setting the allowable range of the zoom amount, it is possible to set it not only for each camera position but also for each direction of the camera 231 (tilt direction or pan direction).

[0082] FIG. 12 is a first explanatory diagram regarding the setting of photography restrictions in this embodiment. FIG. 12 shows screen 62 as an example of a screen used when setting photography restrictions. Screen 62 includes a stadium image 600 and four drone images 621a, 621b, 621c, and 621d. Drone images 621a to 621d correspond to multiple points P1, P2, P3, and P4 for which photography restriction information is input when setting photography restrictions in this embodiment. That is, for each of points P1 to P4, a restricted range of photography direction and a restricted range of zoom amount are input. Point P1 is the center of the upper goal line. Point P2 is the intersection of the goal line and touch line on the upper right. Point P3 is the intersection of the center line and touch line on the right. Point P4 is the position of the center mark.

[0083] When the user selects one of drone images 621a to 621d on display unit 330, the display transitions to a setting screen for the location corresponding to the selected drone image (for example, screen 63 in FIG. 13). Note that although omitted in FIG. 12, screen 62 can also include an operation guidance message.

[0084] FIG. 13 is a second explanatory diagram regarding the setting of photography restrictions in this embodiment. FIG. 13 shows screen 63 as an example of a screen for setting photography restrictions. Screen 63 includes a stadium image 600, a drone image 621a, a direction of travel image 631a, a pan direction restriction range image 632, and a virtual captured image 634. Drone image 621a in FIG. 13 is the same as drone image 621a in FIG. 12 and corresponds to drone image 621a being selected from drone images 621a to 621d in FIG. 12. In other words, if drone image 621b is selected in FIG. 12, drone image 621b of point P2 is displayed on screen 63 instead of drone image 621a of point P1.

[0085] The traveling direction image 631a indicates the traveling direction of the drone image 621a. In the example of Fig. 13, the traveling direction image 631a corresponding to point P1 is pointing left, but as shown in Fig. 15, the traveling direction image 631b corresponding to point P1 may be pointing downward. The pan direction limit range image 632 indicates the limited range of the pan direction when the drone image 621a is at point P1 and the traveling direction is the direction indicated by the traveling direction image 631a (leftward in Fig. 13).

[0086] The virtual captured image 634 is a virtual captured image (simulation image) that shows the imaging range of the camera 231 when the drone image 621a is at point P1 and the direction of travel is the direction indicated by the travel direction image 631a (leftward in FIG. 13 ). Pan adjustment buttons 635L and 635R are superimposed on the virtual captured image 634.

[0087] When the user presses the pan adjustment button 635L, the limited range image 632 rotates to the left (or counterclockwise in FIG. 13 ), and when the user presses the pan adjustment button 635R, the limited range image 632 rotates to the right (or clockwise in FIG. 13 ) (note that the size of the limited range image 632 does not change). The virtual captured image 634 also changes as the limited range image 632 rotates. When the user presses the first enter button (not shown), the adjustment of the pan direction limited range image 632 is completed, and the screen transitions to the next setting screen (e.g., screen 64 in FIG. 14 ). Note that the screen 63 may also have another button, such as a first back button (not shown) for returning to the screen 62 in FIG. 12 . Although not shown in FIG. 13 , the screen 63 may also include an operation guidance message.

[0088] The limit range for the direction of travel opposite to the direction of travel image 631a (to the right in FIG. 13) is calculated using the limit range for the direction of travel indicated by the direction of travel image 631a. In other words, the limit range is calculated and set so as to be symmetrical with respect to the imaginary line connecting the center mark and the penalty mark.

[0089] FIG. 14 is a third explanatory diagram regarding the setting of photography restrictions in this embodiment. FIG. 14 shows screen 64 as an example of a screen for setting photography restrictions. Screen 64 includes a stadium image 600, a drone image 621a, a direction of travel image 631a, and a virtual photographed image 644. Drone image 621a in FIG. 14 is the same as drone image 621a in FIGS. 12 and 13 and corresponds to the selection of drone image 621a from drone images 621a to 621d in FIG. 12. In other words, if drone image 621b is selected in FIG. 12, drone image 621b of point P2 is displayed on screen 64. As in FIG. 13, direction of travel image 631a indicates the direction of travel of drone image 621a.

[0090] The virtual captured image 644 is a virtual captured image (simulation image) that shows the imaging range of the camera 231 when the drone image 621a is at point P1 and the direction of travel is the direction indicated by the travel direction image 631a (leftward in FIG. 14 ). In other words, the virtual captured image 644 in FIG. 14 is the same as the virtual captured image 634 in FIG. 13 . However, tilt adjustment buttons 646U and 646D are superimposed on the virtual captured image 644.

[0091] When the user presses the tilt adjustment button 646U, the tilt direction of the camera 231 moves upward. When the user presses the tilt adjustment button 646D, the tilt direction of the camera 231 moves downward. When the user presses the first tilt direction lower limit determination button (not shown), adjustment of the lower limit value of the tilt direction is completed. When the user presses the first tilt direction upper limit determination button (not shown), adjustment of the upper limit value of the tilt direction is completed. Once the upper and lower limit values ​​of the tilt direction are set, the next setting screen (e.g., screen 65 in FIG. 15 ) is displayed. Note that screen 64 may also include another button, such as a second back button (not shown), for returning to screen 62 in FIG. 12 or screen 63 in FIG. 13 . Although not shown in FIG. 14 , screen 64 may also include an operation guidance message. Furthermore, the tilt adjustment buttons 646U and 646D of screen 64 may be included in screen 63, allowing the limit ranges of the pan and tilt directions to be set together on screen 63.

[0092] FIG. 15 is a fourth explanatory diagram regarding the setting of photography restrictions in this embodiment. FIG. 15 shows screen 65 as an example of a screen for setting photography restrictions. Screen 65 includes a stadium image 600, a drone image 621a, a direction of travel image 631b, and a virtual photographed image 654. Drone image 621a in FIG. 15 is the same as drone image 621a in FIGS. 12 to 14 and corresponds to the drone image 621a being selected from drone images 621a to 621d in FIG. 12. In other words, if drone image 621b is selected in FIG. 12, drone image 621b of point P2 is displayed on screen 65. Direction of travel image 631b indicates the direction of travel of drone image 621a. In the example of FIG. 15, the direction of travel image 631b corresponding to point P1 is pointing downward.

[0093] The virtual captured image 654 is a virtual captured image (simulation image) that shows the imaging range of the camera 231 when the drone image 621a is at point P1 and the direction of travel is the direction indicated by the travel direction image 631b (downward in FIG. 15 ). Tilt adjustment buttons 646U and 646D are superimposed on the virtual captured image 654, similar to the virtual captured image 644 in FIG. 14 .

[0094] In FIG. 15 , when the user presses tilt adjustment button 646U, the tilt direction of camera 231 moves upward. When the user presses tilt adjustment button 646D, the tilt direction of camera 231 moves downward. When the user presses second tilt direction lower limit determination button (not shown), adjustment of the lower limit of the tilt direction is completed. When the user presses second tilt direction upper limit determination button (not shown), adjustment of the upper limit of the tilt direction is completed. Once the upper and lower limit values ​​of the tilt direction are set, the screen transitions to the next setting screen (e.g., screen 62 in FIG. 12 ). Note that screen 65 may also have another button, such as a third back button (not shown) for returning to screen 62 in FIG. 12 or screen 64 in FIG. 14 . Although not shown in FIG. 15 , screen 65 may also include an operation guidance message.

[0095] 13 to 15, the pan direction limit range is input (FIG. 13) and the tilt direction limit range is input (FIGS. 14 and 15) for point P1. In addition to or instead of this, the zoom amount limit range of camera 231 may be input for point P1. The zoom amount limit range can be input, for example, by providing a zoom-up button and a zoom-down button (neither of which is shown) on each of screens 63 to 65 and setting them in accordance with the settings of the pan direction limit range and the tilt direction limit range.

[0096] For the other limit input points P2 to P4, the limit range of the shooting direction (pan direction and tilt direction) and / or the limit range of the zoom amount are input in the same manner as for point P1. More specifically, for point P2, the limit range of the pan direction and tilt direction is set for the drone image 621b when facing left and downward in FIGS. 12 and 13. For point P3, the limit range of the pan direction and tilt direction is set for the drone image 621c when facing left and downward in FIGS. 12 and 13. For point P4, the limit range of the pan direction and tilt direction is set for the drone image 621d when facing left and downward in FIGS. 12 and 13. Then, based on the limit range input for each of points P1 to P4, the server 40 calculates the limit range for each point other than points P1 to P4. In this case, the limit range is calculated, for example, using the following method.

[0097] For each point on the line (line 1-2) connecting point P1 and point P2, the limit range is calculated and set so that the difference between the limit range of point P1 and the limit range of point P2 changes gradually (e.g., linearly) with respect to the pan direction, tilt direction, and zoom amount (in other words, assuming proportional change). The same applies to each point on the line (line 2-3) connecting point P2 and point P3, each point on the line (line 3-4) connecting point P3 and point P4, and each point on the line (line 1-4) connecting point P1 and point P4. In this way, the limit range is calculated and set for each point on the rectangular frame line (lines 1-2, 2-3, 3-4, 1-4) connecting points P1 to P4. Note that instead of proportional change, the limit range for each point may be calculated and set assuming other changes (e.g., a change in a circular arc curve or a change in a quadratic curve).

[0098] Next, for each point on the line (line 1-4) connecting point P1 and point P4, and for each point on the line (line 2-3) connecting point P2 and point P3 (each point on a line perpendicular to lines 1-4 and 2-3), a limit range is calculated and set in the same way as for each point on line 1-2 or line 3-4. As a result, a limit range is calculated and set for each point inside the rectangle connecting points P1 to P4. In other words, a limit range is calculated and set for the upper right rectangle of the four rectangles obtained by dividing the soccer court in FIG. 12 into four.

[0099] The remaining three rectangles (lower right, upper left, and lower left) in the stadium 90 are calculated and set based on the limited range of the upper right rectangle. That is, for the lower right rectangle, the limited range in the pan direction is set to be line-symmetrical to the limited range of the upper right rectangle across the center line, and the same limited ranges as the limited range of the upper right rectangle are used for the tilt direction and zoom amount. For the upper left rectangle, the limited range in the pan direction is set to be line-symmetrical to the limited range of the upper right rectangle across the line connecting the two penalty marks (points where the ball is placed during a penalty kick), and the same limited ranges as the limited range of the upper right rectangle are used for the tilt direction and zoom amount. For the lower left rectangle, the limited range in the pan direction is set to be point-symmetrical to the limited range of the upper right rectangle across the center mark, and the same limited ranges as the limited range of the upper right rectangle are used for the tilt direction and zoom amount. The outer parts of the stadium 90 can also be calculated and set in a similar manner.

[0100] Although the above explanation is based on the assumption that the altitude of the drone 20 is constant, the limit ranges can also be set for different altitudes. For example, the limit range in the pan direction is the same regardless of the altitude, the limit range in the tilt direction is increased as the altitude increases, and the limit range in the zoom amount is decreased as the altitude increases.

[0101] (A-2-3. Control during aerial photography (S20 in FIG. 7)) (A-2-3-1. Overview of control during aerial photography) As described above, the control during aerial photography (S20 in FIG. 7) is control when performing aerial photography. In the control during aerial photography, in step S201 in FIG. 7, the server 40 executes control (outbound movement control) to move the drone 20 from the takeoff point Pto (FIG. 8) to the field to be photographed (stadium 90, etc.). In the following step S202, the server 40 executes control (photography target field photography control (or aerial photography control)) to perform aerial photography using the drone 20 in the field to be photographed. Finally, in step S203, the server 40 executes control (return movement control) to move the drone 20 from the field to be photographed to the target landing point Pln.

[0102] (A-2-3-2. Outward Movement Control (S201 in FIG. 7)) As described above, in outward movement control, the drone 20 is moved from the takeoff point Pto to the field to be photographed (such as the stadium 90). In the outward movement control of this embodiment, when the drone 20 (camera 231) enters the field to be photographed (or the area above the field to be photographed), if the field to be photographed is not within the shooting range of the camera 231, the shooting direction, camera position, or zoom amount is automatically adjusted so that the field to be photographed is within the shooting range of the camera 231. Note that this control is essentially the same even if it is positioned as part of the aerial photography control (S202).

[0103] (A-2-3-3. Aerial Photography Control (S202 in FIG. 7)) (A-2-3-3-1. Overview) As described above, in the aerial photography control of this embodiment, aerial photography is performed by the drone 20 in the field to be photographed. In aerial photography, the flight of the drone 20 and the photography direction and zoom amount of the camera 231 are basically controlled by manual operation by the user of the control device 30. Note that the pan direction (horizontal direction) of the photography direction of the camera 231 is controlled by control of the attitude of the drone 20 or the camera holding unit 232 (camera actuator). Furthermore, the tilt direction (vertical direction) and zoom amount of the camera 231 are controlled by control of the camera holding unit 232 (camera actuator). Furthermore, as already mentioned, in the aerial photography control of this embodiment, operation restriction control is executed to impose restrictions on user operation regarding the flight area of ​​the drone 20 and the photography direction and zoom amount of the camera 231.

[0104] (A-2-3-3-2. Aerial Photography Control in Normal Photography Mode (FIG. 16)) FIG. 16 is a flowchart of operation restriction control in normal photography mode in this embodiment. In step S2001, the server 40 (photography control unit 460) identifies the geofence 91 for normal photography mode. In step S2002, it is determined whether the flight position of the drone 20 (position of the camera 231) is within the geofence 91. If the flight position is within the geofence 91 (S2002: true), the process proceeds to step S2004. If the flight position is not within the geofence 91 (S2002: false), in step S2003, the server 40 (photography control unit 460) restricts the flight position and notifies the user via the display unit 330 that the flight position is being restricted. In addition to or instead of notifying the user via the display unit 330, the notification may be made by sound from a speaker (not shown) or vibration from a vibration generator (the same applies to steps S2006 and S2008 described below). In addition, cases where the flight position is not within the geofence 91 include, for example, when the user of the control device 30 makes a mistake in operation, or when the wind causes the drone 20 to move in a way that the user did not intend.

[0105] In step S2004, the server 40 (photography control unit 460) identifies the limited range of the photography direction and zoom amount of the camera 231 that corresponds to the current position (position of the camera 231) of the drone 20. The limited range identified here is the one that corresponds to the current position among those set in step S103 of FIG. 7 .

[0106] In step S2005, the server 40 (imaging control unit 460) determines whether the imaging direction is within the restricted range. If the imaging direction is within the restricted range (S2005: true), the process proceeds to step S2007. If the imaging direction is not within the restricted range (S2005: no), in step S2006, the server 40 restricts the imaging direction and notifies the user via the display unit 330 that the imaging direction is restricted.

[0107] In step S2007, the server 40 determines whether the zoom amount is within the limit range. If the zoom amount is within the limit range (S2007: YES), the current process ends and the process returns to step S2001. If the zoom amount is not within the limit range (S2007: NO), in step S2008, the server 40 limits the zoom amount and notifies the user via the display unit 330 that the zoom amount is being limited.

[0108] It is also possible to provide a function for manually canceling the restrictions on the shooting direction or zoom amount in order to accommodate cases where it is desired to shoot outside the field (for example, the spectator seats, benches, or manager). Such a manual cancellation function can be performed, for example, by operating the control device 30 (for example, by operating a button).

[0109] (A-2-3-3-3. Aerial photography control in first dolly photography mode (FIGS. 17 to 20)) FIG. 17 is a flowchart of operation restriction control in the first dolly photography mode in this embodiment. FIGS. 18, 19, and 20 are first to third explanatory diagrams of operation restriction control in the first dolly photography mode in this embodiment. Note that while FIGS. 10 to 15 show examples of the screen of the display unit 330 of the control device 30, FIGS. 18 to 20 show simplified views of the actual drone 20. In the first dolly photography mode, dolly photography is possible in which the altitude of the drone 20 (or camera 231) is kept constant, only linear movement in the horizontal direction is possible, and the pan direction is fixed at a target pan direction.

[0110] 18 , the direction of linear movement of the drone 20 in the horizontal direction is indicated by arrow 700, the shooting direction of the camera 231 is indicated by arrows 701a and 701b, and the shooting area of ​​the camera 231 is indicated by symbols 702a and 702b. In FIG. 18 , the shooting direction 701a and shooting area 702a in state S1 are the same as the shooting direction 701b and shooting area 702b in state S2. Furthermore, in the first dolly shooting mode, if the drone 20 (or the camera 231) cannot move further linearly due to the limited range of the shooting direction (and zoom amount) of the camera 231, the user of the control device 30 can change the shooting direction of the camera 231 to enable further linear movement (details will be described later).

[0111] In step S2011 of FIG. 17 , the server 40 identifies the geofence 91 ( FIGS. 18 to 20 ) for the first dolly photography mode. In step S2012, it is determined whether the flight position of the drone 20 (the position of the camera 231) is within the geofence 91. If the flight position is within the geofence 91 (S2012: true), the process proceeds to step S2014. If the flight position is not within the geofence 91 (S2012: false), in step S2013, the server 40 restricts the flight position so that it returns to within the geofence 91. Note that cases where the flight position is not within the geofence 91 include, for example, cases where the user of the control device 30 makes an incorrect operation or where the drone 20 moves in a manner unintended by the user due to wind.

[0112] In step S2014, the server 40 determines a range (permissible linear movement range) in which linear movement (dolly shooting) is permitted in accordance with the target shooting direction (target pan direction) and the current zoom amount. The permissible linear movement range is calculated from the limited range of the shooting direction and zoom amount set in step S103 of FIG. 7 (the permissible linear movement range may be calculated only from the shooting direction set in step S103 of FIG. 7). Note that if it is determined that shooting will be performed only within the geofence 91, the limited range may be set in step S103 only for each point within the geofence 91 (e.g., points P2 and P3).

[0113] In step S2015, the server 40 determines whether a user operation has requested that the drone 20 (camera 231) move outside the allowable linear movement range. This determination is made, for example, based on whether an operation command has been input when the drone 20 is located at the edge of the allowable linear movement range, requesting that the drone 20 move further back beyond the allowable linear movement range. If a request for movement outside the allowable linear movement range has been made (S2015: true), the process proceeds to step S2016. If a request for movement outside the allowable linear movement range has not been made (S2015: false), the process ends and returns to step S2011.

[0114] In step S2016, the server 40 determines whether a linear movement restriction release operation (hereinafter also referred to as a "restriction release operation") has been performed. The restriction release operation is performed, for example, when the drone 20 is located at the edge of the linear movement allowable range, by the user changing the shooting direction of the camera 231 toward the center of the linear movement allowable range. Note that if a restriction release operation is performed between steps S2015 and S2016, a notification that the linear movement restriction will be released may be displayed on the display unit 330 of the control device 30.

[0115] If the restriction release operation is not performed (S2016: true), in step S2017, the server 40 restricts linear movement. FIG. 18 shows how the position of the drone 20 (camera 231) changes from state S1 on the bottom to state S2 on the top in FIG. 18 due to a user operation. State S2 corresponds to the case where the drone 20 is located at the edge of the linear movement allowable range (S2015: true). Therefore, if the restriction release operation is not performed in state S2 (S2016: true), the position of the drone 20 (camera 231) cannot exceed the linear movement allowable range (i.e., cannot move above state S2) (S2017).

[0116] If the restriction release operation is being performed (S2016: False), in step S2018, the server 40 permits further linear movement beyond the linear movement allowable range. As described in detail below, the value of the permitted further linear movement can be changed depending on the extent of the restriction release operation (e.g., the amount of change in the shooting direction). Figure 19 shows a transition to state S3 when the drone 20 (camera 231) is in state S2 of Figure 18 and the user performs a restriction release operation (S2016: False), further linear movement is permitted (S2018), and the user actually performs further linear movement. Compared to state S2 of Figure 18, in state S3 of Figure 19, the shooting direction 701c of the camera 231 (and the orientation of the shooting area 702c) is toward the center of the linear movement allowable range.

[0117] When a further restriction release operation is performed in state S3 of Figure 19 (S2016: false), further linear movement is permitted (S2018). Figure 20 shows a state in which, when the drone 20 (camera 231) is in state S3 of Figure 19, the user performs a further restriction release operation (S2016: false), and further linear movement is permitted (S2018), resulting in a transition to state S4 as a result of the user actually performing further linear movement. Compared to state S3 of Figure 19, in state S4 of Figure 20, the imaging direction 701d of the camera 231 (and the orientation of the imaging area 702d) is toward the center of the linear movement allowable range.

[0118] (A-2-3-3-4. Aerial Photography Control in Second Dolly Photography Mode (FIG. 21)) FIG. 21 is a flowchart of operation restriction control in the second dolly photography mode in this embodiment. In the operation restriction control in the first dolly photography mode (FIG. 17), when a request for movement outside the allowable range of linear movement is made (S2015: true), the entity that performs the restriction release operation is the user of the control device 30. In contrast, in the operation restriction control in the second dolly photography mode (FIG. 21), when a request for movement outside the allowable range of linear movement is made (S2025: true), the entity that performs the restriction release operation is the server 40.

[0119] Steps S2021, S2022, S2023, S2024, and S2025 in FIG. 21 are similar to steps S2011, S2012, S2013, S2014, and S2015 in FIG. 17 . When a request for movement outside the allowable linear movement range is made (S2025: True), in step S2026, the server 40 automatically performs a restriction release operation according to the amount of further linear movement of the drone 20 requested by the user. That is, as the amount of further linear movement increases, the amount of correction of the shooting direction increases (see FIGS. 19 and 20 ). Therefore, even in the second dolly shooting mode ( FIG. 21 ), a change from state S2 in FIG. 18 to state S3 in FIG. 19 to state S4 in FIG. 20 is possible. However, as described above, it is the server 40, not the user, that corrects the shooting direction. Note that if the amount of further linear movement reaches the maximum value, further linear movement is not permitted.

[0120] Conversely, if the amount of further linear movement decreases when the drone 20 is outside the linear movement allowable range (in other words, when the user returns the position of the drone 20 to the linear movement allowable range side), the server 40 automatically reduces the amount of correction of the shooting direction to bring the pan direction shooting direction closer to the target pan direction. For example, in state S4 of FIG. 20 , when the user returns the position of the drone 20 to the linear movement allowable range side, the server 40 automatically returns the shooting direction to, for example, state S3 of FIG. 19 . Furthermore, in state S3 of FIG. 19 , when the user returns the position of the drone 20 to the linear movement allowable range side, the server 40 automatically returns the shooting direction to, for example, state S2 of FIG. 18 .

[0121] Furthermore, when the drone 20 is outside the linear movement allowable range (S2025: true) and the shooting direction is being automatically corrected (S2026), in step S2027, the server 40 determines whether the user has performed an operation (shooting direction return operation) to return the shooting direction (pan direction) to the target shooting direction (target pan direction). If the shooting direction return operation has been performed (S2027: true), in step S2028, the server 40 reduces the linear movement amount according to the amount of the shooting direction return operation. For example, when the shooting direction return operation is performed in state S4 of FIG. 20, the server 40 automatically returns the position of the drone 20 to, for example, state S3 of FIG. 19. Furthermore, when the shooting direction return operation is performed in state S3 of FIG. 19, the server 40 automatically returns the position of the drone 20 to, for example, state S2 of FIG. 18.

[0122] [A-3. Effects of this embodiment] According to this embodiment, the shooting direction is limited using the allowable range of shooting direction (S103 in FIG. 7, FIGS. 12 to 15) that is changed depending on the position of the drone 20 (camera position of the mobile camera 231), and the user is notified of the allowable range of shooting direction (S2006 in FIG. 16). This makes it easier to shoot using the allowable range of shooting direction depending on the camera position in a configuration in which the camera 231 is mobile and the shooting direction is adjusted manually or automatically, thereby improving convenience and operability. Therefore, for example, it is possible to prevent the camera 231 from being pointed in an undesirable shooting direction.

[0123] In this embodiment, the photography control unit 460 (photography control device) can set the allowable range of the photography direction so that at least a part of the stadium 90 (photography target area) is included in the angle of view of the camera 231 (S103 in FIG. 7, FIGS. 12 to 15). This makes it possible to prevent the user or the photography control unit 460 (photography control device) from accidentally changing the photography direction so that the camera 231 is pointed outside the stadium 90.

[0124] In this embodiment, the photography system 10 includes a drone 20 (unmanned aerial vehicle) that flies in the air (FIG. 1), and the camera 231 is provided on the drone 20 (FIGS. 2 and 3). This prevents the camera 231 from being pointed in an undesirable photography direction during aerial photography, thereby improving convenience and operability.

[0125] In this embodiment, the imaging control unit 460 (imaging control device) limits the imaging direction of the camera 231 by the attitude of the drone 20 (unmanned aerial vehicle) or the camera holding unit 232 (camera actuator) (S2006 in FIG. 16 ). This makes it possible to limit the imaging direction in a simple manner.

[0126] In this embodiment, the camera 231 can manually or automatically adjust the zoom amount as well as the shooting direction. The shooting control unit 460 (shooting control device) sets an allowable range for the zoom amount depending on the position (camera position) of the drone 20 or the tilt direction or pan direction (S103 in FIG. 7 ). The shooting control unit 460 (shooting control device) also limits the zoom amount using the allowable range and notifies the user of the allowable range of the zoom amount (S2008 in FIG. 16 ). This prevents the use of an undesirable camera magnification, improving convenience and operability in a configuration in which the camera 231 is mobile and the zoom amount is adjusted manually or automatically.

[0127] In this embodiment, in the normal shooting mode (manual shooting direction operation mode) in which the shooting direction is manually controlled, the shooting control unit 460 (shooting control device) limits the shooting direction operated by the user using the allowable range of shooting direction set according to the position (camera position) of the drone 20 (S2005, S2006 in FIG. 16 ). This prevents the camera 231 from being pointed in an undesirable shooting direction when the user manually controls the shooting direction, thereby improving convenience and operability.

[0128] In this embodiment, in the normal shooting mode (manual zoom amount operation mode) in which the zoom amount is manually operated, the shooting control unit 460 (shooting control device) limits the zoom amount operated by the user using the allowable range of the zoom amount set according to the position (camera position) of the drone 20 (S2007, S2008 in FIG. 16 ). This prevents an undesirable camera magnification when the user manually operates the zoom amount, thereby improving convenience and operability.

[0129] In this embodiment, the imaging control unit 460 (imaging control device) can set a first dolly imaging mode and a second dolly imaging mode, which allow only horizontal linear movement of the camera 231 while maintaining a constant altitude and fixing the pan direction at a target pan direction ( FIGS. 17 and 21 ). The imaging control unit 460 also sets a linear movement allowable range, which is the range within which horizontal linear movement of the camera 231 is allowed in the first dolly imaging mode and the second dolly imaging mode. If the pan direction is the target pan direction, the imaging control unit 460 restricts the camera 231 from further linear movement (movement outside the linear movement allowable range) ( S2017 in FIG. 17 , S2025 in FIG. 21 : false). This prevents the imaging range of the camera 231 from moving beyond the stadium 90 (outside the field to be photographed) while the pan direction remains the target pan direction in the first dolly imaging mode and the second dolly imaging mode.

[0130] In the first dolly shooting mode of this embodiment, when the camera 231 is at one end of its allowable linear movement range (S2015: true in FIG. 17 ), if a predetermined operation is performed (S2016: false), the shooting control unit 460 (shooting control device) allows the camera 231 to move further linearly (moving in a direction beyond the allowable linear movement range) (S2018). The predetermined operation is a first manual pan direction operation to move the pan direction closer to the center of the allowable linear movement range, or a manual zoom out operation to zoom out the camera 231 (S2016). As a result, in the first dolly shooting mode, if an operation (first manual pan direction operation or manual zoom out operation) is performed to maintain the camera 231's shooting range, for example, within the stadium 90 (within the field to be shot), the camera 231 is allowed to move further linearly (moving in a direction beyond the allowable linear movement range). This improves operability during dolly shooting.

[0131] In the second dolly shooting mode of this embodiment, when the camera 231 moves in a linear direction beyond the linear movement allowable range (S2025 in FIG. 21: true), the shooting control unit 460 (shooting control device) performs a first automatic pan direction operation to move the pan direction closer to the center of the linear movement allowable range (S2026 in FIG. 21, FIGS. 19 and 20). This makes it possible to prevent the shooting range of the camera 231 from going beyond the stadium 90 (outside the field to be shot) in the second dolly shooting mode, for example.

[0132] In the second dolly photography mode of this embodiment, when the camera 231 is outside the linear movement allowable range (S2025 in FIG. 21: true) and a second pan direction manual operation is performed to move the pan direction closer to the target pan direction (S2027: true), the photography control unit 460 (photography control device) performs an automatic camera position operation to move the camera 231 closer to the linear movement allowable range (S2028). This makes it possible to prevent the photography range of the camera 231 from going beyond the stadium 90 (outside the photography target area) when the pan direction of the camera 231 is moved closer to the target pan direction during the automatic restriction release operation (S2027).

[0133] In the second dolly shooting mode of this embodiment, when the amount of further movement of the camera 231 decreases (in other words, when the camera 231 returns from outside the linear movement allowable range toward the linear movement allowable range), the shooting control unit 460 (shooting control device) performs a second automatic pan direction operation to bring the pan direction closer to the target pan direction (S2026 in FIG. 21). This makes it possible to automatically adjust the pan direction to relax the restriction when the shooting direction, which was restricted by exceeding the linear movement allowable range, is returned to the linear movement allowable range.

[0134] In this embodiment, when the camera 231 enters the field to be photographed, if the field to be photographed is not within the photographing range of the camera 231, the photographing control unit 460 (photographing control device) automatically adjusts the photographing direction, camera position, or zoom amount so that the field to be photographed is within the photographing range of the camera 231 (S201 in FIG. 7 ). This allows for a smooth start of photographing the field to be photographed.

[0135] In this embodiment, the image capture system 10 further includes a display unit 330 (notification means) that notifies the user of the allowable ranges of the image capture direction and zoom amount (FIGS. 4 and 5). When the image capture direction or zoom amount is restricted, the display unit 330 notifies the user by displaying a message indicating that the restriction has been imposed (S2006 and S2008 in FIG. 16). This makes it possible to notify the user that the image capture direction or zoom amount is restricted.

[0136] In this embodiment, the photography system 10 further includes a display unit 330 (notification means) that notifies the user of the photography direction and the allowable range of zoom amount (FIGS. 4 and 5). The display unit 330 displays information about the photography direction or the allowable range of zoom amount corresponding to the position of the drone 20 (camera position) (S2006, S2008 in FIG. 16). This makes it easier for the user to operate the camera.

[0137] <B. Modifications> The present invention is not limited to the above-described embodiment, and various configurations can be adopted based on the contents of this specification. For example, the following configurations can be adopted.

[0138] [B-1. Configuration] (B-1-1. Photography System 10) The photography system 10 of the above embodiment was intended to photograph sports (soccer, tennis, etc.) taking place in a stadium 90 (FIG. 8). However, this is not limited to this, if attention is paid to limitations on the photography direction or zoom amount of the camera 231 using an allowable range that is changed depending on the position of the camera 231. For example, the photography subject is not limited to the sports mentioned above, and the system can also be applied to other events (concerts, ceremonies, etc.) where people gather.

[0139] In the above embodiment, the imaging system 10 has the configuration shown in Fig. 1. However, the present invention is not limited to this configuration, for example, when attention is focused on the limitation of the imaging direction or zoom amount of the camera 231 using the allowable range changed according to the position of the camera 231.

[0140] FIG. 22 is an overall configuration diagram of an imaging system 10A (hereinafter also referred to as "system 10A") according to a first modified example of the present invention. In the system 10A shown in FIG. 22, a drone 20, a control device 30, a server 40, and a base station 52 are connected to each other so as to be able to communicate with each other via a communication network 50 such as an internet line. Here, unlike the system 10 of FIG. 1, the drone 20 wirelessly communicates directly with the internet line 50 using a communication method such as LTE without going through the control device 30. Therefore, the drone 20, the control device 30, and the base station 52 do not need to communicate directly wirelessly; they only need to be connected to the internet line 50 in remote locations. Therefore, this system configuration is suitable for cases where the drone 20 and the control device 30 are located in remote locations (e.g., when a pilot remotely controls the drone).

[0141] Fig. 23 is an overall configuration diagram of an imaging system 10B (hereinafter also referred to as "system 10B") according to a second modified example of the present invention. In the example of system 10B shown in Fig. 23, a drone 20, a control device 30, a base station 52, and a server 40 are connected to each other so as to be able to communicate with each other via a communication network 50 such as an internet line, and the drone 20 and the base station 52 are connected to the internet line 50 by satellite communication via a satellite 54.

[0142] FIG. 24 is an overall configuration diagram of an imaging system 10C (hereinafter also referred to as "system 10C") according to a third modified example of the present invention. The example of system 10C shown in FIG. 24 illustrates a redundant system in which multiple servers 40 are connected to a single drone 20 via multiple internet lines 50. In this case, even if an abnormality occurs in the server 40 or the internet line 50, the operation of system 10C can be continued using the other redundant servers 40 and internet lines 50, thereby improving the reliability of system 10C. Note that, in FIGS. 22 and 23 , the drone 20 and the control device 30 can be controlled from a remote location, and thus the configuration is suitable for remote control. However, the present invention is not limited to this, and the configuration can also be applied to visual flight in which the pilot manually controls the drone 20 while watching it.

[0143] The device described in the above embodiment may be realized as a single device, or may be realized by a plurality of devices (e.g., cloud server 40, drone 20, control device 30) that are partly or entirely connected via a communication network 50. For example, each functional unit and memory unit of the server 40 may be implemented in a different server 40, drone 20, or control device 30 that is connected to each other via the communication network 50.

[0144] (B-1-2. Drone 20) In the above embodiment, basic behavior control of the drone 20 during shooting was performed based on operation of the control device 30 ( FIG. 5 ). However, for example, if attention is focused on limiting the shooting direction or zoom amount of the camera 231 using an allowable range that is changed depending on the position of the camera 231, the basic behavior control of the drone 20 is not limited to this, and may be automatic control (without manual operation) by the drone 20 itself or based on commands from the server 40.

[0145] In the above embodiment, the drone 20 is used as an example of a mobile body for aerial photography, but this is not limiting, for example, if attention is paid to the limitations on the photography direction or zoom amount of the camera 231 using an allowable range that is changed according to the position of the camera 231. The mobile body may be, for example, a camera system that is movable along a wire placed in the air within the stadium 90. Alternatively, the mobile body may be a mobile camera system (including one that moves on rails) that is placed on the ground.

[0146] (B-1-3. Control Device 30) The control device 30 in the above embodiment has the configuration shown in Figures 4 and 5. However, this is not limited to this, for example, if attention is paid to the restriction on the shooting direction or zoom amount of the camera 231 using an allowable range that is changed depending on the position of the camera 231. With regard to the operation input unit 320, for example, the number and arrangement of input sticks, and the number, shape, and arrangement of buttons can be changed as appropriate. Alternatively, the configuration realized by the operation input unit 320 in Figure 5 can be replaced with a touch panel.

[0147] The control input unit 320 may have a takeoff button and a landing button that instruct automatic takeoff and landing, a flight start button that instructs the aircraft to automatically fly to a specified position and hover there, a home button that performs a return operation to the starting position, a mode switch button that switches flight modes, etc.

[0148] [B-2. Control] The series of processes described in connection with the above embodiment may be realized using software, hardware, or a combination of software and hardware. A computer program for realizing each function of the server 40 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium on which such a computer program is stored may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. Furthermore, the above computer program may be distributed, for example, via a communication network 50 without using a recording medium.

[0149] The flowcharts used in the above embodiments do not necessarily have to be executed in the order shown in the drawings. Some processing steps may be executed in parallel. In addition, additional processing steps may be employed, and some processing steps may be omitted.

[0150] In the above embodiment, a limit range is set for each of the shooting direction and zoom amount of the camera 231 (FIGS. 7, 16, 17, and 21). However, for example, by focusing on the limit on the shooting direction or zoom amount of the camera 231 using an allowable range that is changed depending on the position of the camera 231, a configuration is also possible in which a limit range is set for only one of the shooting direction or zoom amount. Furthermore, regarding the shooting direction, limit ranges are set for both the pan direction (horizontal direction) and the tilt direction (vertical direction), but it is also possible to use only one of them.

[0151] The operation restriction control of the above embodiment allows for the normal shooting mode ( FIG. 16 ), the first dolly shooting mode ( FIG. 17 ), and the second dolly shooting mode ( FIG. 21 ). However, this is not limiting, for example, if attention is paid to the restriction of the shooting direction or zoom amount of the camera 231 using the allowable range changed according to the position of the camera 231. For example, it is also possible to use only one or two of the above three shooting modes. Alternatively, it is also possible to use another shooting mode in addition to or instead of the above three shooting modes.

[0152] In the above embodiment, the limit ranges for the shooting direction and zoom amount were set in advance for each position (camera position) of the drone 20 (in other words, for each position coordinate) (S103 in FIG. 7, and FIGS. 12 to 15). However, for example, another method may be used by focusing on the limit on the shooting direction or zoom amount of the camera 231 using an allowable range that is changed depending on the position of the camera 231. For example, the limit range for the shooting direction or zoom amount may be specified by determining the relationship between the image captured by the camera 231 and the prohibited area or a mark indicating the prohibited area for each position (camera position) of the drone 20.

[0153] 25 is an explanatory diagram regarding restrictions on the shooting direction according to a first modified example of the present invention. In the first modified example, in setting the shooting restrictions (S103 in FIG. 7), the user sets a shooting target area. The shooting target area is set, for example, in terms of its position relative to the shooting target field. As described above, the shooting target field may refer to a two-dimensional location to be photographed (for example, the outline defining the stadium 90 (for example, the frame consisting of the touchlines and goal lines of a soccer field)).

[0154] Next, when actually photographing using the drone 20 (S202 in FIG. 7 ), the server 40 estimates the photographing area of ​​the camera 231 based on the current position (camera position) and attitude of the drone 20 and the operating state and zoom amount of the camera holding unit 232 (camera actuator). Then, if the estimated photographing area of ​​the camera 231 is likely to include an area other than the photographing target area (e.g., a prohibited area), the server 40 restricts the photographing direction (pan direction and tilt direction) and zoom amount of the camera 231. In other words, the server 40 does not permit a photographing direction or zoom amount that would cause an area other than the photographing target area to enter the photographing area of ​​the camera. Alternatively, if the ratio of the area other than the photographing target area included in the estimated photographing area of ​​the camera 231 reaches or exceeds a predetermined value, the server 40 may restrict the photographing direction and zoom amount of the camera 231.

[0155] 25 shows a screen 66 as an example of a screen displayed on the display unit 330 of the control device 30 when actually taking pictures using the drone 20. The screen 66 includes a stadium image 600, a drone image 661, a direction of travel image 662, an estimated shooting area image 663, and a shot image 664. The shape of the drone image 661 in FIG. 25 is the same as the drone image 621a in FIG. 12, but it moves across the screen 66 according to the current position (camera position) of the drone 20.

[0156] The traveling direction image 662 has a shape similar to the traveling direction image 631a in Fig. 13, but indicates the actual traveling direction of the drone 20 (and the drone image 661). The estimated shooting area image 663 indicates the shooting area (particularly the shooting area in the pan direction) of the camera 231 estimated according to the position of the drone 20 (and the image 661).

[0157] The estimated photographed images 634, 644, and 654 in Figures 13 to 15 are estimated photographed images (simulation images). In contrast, the photographed image 664 in Figure 25 is an actual photographed image taken by the camera 231. A boundary image 665 indicating the boundary of the photographed area is superimposed on the photographed image 644. The boundary of the photographed area is determined, for example, by detecting features of the stadium 90 (goal lines, touch lines, etc.) from the photographed image 644 and determining the boundary from the position, angle, etc. of the feature in the photographed image 644.

[0158] The imaging control unit 460 of the server 40 then calculates the ratio R1 between the imaging target area and the other area in the captured image 664, and when the ratio R1 exceeds a predetermined threshold (first ratio threshold), determines that the imaging direction has exceeded the restricted range. Then, the imaging control unit 460 causes the display unit 330 of the control device 30 to display a message indicating that the imaging direction has exceeded the restricted range. In addition to or instead of the notification via the display unit 330, the notification may be made by sound from a speaker (not shown) or vibration from a vibration generator.

[0159] Note that the determination of whether the shooting direction exceeds the limit range may be made by a method other than using the ratio R1 between the shooting target area and other areas in the captured image 664. For example, the determination may be made based on the outer edge information of the shooting target area input by the user. With regard to the outer edge information of the shooting target area referred to here, for example, if the position of the outer edge of the shooting target area enters a predetermined area in the captured image 664, or if the outer edge of the shooting target area is present in the captured image 664, the shooting direction may be determined to have exceeded the limit range. Alternatively, the shooting direction may be determined to have exceeded the limit range if the area value or ratio of the shooting target area in the captured image 665 is below a predetermined threshold, or if the area value or ratio of the area other than the shooting target area in the captured image 665 exceeds a predetermined threshold.

[0160] 26 is an explanatory diagram of restrictions on the shooting direction according to a second modification of the present invention. In the second modification, in setting the shooting restrictions (S103 in FIG. 7), the user sets a no-shooting area. The no-shooting area is defined using, for example, a marker. The marker may be, for example, the outline (touch line, goal line) that defines the stadium 90, a corner flag, a goal post, a colored mark, or a three-dimensional code.

[0161] Next, when actually taking an image using the drone 20 (S202 in FIG. 7), the server 40 determines whether or not the mark is present in the image captured by the camera 231. If a mark is detected in the captured image, the server 40 restricts the shooting direction (pan direction and tilt direction) and zoom amount of the camera 231. In other words, if a mark is detected, further changes in the shooting direction and reductions in the zoom amount are not permitted.

[0162] 26 shows a screen 67 as an example of a screen displayed on the display unit 330 of the control device 30 when actually taking pictures using the drone 20. The screen 67 includes a stadium image 600, a drone image 661, a direction of travel image 662, an estimated photography area image 663, a prohibited photography area image 671, and a photographed image 672. The drone image 661, direction of travel image 662, and estimated photography area image 663 in FIG. 26 are the same as those in FIG. 25.

[0163] The photography-prohibited area image 671 indicates the photography-prohibited area. The photography-prohibited area is set by the user, for example, in step S103 of Fig. 7. Like the photographed image 664 of Fig. 25, the photographed image 672 of Fig. 26 is an image actually captured by the camera 231. A boundary image 673 indicating the boundary of the photography-prohibited area is superimposed on the photographed image 672. The boundary of the photography-prohibited area is determined, for example, by detecting features of the stadium 90 (goal lines, touch lines, etc.) from the photographed image 644 and determining the position, angle, etc. of the feature in the photographed image 672.

[0164] The shooting control unit 460 of the server 40 then calculates the ratio R2 of the prohibited area to the entire captured image 664, and when the ratio R2 exceeds a predetermined threshold (second ratio threshold), determines that the shooting direction has exceeded the restricted range. Then, the display unit 330 of the control device 30 displays a message indicating that the shooting direction has exceeded the restricted range. In addition to or instead of the notification via the display unit 330, the notification may be made by sound from a speaker (not shown) or vibration from a vibration generator. Note that, as in the first modified example ( FIG. 25 ), the determination of whether the shooting direction has exceeded the restricted range may be made by a method other than using the ratio R2 of the prohibited area to the entire captured image 664.

[0165] DESCRIPTION OF SYMBOLS 10, 10A, 10B, 10C... Photography system 20... Drone (unmanned aerial vehicle) 90... Stadium (area to be photographed) 231... Photography camera (mobile camera) 232... Camera holding unit (camera actuator) 330... Display unit (notification means) 460... Photography control unit (photography control device)

Claims

1. A mobile camera capable of manually or automatically adjusting a shooting direction including at least one of a pan direction and a tilt direction by a user, A shooting control device for controlling shooting of the camera A shooting system comprising: The shooting control device is The allowable range of the shooting direction is changed according to the absolute position indicating the latitude and longitude of the camera or the camera position as the relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, The shooting direction is restricted using the allowable range, or the allowable range of the shooting direction is notified to the user, Further, the shooting control device can set a dolly shooting mode in which only linear movement in the horizontal direction is possible while keeping the altitude of the camera constant and at least the pan direction is fixed at a target pan direction. Further, the shooting control device is In the dolly shooting mode, a linear movement allowable range that is the range in which linear movement of the camera in the horizontal direction is allowed is set, When at least the pan direction is the target pan direction, movement of the camera outside the linear movement allowable range is restricted A shooting system characterized by the above.

2. In the shooting system according to claim 1, In the dolly shooting mode, when a predetermined operation is performed when the camera is at one end of the linear movement allowable range, the shooting control device allows the camera to move in the direction of the linear movement beyond the linear movement allowable range, The predetermined operation is A first pan direction manual operation for bringing the pan direction closer to the center side of the linear movement allowable range, or A zoom-out manual operation for zooming out the camera A shooting system characterized by the above.

3. A mobile camera capable of manually or automatically adjusting a shooting direction including at least one of a pan direction and a tilt direction by a user, A shooting control device for controlling shooting of the camera A shooting system comprising: The shooting control device is The allowable range of the shooting direction is changed according to the absolute position indicating the latitude and longitude of the camera or the camera position as the relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, The shooting direction is restricted using the allowable range, or the allowable range of the shooting direction is notified to the user, Furthermore, the imaging control device can set a dolly shooting mode in which only linear movement in the horizontal direction is possible while keeping the altitude of the camera constant, and at least the pan direction is fixed in the target pan direction, and dolly shooting is possible. In the dolly shooting mode, when the camera moves in the direction of the linear movement beyond the allowable range of the linear movement, the imaging control device performs a first pan direction automatic operation for bringing the pan direction closer to the center side of the allowable range of the linear movement, or a zoom-out automatic operation for zooming out the camera to be performed. A shooting system characterized by the above.

4. In the shooting system according to claim 3, in the dolly shooting mode, when the imaging control device performs a second pan direction manual operation for bringing the pan direction closer to the target pan direction when the camera is outside the allowable range of the linear movement, the imaging control device performs a camera position automatic operation for bringing the camera closer to the allowable range of the linear movement. A shooting system characterized by the above.

5. In the shooting system according to claim 3, in the dolly shooting mode, when the camera returns from outside the allowable range of the linear movement toward the allowable range of the linear movement, the imaging control device performs a second pan direction automatic operation for bringing the pan direction closer to the target pan direction. A shooting system characterized by the above.

6. A mobile camera capable of manually or automatically adjusting a shooting direction including at least one of a pan direction or a tilt direction by a user, and an imaging control device for controlling shooting of the camera A shooting system comprising: The imaging control device changes the allowable range of the shooting direction according to the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, limits the shooting direction using the allowable range, or notifies the user of the allowable range of the shooting direction, Furthermore, when the camera enters the shooting target area, if the shooting target area does not enter the shooting range of the camera, the imaging control device automatically adjusts the shooting direction, the camera position, or the zoom amount of the camera so that the shooting target area enters the shooting range of the camera. A shooting system characterized by the above. **Claim 7**: A mobile camera capable of manually or automatically adjusting a shooting direction including at least one of a pan direction and a tilt direction by a user, a shooting control device for controlling shooting of the camera, and a shooting system comprising: The shooting control device: changes an allowable range of the shooting direction according to an absolute position indicating the latitude and longitude of the camera or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, limits the shooting direction using the allowable range, or notifies the user of the allowable range of the shooting direction, the camera is capable of manually or automatically adjusting a zoom amount in addition to the shooting direction, The shooting control device: sets an allowable range of the zoom amount according to the camera position, the tilt direction, or the pan direction, limits the zoom amount using the allowable range of the zoom amount, or notifies the user of the allowable range of the zoom amount, Furthermore, the shooting control device: the ratio between the shooting target area and the area outside the shooting target area in the captured image of the camera, the outer edge information of the shooting target area input by the user, the value or ratio of the area of the shooting target area in the captured image, or the value or ratio of the area of the area outside the shooting target area in the captured image sets the allowable range of the shooting direction or the zoom amount based on at least any one of the information. A shooting system characterized by the above. **Claim 8**: A mobile camera capable of manually or automatically adjusting a shooting direction including at least one of a pan direction and a tilt direction by a user, a shooting control device for controlling shooting of the camera, and a shooting method using the shooting system comprising: The shooting control device: changes an allowable range of the shooting direction according to an absolute position indicating the latitude and longitude of the camera or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, limits the shooting direction using the allowable range, or notifies the user of the allowable range of the shooting direction, Furthermore, the shooting control device can set a dolly shooting mode in which only linear movement in the horizontal direction is possible while keeping the altitude of the camera constant and at least the pan direction is fixed to a target pan direction, Furthermore, the shooting control device: Set a linear movement allowable range which is the range in which horizontal linear movement of the camera is allowed in the dolly shooting mode, When at least the pan direction is the target pan direction, restrict the camera from moving outside the linear movement allowable range A shooting method characterized by the above.

9. In the shooting method according to Claim 8, In the dolly shooting mode, when a predetermined operation is performed when the camera is at one end of the linear movement allowable range, the shooting control device allows the camera to move in the direction of the linear movement beyond the linear movement allowable range, The predetermined operation is A first pan direction manual operation for bringing the pan direction closer to the center side of the linear movement allowable range, or A zoom out manual operation for zooming out the camera A shooting method characterized by the above.

10. A mobile camera capable of manually or automatically adjusting a shooting direction including at least one of a pan direction or a tilt direction by a user, A shooting control device for controlling shooting of the camera A shooting method using a shooting system including: The shooting control device Changes the allowable range of the shooting direction according to the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, Restricts the shooting direction using the allowable range, or notifies the user of the allowable range of the shooting direction, Furthermore, the shooting control device can set a dolly shooting mode in which only horizontal linear movement is possible while keeping the altitude of the camera constant and at least the pan direction is fixed to the target pan direction, In the dolly shooting mode, when the camera moves in the direction of the linear movement beyond the linear movement allowable range, the shooting control device Performs a first pan direction automatic operation for bringing the pan direction closer to the center side of the linear movement allowable range, or Performs a zoom out automatic operation for zooming out the camera This is what is done A shooting method characterized by the above.

11. In the shooting method according to Claim 10, In the dolly shooting mode, when a second pan direction manual operation for bringing the pan direction closer to the target pan direction is performed when the camera is outside the linear movement allowable range, the shooting control device performs a camera position automatic operation for bringing the camera closer to the linear movement allowable range A photographing method characterized by the following.

12. In the photographing method according to Claim 10, in the dolly shot mode, when the camera returns from outside the linear movement allowable range toward the linear movement allowable range, the photographing control device performs a second pan direction automatic operation for bringing the pan direction closer to the target pan direction. A photographing method characterized by the following.

13. A photographing method using a mobile camera capable of manually or automatically adjusting a photographing direction including at least one of a pan direction and a tilt direction by a user, and a photographing control device for controlling photographing of the camera wherein the photographing control device changes an allowable range of the photographing direction according to an absolute position indicating the latitude and longitude of the camera or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional photographing target area or photographing target point, limits the photographing direction using the allowable range, or notifies the user of the allowable range of the photographing direction, and further, when the camera enters the photographing target area, if the photographing target area does not enter the photographing range of the camera, the photographing control device automatically adjusts the photographing direction, the camera position, or the zoom amount of the camera so that the photographing target area enters the photographing range of the camera. A photographing method characterized by the following.

14. A photographing method using a mobile camera capable of manually or automatically adjusting a photographing direction including at least one of a pan direction and a tilt direction by a user, and a photographing control device for controlling photographing of the camera wherein the photographing control device changes an allowable range of the photographing direction according to an absolute position indicating the latitude and longitude of the camera or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional photographing target area or photographing target point, limits the photographing direction using the allowable range, or notifies the user of the allowable range of the photographing direction, the camera is capable of manually or automatically adjusting a zoom amount in addition to the photographing direction, the photographing control device ​ ​ ​ ​ ​ The ratio between the subject area and the area outside the subject area in the captured image of the camera, The outer edge information of the subject area input by the user, The value or ratio of the area of the subject area in the captured image, or The value or ratio of the area of the area outside the subject area in the captured image Based on at least any one of the above information, set the allowable range of the shooting direction or the zoom amount A shooting method characterized by the above.

15. A shooting control device for controlling the shooting of a mobile camera whose shooting direction including at least one of the pan direction and the tilt direction can be manually or automatically adjusted by the user, The shooting control device, Change the allowable range of the shooting direction according to the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional subject area or subject point, Limit the shooting direction using the allowable range, or notify the user of the allowable range of the shooting direction, Furthermore, the shooting control device can set a dolly shooting mode in which only linear movement in the horizontal direction is possible while keeping the altitude of the camera constant and at least the pan direction is fixed at the target pan direction, Furthermore, the shooting control device, Set a linear movement allowable range which is the range in which linear movement of the camera in the horizontal direction is allowed in the dolly shooting mode, When at least the pan direction is the target pan direction, restrict the camera from moving outside the linear movement allowable range A shooting control device characterized by the above.

16. In the shooting control device according to Claim 15, In the dolly shooting mode, when a predetermined operation is performed when the camera is at one end of the linear movement allowable range, the shooting control device allows the camera to move in the direction of the linear movement beyond the linear movement allowable range, The predetermined operation, A first pan direction manual operation for bringing the pan direction closer to the center side of the linear movement allowable range, or A zoom out manual operation for zooming out the camera A shooting control device characterized by the above.

17. A shooting control device for controlling the shooting of a mobile camera whose shooting direction including at least one of the pan direction and the tilt direction can be manually or automatically adjusted by the user, The shooting control device, changing the allowable range of the shooting direction according to the absolute position indicating the latitude and longitude of the camera or the camera position as the relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, restricting the shooting direction using the allowable range, or notifying the user of the allowable range of the shooting direction, furthermore, the shooting control device can set a dolly shooting mode in which only linear movement in the horizontal direction is possible while keeping the altitude of the camera constant and at least the pan direction is fixed at a target pan direction, in the dolly shooting mode, when the camera moves in the direction of the linear movement beyond the linear movement allowable range, the shooting control device, performs a first pan direction automatic operation for bringing the pan direction closer to the center side of the linear movement allowable range, or a zoom-out automatic operation for zooming out the camera is performed A shooting control device characterized by the above.

18. In the shooting control device according to Claim 17, in the dolly shooting mode, when the shooting control device performs a second pan direction manual operation for bringing the pan direction closer to the target pan direction when the camera is outside the linear movement allowable range, the shooting control device performs a camera position automatic operation for bringing the camera closer to the linear movement allowable range A shooting control device characterized by the above.

19. In the shooting control device according to Claim 17, in the dolly shooting mode, when the camera returns from outside the linear movement allowable range toward the linear movement allowable range, the shooting control device performs a second pan direction automatic operation for bringing the pan direction closer to the target pan direction A shooting control device characterized by the above.

20. A shooting control device for controlling shooting of a mobile camera capable of manually or automatically adjusting a shooting direction including at least one of a pan direction and a tilt direction by a user, the shooting control device, changing the allowable range of the shooting direction according to the absolute position indicating the latitude and longitude of the camera or the camera position as the relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, restricting the shooting direction using the allowable range, or notifying the user of the allowable range of the shooting direction, Furthermore, when the camera enters the imaging target area, if the imaging target area is not within the imaging range of the camera, the imaging control device automatically adjusts the imaging direction, the camera position, or the zoom amount of the camera so that the imaging target area is within the imaging range of the camera. An imaging control device characterized by the above.

21. An imaging control device for controlling imaging of a mobile camera capable of manually or automatically adjusting an imaging direction including at least one of a pan direction and a tilt direction by a user, wherein the imaging control device, changes an allowable range of the imaging direction according to an absolute position indicating the latitude and longitude of the camera, or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional imaging target area or imaging target point, limits the imaging direction using the allowable range, or notifies the user of the allowable range of the imaging direction, the camera is capable of manually or automatically adjusting a zoom amount in addition to the imaging direction, wherein the imaging control device, sets an allowable range of the zoom amount according to the camera position, the tilt direction, or the pan direction, limits the zoom amount using the allowable range of the zoom amount, or notifies the user of the allowable range of the zoom amount, furthermore, the imaging control device, the ratio between the imaging target area and the area outside the imaging target area in the captured image of the camera, the outer edge information of the imaging target area input by the user, the value or ratio of the area of the imaging target area in the captured image, or the value or ratio of the area of the area outside the imaging target area in the captured image sets the allowable range of the imaging direction or the zoom amount based on at least any one of the information. An imaging control device characterized by the above.

22. A program executed by an imaging control device for controlling imaging of a mobile camera capable of manually or automatically adjusting an imaging direction including at least one of a pan direction and a tilt direction by a user, in the imaging control device, changing an allowable range of the imaging direction according to an absolute position indicating the latitude and longitude of the camera, or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional imaging target area or imaging target point; limiting the imaging direction using the allowable range, or notifying the user of the allowable range of the imaging direction are executed. The photographing control device can set a dolly shooting mode in which only linear movement in the horizontal direction is possible while keeping the altitude of the camera constant, and at least the pan direction is fixed in the target pan direction, Furthermore, in the photographing control device, a step of setting a linear movement allowable range which is a range in which linear movement of the camera in the horizontal direction is allowed in the dolly shooting mode; a step of restricting the camera from moving outside the linear movement allowable range when at least the pan direction is the target pan direction A program characterized by executing.

23. In the program according to claim 22, in the photographing control device, when a predetermined operation is performed when the camera is at one end of the linear movement allowable range in the dolly shooting mode, the camera is allowed to move in the direction of the linear movement beyond the linear movement allowable range Executing the step of The predetermined operation is a first pan direction manual operation for bringing the pan direction closer to the center side of the linear movement allowable range, or a zoom out manual operation for zooming out the camera A program characterized by being.

24. A program executed by a photographing control device that controls photographing of a mobile camera in which at least one of a pan direction or a tilt direction can be adjusted manually or automatically by a user, in the photographing control device, changing an allowable range of the photographing direction according to an absolute position indicating the latitude and longitude of the camera or a relative position of the mobile camera with respect to a two-dimensional or three-dimensional photographing target area or a photographing target point; limiting the photographing direction using the allowable range, or notifying the user of the allowable range of the photographing direction Executing, The photographing control device can set a dolly shooting mode in which only linear movement in the horizontal direction is possible while keeping the altitude of the camera constant, and at least the pan direction is fixed in the target pan direction, In the photographing control device, when the camera moves in the direction of the linear movement beyond the linear movement allowable range in the dolly shooting mode, a first pan direction automatic operation for bringing the pan direction closer to the center side of the linear movement allowable range, or a zoom out automatic operation for zooming out the camera Executing the step of A program characterized by that.

25. In the program according to Claim 24, in the shooting control device, when the camera is outside the linear movement allowable range during the dolly shooting mode, if a second pan direction manual operation for approaching the pan direction to the target pan direction is performed, a step of performing a camera position automatic operation for approaching the camera to the linear movement allowable range is executed A program characterized by the above.

26. In the program according to Claim 24, in the shooting control device, in the case of the dolly shooting mode, when the camera returns from outside the linear movement allowable range toward the linear movement allowable range, a second pan direction automatic operation for approaching the pan direction to the target pan direction is performed A program characterized by the above.

27. A program executed by a shooting control device that controls shooting of a mobile camera whose shooting direction including at least one of a pan direction and a tilt direction can be adjusted manually or automatically by a user, in the shooting control device, a step of changing an allowable range of the shooting direction according to an absolute position indicating the latitude and longitude of the camera or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or a shooting target point; a step of restricting the shooting direction using the allowable range, or a step of notifying the user of the allowable range of the shooting direction; when the camera enters the shooting target area, if the shooting target area does not enter the shooting range of the camera, automatically adjusting the shooting direction, the camera position, or the zoom amount of the camera so that the shooting target area enters the shooting range of the camera A program characterized by executing the above.

28. A program executed by a shooting control device that controls shooting of a mobile camera whose shooting direction including at least one of a pan direction and a tilt direction can be adjusted manually or automatically by a user, in the shooting control device, a step of changing an allowable range of the shooting direction according to an absolute position indicating the latitude and longitude of the camera or a camera position as a relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or a shooting target point; a step of restricting the shooting direction using the allowable range, or a step of notifying the user of the allowable range of the shooting direction is executed, The camera can adjust the zoom amount manually or automatically in addition to the shooting direction. Furthermore, in the shooting control device, a step of setting an allowable range of the zoom amount according to the camera position, the tilt direction, or the pan direction; a step of limiting the zoom amount using the allowable range of the zoom amount, or a step of notifying the user of the allowable range of the zoom amount are executed, Furthermore, in the shooting control device, a ratio between the shooting target area and the area outside the shooting target area in the captured image of the camera, outer edge information of the shooting target area input by the user, a value or ratio of the area of the shooting target area in the captured image, or a value or ratio of the area of the area outside the shooting target area in the captured image Based on at least any one of the information, a step of setting the allowable range of the shooting direction or the zoom amount is executed. A program characterized by the above.