Shooting system, shooting method, shooting control device, and program

Abstract

[Problem] To provide an imaging system, an imaging method, an imaging control device and a program with which convenience or operability at the time of capturing an image of an imaging target can be improved. [Solution] An imaging system (10) comprises: a mobile camera (231) that is capable of adjusting an imaging direction, including a panning direction and / or a tilting direction, manually by a user or automatically; and an imaging control device (460) that controls image capture executed by of the camera. The imaging control device sets, according to the position of the mobile camera, a permissible range of an imaging direction or of a zoom amount, and controls the imaging direction or the zoom amount using the permissible range, or notifies the user of the permissible range of the imaging direction or the zoom amount.

Description

Technical Field

[0001] The present invention relates to a photographing system, a photographing method, a photographing control device, and a program.

Background Art

[0002] In Patent Document 1, an unmanned helicopter is provided that can obtain an image following the traveling direction and movement of the aircraft while moving toward a photographing target, and can also grasp the situation of the aircraft and the surrounding situation even when data communication is interrupted (

[0009] , abstract). To achieve this object, Patent Document 1 (abstract) provides an unmanned helicopter equipped with a camera at the lower part of the aircraft, which has four modes: a normal mode, a neutral mode, a pan-following mode, and an emergency mode.

[0003] In the normal mode, both the pan (horizontal) direction and the tilt (vertical) direction are held toward the photographing target. In the neutral mode, the pan (horizontal) direction is held in the nose direction, and the tilt (vertical) direction is held at a predetermined angle obliquely downward. In the pan-following mode, the pan (horizontal) direction is held at an arbitrary fixed angle with respect to the nose direction, and the tilt (vertical) direction can be arbitrarily operated by remote control. In the emergency mode, the pan (horizontal) direction is held in the nose direction, and the tilt (vertical) direction is held at a predetermined angle obliquely downward.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] As described above, Patent Document 1 (abstract) discloses four modes—normal mode, neutral mode, pan-following mode, and emergency mode—for use in an unmanned helicopter equipped with a camera on the underside of the aircraft. However, the four shooting modes of Patent Document 1 have room for improvement in terms of convenience or operability when photographing the subject.

[0006] For example, when using an unmanned aerial vehicle (aerial vehicle) equipped with a camera that the user can control in both pan (horizontal) and tilt (vertical) directions to film a sporting event taking place in a stadium, there is a risk that the user may film in an unintended direction (for example, outside the stadium). In the normal mode of Patent Document 1, both the pan (horizontal) and tilt (vertical) directions are held toward the shooting target (point of interest), so it cannot cope with situations where the subject of interest changes rapidly. Also, in pan-tracking mode, the pan (horizontal) direction is held at an arbitrary fixed angle relative to the direction of the aircraft's nose, and the tilt (vertical) direction can be arbitrarily controlled by remote control, but in filming a sporting event, constantly keeping the orientation of the unmanned aerial vehicle toward the subject of interest may impair convenience or operability.

[0007] The challenges described above apply not only when mounting cameras on aerial vehicles, but also when mounting cameras on ground-based vehicles.

[0008] This invention has been made in consideration of the above-mentioned problems, and aims to provide a shooting system, shooting method, shooting control device, and program that can improve the convenience or operability of shooting the target object. [Means for solving the problem]

[0009] A shooting system according to one aspect of the present invention is: A mobile camera whose shooting direction, including at least one of the pan or tilt directions, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and It is equipped with, The aforementioned imaging control device is The permissible range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. It is characterized by the following:

[0010] According to the present invention, the shooting direction is restricted using a tolerance range for the shooting direction that is changed according to the camera position (absolute position or relative position) of the mobile camera, or the tolerance range for the shooting direction is notified to the user. This makes it easier to shoot using a tolerance range for the shooting direction that corresponds to the camera position in a configuration where the camera is mobile and the shooting direction is adjusted manually or automatically, thereby improving convenience or operability. Therefore, for example, it is possible to prevent the camera from being pointed in an undesirable shooting direction.

[0011] The shooting control device may set the allowable range of the shooting direction so that at least a portion of the area to be photographed is included in the camera's field of view. This makes it possible to prevent the user or the shooting control device from accidentally changing the shooting direction so that the camera is pointed outside the area to be photographed.

[0012] The aforementioned shooting system may include an unmanned aerial vehicle that flies through the air. The camera may also be mounted on the unmanned aerial vehicle. This prevents the camera from being pointed in an undesirable direction during aerial photography, thereby improving convenience or operability.

[0013] The aforementioned shooting control device may restrict the shooting direction by at least one of the attitude of the unmanned aerial vehicle or the camera actuator. This makes it possible to restrict the shooting direction in a simple manner.

[0014] The camera may allow manual or automatic adjustment of the zoom amount in addition to the shooting direction. The shooting control device may set an allowable range for the zoom amount according to the camera position or the tilt direction or pan direction. The shooting control device may also limit the zoom amount using the allowable range for the zoom amount, or notify the user of the allowable range for the zoom amount. This makes it possible to improve convenience or operability by preventing the use of undesirable camera magnification in a configuration in which the camera is movable and the zoom amount is adjusted manually or automatically.

[0015] The aforementioned imaging system may include a ground-based mobile body that moves along the ground. The camera may be mounted on the ground-based mobile body. This prevents the camera from being pointed in an undesirable direction during ground imaging, thereby improving convenience or operability.

[0016] In the manual shooting direction operation mode, where the shooting direction is operated manually, the shooting control device may restrict the shooting direction operated by the user using the allowable range of the shooting direction set according to the camera position. This prevents the camera from being pointed in an undesirable shooting direction when the user manually operates the shooting direction, thereby improving convenience or operability.

[0017] In the manual zoom mode, where the zoom amount is manually controlled, the shooting control device may limit the zoom amount operated by the user using the allowable range of the zoom amount set according to the camera position. This prevents undesirable camera magnification when the user manually controls the zoom amount, thereby improving convenience or operability.

[0018] The shooting control device may be able to set a dolly shooting mode that allows only horizontal linear movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. Furthermore, the shooting control device may set a linear movement tolerance range, which is the range in which the camera's horizontal linear movement is permitted in the dolly shooting mode. In addition, the shooting control device may restrict the camera from moving outside the linear movement tolerance range, at least when the pan direction is the target pan direction. This makes it possible to prevent the camera's shooting range from going outside the target area, for example, while the pan direction remains the target pan direction, in dolly shooting mode.

[0019] In the dolly shooting mode, the shooting control device may allow the camera to move beyond the linear movement range in the direction of linear movement when a predetermined operation is performed while the camera is at one end of the linear movement range. The predetermined operation may be a first pan direction manual operation to bring the pan direction closer to the center of the linear movement range, or a zoom-out manual operation to zoom out the camera. As a result, in dolly shooting mode, when an operation is performed to keep the camera's shooting range within, for example, a stadium or event venue (first pan direction manual operation or zoom-out manual operation), the camera is allowed to move beyond the linear movement range in the direction of linear movement. Therefore, it is possible to improve the operability in dolly shooting.

[0020] In the dolly shooting mode, when the camera moves in the direction of linear movement beyond the allowable linear movement range, the shooting control device may perform a first automatic pan direction operation to bring the pan direction closer to the center of the allowable linear movement range, or an automatic zoom-out operation to zoom out the camera. This makes it possible to prevent the camera's shooting range from extending, for example, outside the area to be photographed, in dolly shooting mode.

[0021] In the dolly shooting mode, when the camera is outside the linear movement allowable range and a second pan direction manual operation for approaching the pan direction to the target pan direction is performed, the shooting control device may perform a camera position automatic operation for approaching the camera to the linear movement allowable range. Thereby, when the pan direction of the camera is approaching the target pan direction during the first pan direction automatic operation, it is possible to prevent the shooting range of the camera from going outside the shooting target area, for example.

[0022] In the dolly shooting mode, when the camera returns from outside the linear movement allowable range toward the linear movement allowable range, the shooting control device may perform a second pan direction automatic operation for approaching the pan direction to the target pan direction. Thereby, when returning the shooting direction that was restricted by exceeding the horizontal movement allowable range to the linear movement allowable range side, the pan direction can be automatically adjusted to loosen the restriction.

[0023] When the camera enters the shooting target area, if the shooting target area is not within the shooting range of the camera, the shooting control device may automatically adjust the shooting direction, the camera position, or the zoom amount so that the shooting target area enters the shooting range of the camera. Thereby, it becomes possible to smoothly start shooting the shooting target area.

[0024] The shooting system may further include a notification means for notifying the user of the allowable range of the shooting direction or the zoom amount. Further, when the shooting direction or the zoom amount is restricted, the notification means may notify that the restriction is imposed by image display on a display unit or the lighting or blinking of light or voice or vibration. Thereby, it becomes possible to notify the user that the shooting direction or the zoom amount is restricted.

[0025] The notification means may display information regarding the allowable range of the shooting direction or the zoom amount corresponding to the camera position. Thereby, it becomes easier for the user to perform camera operations.

[0026] The aforementioned imaging control device is The ratio of the area to be photographed and the area outside the area to be photographed in the image captured by the camera, The outer edge information of the area to be photographed, entered by the user, The value or percentage of the area of ​​the target region in the captured image, or The area value or percentage of the region outside the target area in the aforementioned captured image. The acceptable range for the shooting direction or zoom amount may be set based on at least one of the following pieces of information. This eliminates the need for the user to input the acceptable range for the shooting direction and zoom amount for every location, simplifying the setting of the acceptable ranges.

[0027] Another aspect of the present invention relates to a photographic method, A mobile camera whose shooting direction, including at least one of the pan or tilt directions, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting method using a shooting system equipped with, The aforementioned imaging control device is The permissible range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. It is characterized by the following:

[0028] A shooting control device according to yet another aspect of the present invention controls the shooting of a mobile camera whose shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, The aforementioned imaging control device is The permissible range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. It is characterized by the following:

[0029] A program according to yet another aspect of the present invention is executed in a shooting control device that controls the shooting of a mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, In the aforementioned imaging control device, The steps include changing the allowable range of the shooting direction according to the camera position, which is the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional shooting target area or shooting target point, A step of restricting the shooting direction using the aforementioned tolerance range, or a step of notifying the user of the aforementioned tolerance range for the shooting direction. It is characterized by performing the following. [Effects of the Invention]

[0030] According to the present invention, it is possible to improve the convenience or operability of photographing the subject. [Brief explanation of the drawing]

[0031] [Figure 1] This is an overall configuration diagram of a shooting system according to one embodiment of the present invention. [Figure 2] This is a functional configuration diagram of the drone according to the above embodiment. [Figure 3] This is a simplified external perspective view showing the drone of the above embodiment. [Figure 4] This is a functional configuration diagram of the control device of the above embodiment. [Figure 5] This is a simplified front view showing the control device of the above embodiment. [Figure 6]This is a functional configuration diagram of the server according to the above embodiment. [Figure 7] This flowchart shows the overall flow of aerial photography control in the above embodiment. [Figure 8] This diagram shows how the drone moves from the takeoff point to the area to be photographed and from the area to be photographed to the landing point in the above embodiment. [Figure 9] This figure shows an example of a list of candidate shooting areas used to select a shooting target area in the above embodiment. [Figure 10] This is the first explanatory diagram relating to the setting of the flight geofence in the above embodiment. [Figure 11] This is a second explanatory diagram relating to the setting of the flight geofence in the above embodiment. [Figure 12] This is a first explanatory diagram relating to the setting of the shooting restriction in the above embodiment. [Figure 13] This is a second explanatory diagram relating to the setting of the shooting restriction in the above embodiment. [Figure 14] This is a third explanatory diagram relating to the setting of the shooting restriction in the above embodiment. [Figure 15] This is a fourth explanatory diagram relating to the setting of the shooting restriction in the above embodiment. [Figure 16] This is a flowchart of the operation restriction control during normal shooting mode in the above embodiment. [Figure 17] This is a flowchart of the operation restriction control during the first dolly shooting mode in the above embodiment. [Figure 18] This is a first explanatory diagram of the operation restriction control during the first dolly shooting mode in the above embodiment. [Figure 19] This is a second explanatory diagram of the operation restriction control during the first dolly shooting mode in the above embodiment. [Figure 20] This is a third explanatory diagram of the operation restriction control during the first dolly shooting mode in the above embodiment. [Figure 21] This is a flowchart of the operation restriction control during the second dolly shooting mode in the above embodiment. [Figure 22]This is an overall configuration diagram of a photographing system according to a first modification example of the present invention. [Figure 23] This is an overall configuration diagram of a photographing system according to a second modification example of the present invention. [Figure 24] This is an overall configuration diagram of a photographing system according to a third modification example of the present invention. [Figure 25] This is an explanatory diagram regarding the limitation of the photographing direction according to the first modification example of the present invention. [Figure 26] This is an explanatory diagram regarding the limitation of the photographing direction according to the second modification example of the present invention.

Mode for Carrying Out the Invention

[0032] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. Further, the embodiments shown below are merely examples, and other known elements and alternative means can be adopted according to the use, purpose, scale, etc.

[0033] <A. One Embodiment> [A-1. Configuration] (A-1-1. Overall Configuration) FIG. 1 is an overall configuration diagram of a photographing system 10 (hereinafter also referred to as "system 10") according to an embodiment of the present invention. The system 10 is for aerial photographing of a predetermined area where people may be active, such as a competition held in an arena 90 (FIG. 8) or an event held in an event venue, by a drone 20. As shown in FIG. 1, in addition to the drone 20, the system 10 includes a control device 30 for an operator to operate the drone 20, and a server 40 for managing the flight and photographing of the drone 20.

[0034] In this embodiment, "competition" refers to a competition that tests skills and abilities, and includes soccer and various other sports. It may also include practice sessions, not just matches. The "field" is the place where the competition takes place, and may include not only the area inside the court as defined by the lines shown in Figure 8, but also the area outside the court. Furthermore, the present invention can be applied to a shooting system that photographs a target area for any application, not limited to competitions or events.

[0035] The drone 20 and the control device 30 are connected to each other via wireless communication (which may include communication via the base station 52). The control device 30 and the server 40 are connected to each other via a communication network 50 such as an internet connection. The drone 20 acquires satellite signals from the artificial satellite 54 for purposes such as determining its own position. The configuration of the system 10 is not limited to that shown in Figure 1; for example, those shown in Figures 22 to 24 can also be used (details will be described later).

[0036] (A-1-2. Drone 20) (A-1-2-1. Overview of Drone 20) Figure 2 is a functional configuration diagram of the drone 20 of this embodiment. Figure 3 is a simplified external perspective view of the drone 20 of this embodiment. As described above, the drone 20 takes aerial photographs of competitions taking place in the stadium 90 (Figure 8), events taking place at event venues, etc. As shown in Figure 2, the drone 20 includes a drone sensor group 200, a communication unit 210, a flight mechanism 220, a shooting mechanism 230, and a drone control unit 240.

[0037] In this specification, "drone" refers to any aircraft having multiple rotors and capable of autonomous attitude control, regardless of its power source (electricity, prime mover, etc.), control method (wireless or wired, fully autonomous or partially manual, etc.), and whether it is manned or unmanned. Drones may also be referred to as Unmanned Aerial Vehicles (UAVs), aircraft, multicopters, RPAS (Remote Piloted Aircraft Systems), or UAS (Unmanned Aircraft Systems).

[0038] (A-1-2-2. Drone sensor group 200) The drone sensor group 200 includes various sensors placed on the drone 20. Specifically, the drone sensor group 200 includes a position measuring unit 201, a direction measuring unit 202, an altimeter 203, a speedometer 204, a gyro sensor 205, an obstacle sensor 206, etc. In addition to these, the drone sensor group 200 may also include various sensors that acquire information such as temperature, atmospheric pressure, wind speed, and acceleration.

[0039] The position measurement unit 201 receives signals from the artificial satellite 54 (Figure 1) and measures the position (absolute position) of the aircraft based on these signals. The position measurement unit 201 is not particularly limited, but for example, it measures its current position using GNSS (Global Navigation Satellite System), GPS (Global Positioning System), etc. As a method for measuring its own position, for example, RTK-GNSS (Real Time Kinematic - Global Navigation Satellite System) can also be used. The position information includes at least two-dimensional coordinate information in a planar view (e.g., latitude, longitude), and preferably three-dimensional coordinate information including altitude information.

[0040] Furthermore, by wirelessly connecting the base station 52, which provides information on the reference point of a fixed station used for relative positioning such as RTK, to the drone 20 and the control device 30, it becomes possible to measure the position of the drone 20 with higher accuracy. Here, when performing RTK measurement using a virtual reference point method with VRS (Virtual Reference Station), the base station 52 can be omitted, or the accuracy of estimating the position coordinates of the base station 52 or the drone 20 can be further improved.

[0041] The direction-measuring unit 202 measures the orientation (heading direction) of the aircraft. The direction-measuring unit 202 consists of a geomagnetic sensor, a compass, etc., which measures the heading direction of the drone 20 by measuring the Earth's magnetic field, for example.

[0042] The altimeter 203 measures the altitude above ground (hereinafter referred to as "altitude H") as the distance to the ground below the drone 20 (vertically downward). The measured altitude H acquired by the altimeter 203 is also called the measured altitude Hd. The airspeed meter 204 detects the flight speed of the drone 20. The gyro sensor 205 detects the angular velocity of the drone 20. The obstacle sensor 206 has multiple control cameras 207 (Figure 3) and measures the position, velocity vector, etc. of a person located below the drone 20, etc., based on the acquired images.

[0043] (A-1-2-3. Communications Section 210) The communication unit 210 is capable of radio communication via the communication network 50 (Figure 1) and includes, for example, a radio communication module. The communication unit 210 can communicate with the control device 30, etc., via the communication network 50 (including the radio base station 52).

[0044] (A-1-2-4.Flight Mechanism 220) The flight mechanism 220 is a mechanism for flying the drone 20, generating thrust for the drone 20 to lift it off the ground and move it in a desired direction. As shown in Figures 2 and 3, the flight mechanism 220 has a plurality of rotors 221 and a plurality of rotor actuators 222. The rotor actuators 222 have, for example, electric motors.

[0045] The flight mechanism 220 may also be provided with a propeller guard (not shown) to prevent the propeller from interfering with obstacles. The number of rotor blades 221 constituting the flight mechanism 220 is not particularly limited, but for example, it can be equipped with 1, 2, 4, 6, or 8 rotor blades. The rotor blades 221 may consist of a single propeller or a plurality of coaxially arranged propellers. The number and shape of the blades of each propeller are not particularly limited.

[0046] (A-1-2-5. Imaging mechanism 230) The imaging mechanism 230 is a mechanism for capturing images of competitions in the stadium 90 (Figure 8), events at event venues, etc., and includes a camera 231, a camera holder 232, and an image processing unit 233. As shown in Figure 3, the camera 231 (imaging device) is located at the bottom of the drone 20 and outputs image data relating to peripheral images captured around the drone 20. The camera 231 is a video camera (color camera) that captures video. The video may also include audio data acquired by a microphone (not shown). In addition to or instead of this, the camera 231 may also capture still images.

[0047] The orientation of the camera 231 (the attitude of the camera 231 relative to the body of the drone 20) can be adjusted by a camera actuator (not shown) incorporated into the camera holder 232. Alternatively, the position of the camera 231 relative to the body of the drone 20 may be fixed. The camera holder 132 may have a mechanism to suppress the transmission of shaking or vibration of the aircraft to the camera 231. The image processing unit 233 performs predetermined image processing on the image data acquired by the camera 231. The image data acquired by the camera 231 can be transmitted to the drone 20's own memory unit, control device 30, server 40, etc. Part or all of the image processing unit 233 may be positioned as part of the shooting control unit 242, which will be described later.

[0048] (A-1-2-6. Drone control unit 240) The drone control unit 240 controls the entire drone 20, including its flight and photography. The drone control unit 240 includes an input / output unit, a calculation unit, and a storage unit (not shown). The drone control unit 240 is equipped with a calculation device such as a CPU (Central Processing Unit) for performing information processing, and storage devices such as RAM (Random Access Memory) and ROM (Read Only Memory). The drone control unit 240 has a flight control unit 241 and a photography control unit 242.

[0049] The flight control unit 241 controls the flight of the drone 20 (attitude angle control and flight operation of the aircraft from takeoff to during flight and landing) via the flight mechanism 220. The flight control unit 241 has a processing unit also called a flight controller. The processing unit may have one or more processors such as a programmable processor (e.g., a central processing unit (CPU), MPU, or DSP). The processing unit has access to memory (storage unit). The memory stores logic, code, and / or program instructions that the processing unit can execute to perform one or more steps. The memory may include, for example, a separable medium such as an SD card or RAM, or an external storage device. Various data acquired from the drone sensor group 200 may be directly transmitted to and stored in the memory. For example, video or still image data captured by the camera 231 can be recorded in the internal memory or external memory.

[0050] The processing unit includes a control module configured to control the state of the drone 20. For example, the control module controls the flight mechanism 220 (thrust generating unit) of the drone 20 to adjust the spatial positioning, attitude angle, angular velocity, angular acceleration, angular jerk velocity, and / or acceleration of the drone 20, which has six degrees of freedom (translational motion x, y, and z, and rotational motion θx, θy, and θz). The control module can control one or more of the camera holder 232 and sensors.

[0051] The flight control unit 241 can control the flight of the drone 20 based on control signals from the control device 30 or based on a preset autonomous flight program. The flight control unit 241 can also control the flight of the drone 20 by controlling the flight mechanism 220 (thrust generating unit) based on various information such as the target field to be photographed, permitted / prohibited flight areas, corresponding flight geofence information, map information including 2D or 3D map data, the current position information of the drone 20, attitude information (heading information), speed information, and acceleration information, and any combination thereof.

[0052] In this specification, “Target Field” (or “Target Area”) means a two-dimensional location to be photographed (for example, the outline defining a sports field 90 (e.g., the frame consisting of the touchlines and goal lines of a soccer field)). “Permitted / Prohibited Flight Area” means a three-dimensional spatial unit where the flight of the drone 20 is permitted or prohibited. “Geofence” indicates a virtual boundary line, and in particular, indicates a fence marking the boundary between the permitted flight area and the prohibited flight area where a mobile object such as the drone 20 is permitted to fly or move. Therefore, if a mobile object such as the drone 20 comes into contact with the geofence, its flight or movement will be restricted so that it does not fly outside the permitted flight area.

[0053] The shooting control unit 242 controls the shooting by the drone 20 via the shooting mechanism 230.

[0054] (A-1-3. Control device 30) (A-1-3-1. Overview of the control device 30) Figure 4 is a functional configuration diagram of the control device 30 of this embodiment. Figure 5 is a simplified front view of the control device 30 of this embodiment. The control device 30 is a portable information terminal that controls the drone 20 by the operator's operation and displays information received from the drone 20 (e.g., position, altitude, battery level, camera image, etc.). In this embodiment, the flight status of the drone 20 (altitude, attitude, etc.) is remotely controlled by the control device 30, but the drone 20 may be controlled autonomously. In that case, when a flight command is transmitted from the operator to the drone 20 via the control device 30, the drone 20 will perform autonomous flight. However, manual operation may be possible during basic operations such as takeoff and return, and in emergencies.

[0055] The control device 30 includes an input / output unit 300 and a communication unit 310. The input / output unit 300 handles various inputs from the user, such as the pilot, outputs to the user, and inputs and outputs of signals between the drone 20 and the server 40. The input / output unit 300 includes an operation input unit 320 and a display unit 330. The operation input unit 320 and the display unit 330 are connected to each other via wired or wireless communication. The control device 30 also includes a processing unit such as a CPU, and storage devices such as RAM and ROM for performing information processing. Furthermore, the control device 30 in this embodiment receives and displays work instructions and the like from the server 40.

[0056] (A-1-3-2. Communications Section 310) The communication unit 310 is located in the same housing as the operation input unit 320 or the display unit 330 and has a communication function that enables wireless communication with the drone 20 using Wi-Fi, 2.4GHz, and 5.6~5.8GHz frequency bands. The communication unit 310 also has a wireless communication function that enables communication with the server 40 via the internet line 50 using communication standards such as LTE (Long Term Evolution). In the system 10 of this embodiment (Figure 1), the drone 20 communicates with the server 40 via the control device 30, so this system configuration is suitable when the drone 20 and the control device 30 are within a distance where direct wireless communication is possible (for example, visual flight by an operator), but is not limited to this.

[0057] (A-1-3-3. Operation input section 320) The operation input unit 320 accepts various inputs from the user, such as the pilot, and allows the pilot to input flight direction and operation commands such as takeoff / landing when operating the drone 20. For example, the control input unit 320 accepts input operations that instruct the three-dimensional flight movements of the drone 20, including takeoff, landing, ascent, descent, turning, forward, backward, and left / right movement. As shown in Figure 4, the operation input unit 320 in this embodiment includes a movement input unit 321, a drone attitude input unit 322, a camera attitude input unit 323, a camera zoom input unit 324, a flight mode switching input unit 325, and a power input unit 326. As shown in Figure 5, the operation input unit 320 as hardware includes a left input stick 327L, a right input stick 327R, a left two-way switch button 328L, a right two-way switch button 328R, and a menu button 329.

[0058] The movement input unit 321 is an input unit for the operator to move the drone 20 in the up-down, left-right, and forward-backward directions, and is composed of a right input stick 327R and a left input stick 327L. Specifically, when the right input stick 327R is moved upward (towards the back), the drone 20 rises, and when the right input stick 327R is moved downward (towards the front), the drone 20 descends. When the right input stick 327R is moved to the right, the drone 20 moves to the right, and when the right input stick 327R is moved to the left, the drone 20 moves to the left. When the left input stick 327L is moved upward (towards the back), the drone 20 moves forward, and when the left input stick 327L is moved downward (towards the front), the drone 20 moves backward.

[0059] The drone attitude input unit 322 is an input unit for the operator to control the attitude of the drone 20, and is composed of a left input stick 327L. The attitude of the drone 20 as referred to here includes yaw turns. That is, when the left input stick 327L is moved to the right, the drone 20 turns to the right, and when the left input stick 327L is moved to the left, the drone 20 turns to the left.

[0060] The camera attitude input unit 323 is an input unit for controlling the attitude of the shooting camera 231, and consists of a right two-way switch button 328R and a right input stick 327R. Specifically, pressing the right side of the right two-way switch button 328R moves the camera 231 to the right. Pressing the left side of the right two-way switch button 328R moves the camera 231 to the left. Also, pressing the right two-way switch button 328R while pushing the right input stick 327R downwards moves the camera 231 downwards. Pressing the right two-way switch button 328R while pushing the right input stick 327R upwards moves the camera 231 upwards.

[0061] The camera zoom input section 323 is an input section for controlling the zoom of the shooting camera 231, and is composed of a left two-way switch button 328L. Specifically, pressing the right side of the left two-way switch button 328L zooms the camera 231 in. Pressing the left side of the left two-way switch button 328L zooms the camera 231 out.

[0062] The flight mode switching unit 325 is an input unit for switching the flight mode of the drone 20, and consists of a menu button 329 and a left input stick 327L. That is, the flight mode is switched by selecting from the menu displayed by pressing the menu button 329 using the left input stick 327L. The power input unit 326 is the part that turns the power of the control device 30 on and off, and consists of a mechanical switch or the like.

[0063] (A-1-3-4.Display section 330) The display unit 330 displays status information of the drone 20, etc., obtained from the drone 20 or the server 40 to the operator. The display unit 330 may consist of a touch panel or LCD monitor integrated into the control device 30, or it may consist of a display device such as an LCD monitor, tablet terminal, or smartphone connected to the control device 30 by wire or wireless connection. The display unit 330 can display images related to various information such as the target field for shooting, flight permitted / prohibited areas, flight geofences, map information, the current position information of the drone 20, attitude information (direction information), speed information, acceleration information, and battery level.

[0064] (A-1-4. Server 40) (A-1-4-1. Overview of Server 40) Figure 6 is a functional configuration diagram of the server 40 in this embodiment. The server 40 manages or controls the flight and photography of the drone 20. As shown in Figure 6, the server 40 has an input / output unit 400, a communication unit 410, a calculation unit 420, and a storage unit 430. The input / output unit 400 is a part for inputting or outputting various types of information (image output, audio output). The communication unit 410 has a modem (not shown) and can communicate with the drone 20, control device 30, etc. via the communication network 50.

[0065] The arithmetic unit 420 includes a CPU and operates by executing a program stored in the memory unit 430. Some of the functions performed by the arithmetic unit 420 can also be implemented using logic ICs (Integrated Circuits). The arithmetic unit 420 can also implement some of the aforementioned programs using hardware (circuit components).

[0066] The storage unit 430 stores programs and data used by the arithmetic unit 420 and includes RAM. The RAM can be volatile memory such as registers, or non-volatile memory such as a hard disk or flash memory. In addition to RAM, the storage unit 430 may also have ROM.

[0067] Server 40 may be a general-purpose computer such as a workstation or personal computer, or it may be logically implemented through cloud computing.

[0068] (A-1-4-2. Arithmetic unit 420) As shown in Figure 6, the calculation unit 420 includes a pre-setting unit 440, a flight control unit 450, and an image capture control unit 460. The pre-setting unit 440 pre-sets the flight and image capture of the drone 20. The flight control unit 450 controls the flight of the drone 20. The image capture control unit 460 controls the image capture by the drone 20.

[0069] As shown in Figure 6, the pre-setting unit 440 includes a target field selection unit 441, a geofence setting unit 442, and a shooting restriction setting unit 443. The target field selection unit 441 is a part that selects the target field by user operation on, for example, the menu button 329 and the left input stick 327L. The geofence setting unit 442 is a part that sets the flight geofence 91 (Figure 18) by user operation on, for example, the menu button 329 and the left input stick 327L. The shooting restriction setting unit 443 is a part that selects the limit range for the shooting direction and zoom amount by user operation on, for example, the menu button 329 and the left input stick 327L. The shooting restriction setting unit 443 includes a shooting direction restriction setting unit 444 and a zoom restriction setting unit 445. The shooting direction restriction setting unit 444 is a part that restricts the shooting direction, and the zoom restriction setting unit 445 is a part that restricts the zoom amount.

[0070] The flight control unit 450 controls the flight of the drone 20. This flight includes, in addition to the flight during photography, takeoff, movement from the takeoff point Pto (Figure 8) to the target photography location (e.g., the stadium 90), movement from the target photography location to the target landing point Pln, and landing.

[0071] The shooting control unit 460 includes a shooting mode setting unit 461, a drone status acquisition unit 462, a shooting direction restriction unit 463, a zoom restriction unit 464, a flight restriction unit 465, and an automatic shooting control unit 466. The shooting mode setting unit 461 is the part that sets the shooting mode. The drone status acquisition unit 462 is the part that acquires the status of the drone 20. The shooting direction restriction unit 463 is the part that restricts the shooting direction of the camera 231. The zoom restriction unit 464 is the part that restricts the zoom amount of the camera 231. The flight restriction unit 465 is the part that restricts the flight of the drone 20. The automatic shooting control unit 466 is the part that controls automatic shooting by the camera 231.

[0072] [A-2. Control] (A-2-1. Overall flow) Next, the various controls of this embodiment will be described. Figure 7 is a flowchart showing the overall flow of aerial photography control in this embodiment. Figure 8 shows how the drone 20 moves from the takeoff point Pto to the target field (for example, the stadium 90) and from the target field to the target landing point Pln in this embodiment. As described above, in this embodiment, the drone 20 takes aerial photographs of competitions taking place in the stadium 90 (Figure 8), events taking place at the event venue, etc. Broadly speaking, the controls of this embodiment can be divided into controls for pre-setting before aerial photography (pre-setting control in step S10) and controls when performing aerial photography (aerial photography control in step S20), as shown in Figure 7.

[0073] (A-2-2. Pre-configured control (S10 in Figure 7)) (A-2-2-1. Selection of the target field for shooting) In step S101 of Figure 7, the server 40 (imaging control unit 460) selects a field to be photographed based on input from the user via the control device 30. Figure 9 shows an example of a field list 60 used to select a field to be photographed in this embodiment. The field list 60 includes the field name, field identification number (field ID), and address information. The user selects a field to be photographed from the field list 60.

[0074] (A-2-2-2. Setting up the flight geofence 91) In step S102 of Figure 7, the server 40 (geofence setting unit 442) sets the flight geofence 91 (Figure 18) based on the user input from the control device 30. Figure 10 is a first explanatory diagram relating to the setting of the flight geofence in this embodiment. Figure 10 shows screen 61a as an example of the screen when setting the flight geofence. Screen 61a includes an image 600 of the stadium 90 (hereinafter also referred to as "stadium image 600"), geofence images 601a and 601b (hereinafter also referred to as "geofence images 601a and 601b"), and an obstacle image 602 (hereinafter also referred to as "obstacle image 602").

[0075] Geofence image 601a is the geofence image registered in the storage unit 430 of server 40, and geofence image 601b is the geofence image modified by the user. Specifically, geofence image 601b is enlarged on the left and bottom sides of geofence image 601a (see arrow 603a). Also, geofence image 601b is enlarged on the right side of geofence image 601a (see arrow 603b). However, because there is an obstacle image 602 in the lower right of stadium image 600 (and because there is an obstacle in the lower right of the actual stadium 90), the enlargement of the lower right of geofence image 601b is restricted. Note that arrows 603a and 603b are to show how geofence image 601a changes to geofence image 601b due to user operation, and are not displayed on the actual screen 61a.

[0076] A flight geofence (hereinafter also referred to as "geofence") indicates a virtual boundary line, and in particular, it indicates a fence marking the boundary between a flight-permitted area and a no-fly-permitted area where mobile objects such as drones 20 are permitted to fly or move. Therefore, if a mobile object such as drone 20 comes into contact with a geofence, its flight or movement will be restricted to prevent it from flying outside the flight-permitted area.

[0077] In the example shown in Figure 10, the geofence is represented as a two-dimensional area (length + width + height), but in reality, it is defined as a three-dimensional area (length + width + height + height). (However, for control purposes, it may be managed as a two-dimensional area.)

[0078] Figure 11 is a second explanatory diagram relating to the setting of aerial geofences in this embodiment. Figure 11 shows screen 61b as an example of a screen when setting up aerial geofences. Screen 61b includes a stadium image 600 and geofence images 601c, 601d, and 601e.

[0079] Geofence image 601c is a geofence image registered in the storage unit 430 of server 40, while geofence images 601d and 601e are geofence images modified or added by the user. Specifically, geofence image 601d is an enlarged version of geofence image 601c with the right, top, and bottom sides enlarged (see arrow 603c). Geofence image 601e is a copy of geofence image 601c added to the left of stadium image 600 (see arrow 603d). Note that arrows 603c and 603d are intended to show how geofence image 601c changes to geofence images 601d and 601e due to user operation, and are not displayed on the actual screen 61b.

[0080] In the examples of Figures 10 and 11, the geofence settings were changed, but it is also possible to use the geofence (geofence image 601a or 601c) registered in the storage unit 430 of the server 40 simply by having the user confirm it. Alternatively, it is possible to omit the confirmation and modification of the geofence, assuming that the geofence registered in the storage unit 430 of the server 40 will be used as is (by proceeding immediately to step S103 after performing step S101, and confirming or modifying the geofence by other operations as needed).

[0081] (A-2-2-3. Setting shooting restrictions) In step S103 of Figure 7, the server 40 (shooting restriction setting unit 443) sets shooting restrictions based on user input from the control device 30 and the position (camera position) of the drone 20 (camera 231). In this embodiment, the shooting restrictions set an allowable range (hereinafter also referred to as the "restricted range") for both the shooting direction and zoom amount of the camera 231 for each camera position. The restricted range may be defined by a numerical range limit using three angle information (angles around the XYZ axes) that define the camera orientation for each position coordinate of the camera 231, or the restricted range may be defined by a three-dimensional vector for the camera direction for each position coordinate of the camera 231. Alternatively, as an alternative, such as Visual SLAM (Simultaneous Localization and Mapping) or spatial coordinate surveying using images, areas where shooting is prohibited can be set on a map (in spatial coordinates), and restrictions can be set on the shooting direction and movement when the spatial coordinates of the prohibited area enter the field of view during shooting. Furthermore, when setting the acceptable range for zoom amount, it is possible to set the range not only for each camera position, but also for each orientation of camera 231 (tilt direction or pan direction).

[0082] Figure 12 is a first explanatory diagram relating to the setting of shooting restrictions in this embodiment. Figure 12 shows screen 62 as an example of a screen when setting shooting restrictions. Screen 62 includes a stadium image 600 and four drone images 621a, 621b, 621c, and 621d. Drone images 621a to 621d correspond to multiple points P1, P2, P3, and P4 into which shooting restriction information is input when setting shooting restrictions in this embodiment. That is, at each point P1 to P4, the restricted range of the shooting direction and the restricted range of the zoom amount are input. Point P1 is the center point of the upper goal line. Point P2 is the intersection of the goal line and the touchline in the upper right. Point P3 is the intersection of the center line and the touchline on the right side. Point P4 is the position of the center mark.

[0083] When the user selects one of the drone images 621a to 621d on the display unit 330, the system transitions to a location setting screen corresponding to the selected drone image (for example, screen 63 in Figure 13). Although omitted in Figure 12, screen 62 can also include operation guidance messages.

[0084] Figure 13 is a second explanatory diagram relating to the setting of shooting restrictions in this embodiment. Figure 13 shows screen 63 as an example of a screen when setting shooting restrictions. Screen 63 includes a stadium image 600, a drone image 621a, a direction of travel image 631a, a pan direction restriction range image 632, and a virtual shooting image 634. The drone image 621a in Figure 13 is the same as the drone image 621a in Figure 12, and corresponds to the selection of drone image 621a from drone images 621a to 621d in Figure 12. In other words, if drone image 621b is selected in Figure 12, then on screen 63, the drone image 621b of point P2 will be displayed instead of the drone image 621a of point P1.

[0085] The direction of travel image 631a shows the direction of travel of the drone image 621a. In the example in Figure 13, the direction of travel image 631a corresponding to point P1 is pointing to the left, but as in Figure 15, the direction of travel image 631b corresponding to point P1 may be pointing downwards. The pan direction restriction range image 632 shows the restriction range for the pan direction when the drone image 621a is at point P1 and the direction of travel is the direction indicated by the direction of travel image 631a (to the left in Figure 13).

[0086] The virtual image 634 shows a virtual image (simulated image) that indicates the imaging range of camera 231 when drone image 621a is at point P1 and the direction of travel is the direction indicated by the direction of travel image 631a (leftward in Figure 13). In addition, pan adjustment buttons 635L and 635R are superimposed on the virtual image 634.

[0087] When the user presses the pan adjustment button 635L, the restricted range image 632 rotates to the left (or counterclockwise in Figure 13), and when the user presses the pan adjustment button 635R, the restricted range image 632 rotates to the right (or clockwise in Figure 13) (the size of the restricted range image 632 does not change). The virtual captured image 634 also changes in accordance with the rotation of the restricted range image 632. When the user presses the first confirmation button (not shown), the adjustment of the pan direction restricted range image 632 is completed, and the system moves to the next setting screen (for example, screen 64 in Figure 14). Screen 63 may also have other buttons, such as a first back button (not shown) to return to screen 62 in Figure 12. In addition, although omitted in Figure 13, screen 63 may also include operation guidance messages.

[0088] The restriction range for the direction of travel opposite to the direction of travel image 631a (to the right in Figure 13) is calculated using the restriction range for the direction of travel shown in the direction of travel image 631a. That is, the restriction range is calculated and set so as to be symmetrical with respect to the imaginary straight line connecting the center mark and the penalty mark.

[0089] Figure 14 is a third explanatory diagram relating to the setting of shooting restrictions in this embodiment. Figure 14 shows screen 64 as an example of a screen when setting shooting restrictions. Screen 64 includes a stadium image 600, a drone image 621a, a direction of travel image 631a, and a virtual shooting image 644. The drone image 621a in Figure 14 is the same as the drone image 621a in Figures 12 and 13, and corresponds to the selection of drone image 621a from drone images 621a to 621d in Figure 12. In other words, if drone image 621b is selected in Figure 12, the drone image 621b of point P2 will be displayed in screen 64. Similar to Figure 13, the direction of travel image 631a indicates the direction of travel of the drone image 621a.

[0090] The virtual image 644 shows a virtual image (simulated image) that indicates the imaging range of camera 231 when drone image 621a is at point P1 and the direction of travel is the direction indicated by the direction of travel image 631a (leftward in Figure 14). In other words, the virtual image 644 in Figure 14 is the same as the virtual image 634 in Figure 13. However, tilt adjustment buttons 646U and 646D are superimposed on the virtual image 644.

[0091] When the user presses the tilt adjustment button 646U, the tilt direction of the camera 231 moves upward, and when the user presses the tilt adjustment button 646D, the tilt direction of the camera 231 moves downward. When the user presses the first tilt direction lower limit determination button (not shown), the adjustment of the lower limit of the tilt direction is completed, and when the user presses the first tilt direction upper limit determination button (not shown), the adjustment of the upper limit of the tilt direction is completed. Once the upper and lower limits of the tilt direction are set, the system moves to the next setting screen (for example, screen 65 in Figure 15). Screen 64 may also have other buttons, such as a second back button (not shown) to return to screen 62 in Figure 12 or screen 63 in Figure 13. Although not shown in Figure 14, screen 64 may also include operation guidance messages. Alternatively, the tilt adjustment buttons 646U and 646D on screen 64 may be included on screen 63, allowing the pan and tilt direction limits to be set together on screen 63.

[0092] Figure 15 is a fourth explanatory diagram relating to the setting of shooting restrictions in this embodiment. Figure 15 shows screen 65 as an example of a screen when setting shooting restrictions. Screen 65 includes a stadium image 600, a drone image 621a, a direction of travel image 631b, and a virtual shooting image 654. The drone image 621a in Figure 15 is the same as the drone image 621a in Figures 12 to 14, and corresponds to the selection of drone image 621a from drone images 621a to 621d in Figure 12. In other words, if drone image 621b is selected in Figure 12, the drone image 621b of point P2 will be displayed on screen 65. The direction of travel image 631b indicates the direction of travel of the drone image 621a. In the example in Figure 15, the direction of travel image 631b corresponding to point P1 is pointing downwards.

[0093] The virtual image 654 shows a virtual image (simulated image) that indicates the imaging range of the camera 231 when the drone image 621a is at point P1 and the direction of travel is the direction indicated by the direction of travel image 631b (downward in Figure 15). Similar to the virtual image 644 in Figure 14, the tilt adjustment buttons 646U and 646D are superimposed on the virtual image 654.

[0094] In Figure 15, when the user presses the tilt adjustment button 646U, the tilt direction of the camera 231 moves upward, and when the user presses the tilt adjustment button 646D, the tilt direction of the camera 231 moves downward. When the user presses the second tilt direction lower limit determination button (not shown), the adjustment of the lower limit of the tilt direction is completed, and when the user presses the second tilt direction upper limit determination button (not shown), the adjustment of the upper limit of the tilt direction is completed. Once the upper and lower limits of the tilt direction are set, the system moves to the next setting screen (for example, screen 62 in Figure 12). Screen 65 may also include other buttons, such as a third back button (not shown), to return to screen 62 in Figure 12 or screen 64 in Figure 14. Although not shown in Figure 15, screen 65 may also include operation guidance messages.

[0095] In the examples shown in Figures 13 to 15, the limit range for the pan direction (Figure 13) and the limit range for the tilt direction (Figures 14 and 15) were input for point P1. In addition to or instead of this, the limit range for the zoom amount of camera 231 may also be input for point P1. The input of the zoom amount limit range can be done, for example, by providing zoom-up buttons and zoom-down buttons (neither shown) on each screen 63 to 65 and setting them in accordance with the settings for the limit range for the pan direction and the limit range for the tilt direction.

[0096] For the other restriction input points P2 to P4, the restriction ranges for the shooting direction (pan direction and tilt direction) and / or the zoom amount are input in the same way as for point P1. More specifically, at point P2, the drone image 621b sets the restriction ranges for the pan direction and tilt direction for the leftward and downward direction cases in Figures 12 and 13. At point P3, the drone image 621c sets the restriction ranges for the pan direction and tilt direction for the leftward and downward direction cases in Figures 12 and 13. At point P4, the drone image 621d sets the restriction ranges for the pan direction and tilt direction for the leftward and downward direction cases in Figures 12 and 13. Then, the server 40 calculates the restriction ranges for each point other than points P1 to P4 based on the restriction ranges input for each of points P1 to P4. In this case, the restriction ranges are calculated using, for example, the following method.

[0097] For each point on the straight line (line 1-2) connecting points P1 and P2, the limit range is calculated and set such that the difference between the limit range of point P1 and the limit range of point P2 changes gradually (for example, linearly) with respect to the pan direction, tilt direction, and zoom amount (in other words, assuming proportional change). The same applies to each point on the straight line (line 2-3) connecting points P2 and P3, each point on the straight line (line 3-4) connecting points P3 and P4, and each point on the straight line (line 1-4) connecting points P1 and P4. As a result, the limit range is calculated and set for each point on the rectangular frame line (lines 1-2, 2-3, 3-4, 1-4) connecting points P1 to P4. Alternatively, the limit range for each point may be calculated and set based on other types of changes (for example, changes in the shape of a circular curve or a quadratic curve) instead of proportional change.

[0098] Next, the restricted range is calculated and set for each point on the line connecting point P1 and point P4 (line 1-4), and for each point on the line connecting point P2 and point P3 (line 2-3) (each point on the line perpendicular to lines 1-4 and 2-3), in the same way as for each point on line 1-2 or line 3-4. This calculates and sets the restricted range for each point inside the rectangle connecting points P1 to P4. In other words, the restricted range is calculated and set for the upper right rectangle of the four rectangles obtained by dividing the soccer field in Figure 12 into four sections.

[0099] The remaining three rectangles (bottom right, top left, and bottom left) in Stadium 90 are calculated and set based on the constraint range of the top right rectangle. Specifically, for the bottom right rectangle, the constraint range is symmetrical to that of the top right rectangle across the center line in the pan direction, and the same constraint range is used for the tilt direction and zoom amount as for the top right rectangle. For the top left rectangle, the constraint range is symmetrical to that of the top right rectangle across the line connecting the two penalty marks (the points where the ball is placed during a penalty kick) in the pan direction, and the same constraint range is used for the tilt direction and zoom amount as for the top right rectangle. For the bottom left rectangle, the constraint range is point-symmetrical to that of the top right rectangle across the center mark in the pan direction, and the same constraint range is used for the tilt direction and zoom amount as for the top right rectangle. The outer parts of Stadium 90 can also be calculated and set in the same way.

[0100] The above explanation assumes that the altitude of the drone 20 is constant, but it is possible to set limit ranges even when the altitude is different. For example, the limit range in the pan direction is the same regardless of altitude, the limit range in the tilt direction widens as the altitude increases, and the limit range for the zoom amount narrows as the altitude increases.

[0101] (A-2-3. Control during aerial photography (S20 in Figure 7)) (A-2-3-1. Overview of control during aerial photography) As described above, the aerial photography control (S20 in Figure 7) is the control performed when aerial photography is carried out. In the aerial photography control, in step S201 in Figure 7, the server 40 executes control to move the drone 20 from the takeoff point Pto (Figure 8) to the target field (e.g., the stadium 90) (outbound movement control). In the following step S202, the server 40 executes control to perform aerial photography with the drone 20 in the target field (target field photography control (or aerial photography control)). Finally, in step S203, the server 40 executes control to move the drone 20 from the target field to the target landing point Pln (return movement control).

[0102] (A-2-3-2. Forward movement control (S201 in Figure 7)) As described above, in the outbound movement control, the drone 20 is moved from the takeoff point Pto to the target field (e.g., stadium 90). In the outbound movement control of this embodiment, when the drone 20 (camera 231) enters the target field (or the area above the target field), if the target field is not within the shooting range of the camera 231, the shooting direction, camera position, or zoom amount is automatically adjusted so that the target field is within the shooting range of the camera 231. Note that this control can be positioned as part of the aerial photography control (S202) and will be substantially the same.

[0103] (A-2-3-3. Aerial photography control (S202 in Figure 7)) (A-2-3-3-1. Overview) As described above, in the aerial photography control of this embodiment, aerial photography is performed by the drone 20 in the target field. In aerial photography, the flight of the drone 20 and the shooting direction and zoom amount of the camera 231 are basically controlled by manual operation by the user of the control device 30. The pan direction (horizontal direction) of the shooting direction of the camera 231 is controlled by the attitude of the drone 20 or the camera holder 232 (camera actuator). The tilt direction (vertical direction) and zoom amount of the camera 231 are controlled by the camera holder 232 (camera actuator). Furthermore, as already mentioned, in the aerial photography control of this embodiment, operation restriction control is implemented to restrict the user's operation regarding the flight area of ​​the drone 20 and the shooting direction and zoom amount of the camera 231.

[0104] (A-2-3-3-2. Aerial shooting control in normal shooting mode (Figure 16)) Figure 16 is a flowchart of the operation restriction control during normal shooting mode in this embodiment. In step S2001, the server 40 (shooting control unit 460) identifies the geofence 91 for normal shooting mode. In step S2002, it is determined whether the flight position of the drone 20 (the position of the camera 231) is within the geofence 91. If the flight position is within the geofence 91 (S2002: true), the process proceeds to step S2004. If the flight position is not within the geofence 91 (S2002: false), in step S2003, the server 40 (shooting control unit 460) restricts the flight position and notifies the user via the display unit 330 that the flight position is restricted. In addition to or instead of notification via the display unit 330, the notification may be made by sound from a speaker (not shown) or vibration from a vibration generator (the same applies to steps S2006 and S2008 described below). The case where the flight position is not within the geofence 91 includes, for example, when the user of the control device 30 makes an error in operation, or when the drone 20 moves in a way unintended by the user due to wind.

[0105] In step S2004, the server 40 (shooting control unit 460) identifies the limit range for the shooting direction and zoom amount of the camera 231 corresponding to the current position of the drone 20 (position of the camera 231). The limit range identified here is the one corresponding to the current position from those set in step S103 in Figure 7.

[0106] In step S2005, the server 40 (shooting control unit 460) determines whether the shooting direction is within the restricted range. If the shooting direction is within the restricted range (S2005: True), the process proceeds to step S2007. If the shooting direction is not within the restricted range (S2005: No), in step S2006, the server 40 restricts the shooting direction and notifies the system via the display unit 330 that the shooting direction has been restricted.

[0107] In step S2007, the server 40 determines whether the zoom amount is within the limit range. If the zoom amount is within the limit range (S2007: True), the process ends and the process returns to step S2001. If the zoom amount is not within the limit range (S2007: No), in step S2008, the server 40 limits the zoom amount and notifies the user via the display unit 330 that the zoom amount has been limited.

[0108] Furthermore, to accommodate situations where, for example, it is desired to film areas outside the target field (e.g., the stands, benches, or the coach), a function to manually release the restrictions on the shooting direction or zoom amount can be provided. Such a manual release function can be performed, for example, by operating the control device 30 (e.g., by pressing a button).

[0109] (A-2-3-3-3. Aerial photography control in the first dolly shooting mode (Figures 17-20)) Figure 17 is a flowchart of the operation restriction control during the first dolly shooting mode in this embodiment. Figures 18, 19, and 20 are the first to third explanatory diagrams of the operation restriction control during the first dolly shooting mode in this embodiment. Note that while Figures 10 to 15 showed examples of the screen of the display unit 330 of the control device 30, Figures 18 to 20 show a simplified representation of the actual drone 20. In the first dolly shooting mode, dolly shooting is possible in which the altitude of the drone 20 (or camera 231) is kept constant, only linear horizontal movement is allowed, and the pan direction is fixed to the target pan direction.

[0110] In Figure 18, the direction of linear movement of the drone 20 in the horizontal direction is indicated by arrow 700, the shooting direction of the camera 231 is indicated by arrows 701a and 701b, and the shooting area of ​​the camera 231 is indicated by symbols 702a and 702b. In Figure 18, the orientation of the shooting direction 701a and shooting area 702a in state S1 is the same as the orientation of the shooting direction 701b and shooting area 702b in state S2. Furthermore, in the first dolly shooting mode, if the drone 20 (or camera 231) becomes unable to move linearly further due to the limitations of the shooting direction (and zoom amount) of the camera 231, the user of the control device 30 can change the shooting direction of the camera 231 to enable further linear movement (details will be described later).

[0111] In step S2011 of Figure 17, the server 40 identifies the geofence 91 for the first dolly shooting mode (Figures 18 to 20). In step S2012, it is determined whether the flight position of the drone 20 (the position of the camera 231) is within the geofence 91. If the flight position is within the geofence 91 (S2012: true), the process proceeds to step S2014. If the flight position is not within the geofence 91 (S2012: false), in step S2013, the server 40 restricts the flight position to return to within the geofence 91. The case where the flight position is not within the geofence 91 includes, for example, cases where the user of the control device 30 makes an error in operation, or where the drone 20 moves in a way unintended by the user due to wind.

[0112] In step S2014, the server 40 identifies the range in which linear movement (dolly shooting) is permitted (linear movement allowable range) corresponding to the target shooting direction (target pan direction) and the current zoom amount. The linear movement allowable range is calculated from the shooting direction and zoom amount limit range set in step S103 in Figure 7 (the linear movement allowable range may also be calculated from only the shooting direction set in step S103 in Figure 7). If it is confirmed that shooting will only take place within the geofence 91, the limit range may be set only for each point within the geofence 91 (for example, points P2 and P3) in step S103.

[0113] In step S2015, the server 40 determines whether a user has requested that the drone 20 (camera 231) move outside the permissible linear movement range. This determination is made, for example, based on whether an operation command has been input to move the drone 20 further beyond the permissible linear movement range when the drone 20 is located at the edge of the permissible linear movement range. If a request to move outside the permissible linear movement range has been made (S2015: true), the process proceeds to step S2016. If no request to move outside the permissible linear movement range has been made (S2015: false), the process ends and the process returns to step S2011.

[0114] In step S2016, the server 40 determines whether a linear movement restriction release operation (hereinafter also referred to as "restriction release operation") has been performed. The restriction release operation is performed, for example, when the drone 20 is located at the edge of the linear movement allowable range, by the user changing the shooting direction of the camera 231 toward the center of the linear movement allowable range. If a restriction release operation is performed between steps S2015 and S2016, a notification that the linear movement restriction has been released may be displayed on the display unit 330 of the control device 30.

[0115] If the restriction release operation is not performed (S2016: true), in step S2017, the server 40 restricts linear movement. Figure 18 shows how the position of the drone 20 (camera 231) changes from the lower state S1 to the upper state S2 in Figure 18 due to user operation. State S2 corresponds to the case where the drone 20 is located at the edge of the allowable linear movement range (S2015: true). Therefore, in state S2, if the restriction release operation is not performed (S2016: true), the position of the drone 20 (camera 231) cannot exceed the allowable linear movement range (i.e., it cannot move above state S2) (S2017).

[0116] If a restriction removal operation is performed (S2016: false), in step S2018, the server 40 allows further linear movement beyond the permissible linear movement range. As will be described in detail below, the value of the further linear movement allowed can be changed depending on the degree of the restriction removal operation (e.g., the amount of change in the shooting direction). Figure 19 shows the transition to state S3 when the drone 20 (camera 231) is in state S2 in Figure 18, the user performs a restriction removal operation (S2016: false), and further linear movement is permitted (S2018), and the user actually performs further linear movement. Compared to state S2 in Figure 18, in state S3 in Figure 19, the shooting direction 701c of the camera 231 (and the orientation of the shooting area 702c) is directed towards the center of the permissible linear movement range.

[0117] If a further restriction removal operation is performed in state S3 in Figure 19 (S2016: false), further linear movement is permitted (S2018). Figure 20 shows the transition to state S4 when the drone 20 (camera 231) is in state S3 in Figure 19, the user performs a further restriction removal operation (S2016: false), and further linear movement is permitted (S2018), and the user actually performs further linear movement. Compared to state S3 in Figure 19, in state S4 in Figure 20, the shooting direction 701d of the camera 231 (and the orientation of the shooting area 702d) is directed towards the center of the permissible linear movement range.

[0118] (A-2-3-3-4. Aerial photography control in the second dolly shooting mode (Figure 21)) Figure 21 is a flowchart of the operation restriction control during the second dolly shooting mode in this embodiment. In the operation restriction control during the first dolly shooting mode (Figure 17), when a request to move outside the allowable linear movement range was made (S2015: true), the entity that performed the restriction release operation was the user of the control device 30. In contrast, in the operation restriction control during the second dolly shooting mode (Figure 21), when a request to move outside the allowable linear movement range was made (S2025: true), the entity that performed the restriction release operation was the server 40.

[0119] Steps S2021, S2022, S2023, S2024, and S2025 in Figure 21 are the same as steps S2011, S2012, S2013, S2014, and S2015 in Figure 17. When a request is made to move outside the permissible linear movement range (S2025: true), in step S2026, the server 40 automatically performs a restriction release operation according to the amount of further linear movement of the drone 20 requested by the user. That is, as the amount of further linear movement increases, the amount of correction of the shooting direction increases (see Figures 19 and 20). Therefore, even in the second dolly shooting mode (Figure 21), the change from state S2 in Figure 18 → state S3 in Figure 19 → state S4 in Figure 20 is possible. However, as described above, it is the server 40, not the user, that corrects the shooting direction. Note that when the amount of further linear movement reaches the maximum value, no further linear movement is permitted.

[0120] Conversely, when the amount of further linear movement decreases while the drone 20 is outside the allowable linear movement range (in other words, when the user returns the drone 20 to the allowable linear movement range), the server 40 automatically reduces the amount of correction to the shooting direction, bringing the shooting direction in the pan direction closer to the target pan direction. For example, in state S4 of Figure 20, when the user returns the drone 20 to the allowable linear movement range, the server 40 automatically returns the shooting direction to state S3 of Figure 19, for example. Furthermore, in state S3 of Figure 19, when the user returns the drone 20 to the allowable linear movement range, the server 40 automatically returns the shooting direction to state S2 of Figure 18, for example.

[0121] Furthermore, when the drone 20 is outside the allowable range of linear movement (S2025: true) and automatic correction of the shooting direction (S2026) is being performed, in step S2027, the server 40 determines whether the user has performed an operation to return the shooting direction (pan direction) to the target shooting direction (target pan direction) (shooting direction return operation). If a shooting direction return operation is performed (S2027: true), in step S2028, the server 40 reduces the amount of linear movement according to the amount of the shooting direction return operation. For example, if a shooting direction return operation is performed in state S4 of Figure 20, the server 40 automatically returns the position of the drone 20 to state S3 of Figure 19, for example. Furthermore, if a shooting direction return operation is performed in state S3 of Figure 19, the server 40 automatically returns the position of the drone 20 to state S2 of Figure 18, for example.

[0122] [A-3. Effects of this embodiment] According to this embodiment, the shooting direction is restricted using a range of acceptable shooting directions (S103 in Figure 7, Figures 12 to 15) that is changed according to the position of the drone 20 (camera position of the mobile camera 231), and the user is notified of the range of acceptable shooting directions (S2006 in Figure 16). This makes it easier to shoot using the range of acceptable shooting directions according to the camera position when the camera 231 is mobile and the shooting direction is adjusted manually or automatically, thereby improving convenience or operability. Therefore, for example, it is possible to prevent the camera 231 from being pointed in an undesirable shooting direction.

[0123] In this embodiment, the shooting control unit 460 (shooting control device) can set an acceptable range for the shooting direction so that at least a portion of the stadium 90 (shooting target area) is included in the field of view of the camera 231 (S103 in Figure 7, Figures 12 to 15). This prevents the user or the shooting control unit 460 (shooting control device) from accidentally changing the shooting direction so that the camera 231 is pointed outside the stadium 90.

[0124] In this embodiment, the shooting system 10 has a drone 20 (unmanned aerial vehicle) that flies through the air (Figure 1), and the camera 231 is mounted on the drone 20 (Figures 2 and 3). This prevents the camera 231 from being pointed in an undesirable shooting direction during aerial photography, thereby improving convenience or operability.

[0125] In this embodiment, the shooting control unit 460 (shooting control device) restricts the shooting direction of the camera 231 by the attitude of the drone 20 (unmanned aerial vehicle) or the camera holding unit 232 (camera actuator) (S2006 in Figure 16). This makes it possible to restrict the shooting direction in a simple manner.

[0126] In this embodiment, the camera 231 can be adjusted manually or automatically in addition to the shooting direction. The shooting control unit 460 (shooting control device) sets an allowable range for the zoom amount according to the position (camera position) or tilt or pan direction of the drone 20 (S103 in Figure 7). Furthermore, the shooting control unit 460 (shooting control device) limits the zoom amount using the allowable range for the zoom amount and notifies the user of the allowable range for the zoom amount (S2008 in Figure 16). This prevents the use of undesirable camera magnification in a configuration where the camera 231 is movable and the zoom amount can be adjusted manually or automatically, thereby improving convenience or operability.

[0127] In this embodiment, in the normal shooting mode (manual shooting direction operation mode) in which the shooting direction is operated manually, the shooting control unit 460 (shooting control device) restricts the shooting direction operated by the user using an allowable range of shooting directions set according to the position of the drone 20 (camera position) (S2005, S2006 in Figure 16). This prevents the camera 231 from being pointed in an undesirable shooting direction when the user manually operates the shooting direction, thereby improving convenience or operability.

[0128] In this embodiment, in the normal shooting mode in which the zoom amount is manually controlled (manual zoom amount operation mode), the shooting control unit 460 (shooting control device) limits the amount of zoom operated by the user using an allowable range of zoom amount set according to the position of the drone 20 (camera position) (S2007, S2008 in Figure 16). This prevents undesirable camera magnification when the user manually controls the zoom amount, thereby improving convenience or operability.

[0129] In this embodiment, the shooting control unit 460 (shooting control device) can set a first dolly shooting mode and a second dolly shooting mode that enable dolly shooting in which the camera 231's altitude is kept constant, only allowing linear horizontal movement, and fixing the pan direction to the target pan direction (Figures 17 and 21). The shooting control unit 460 also sets a linear movement allowance range, which is the range in which the camera 231's horizontal linear movement is permitted in the first dolly shooting mode and the second dolly shooting mode. When the pan direction is the target pan direction, it restricts the camera 231 from moving further linearly (moving outside the linear movement allowance range) (S2017 in Figure 17, S2025 in Figure 21: false). This makes it possible to prevent the shooting range of the camera 231 from going outside the stadium 90 (outside the target field) while the pan direction remains the target pan direction in the first dolly shooting mode and the second dolly shooting mode.

[0130] In the first dolly shooting mode of this embodiment, when the camera 231 is at one end of the linear movement allowable range (S2015 in Figure 17: true), the shooting control unit 460 (shooting control device) allows the camera 231 to move further in a linear direction (move in the direction of linear movement beyond the linear movement allowable range) when a predetermined operation is performed (S2016: false). The predetermined operation is a first pan direction manual operation to bring the pan direction closer to the center of the linear movement allowable range, or a zoom out manual operation to zoom out the camera 231 (S2016). As a result, in the first dolly shooting mode, when an operation is performed to keep the shooting range of the camera 231 within, for example, the stadium 90 (within the shooting target field) (first pan direction manual operation or zoom out manual operation), the camera 231 is allowed to move further in a linear direction (move in the direction of linear movement beyond the linear movement allowable range). Therefore, it is possible to improve the operability in dolly shooting.

[0131] In the second dolly shooting mode of this embodiment, when the camera 231 moves in the direction of linear movement beyond the allowable linear movement range (S2025 in Figure 21: true), the shooting control unit 460 (shooting control device) performs a first automatic pan direction operation to bring the pan direction closer to the center of the allowable linear movement range (S2026 in Figure 21, Figures 19 and 20). This makes it possible to prevent the shooting range of the camera 231 from going, for example, outside the stadium 90 (outside the field being shot) in the second dolly shooting mode.

[0132] In the second dolly shooting mode of this embodiment, when the camera 231 is outside the linear movement allowable range (S2025 in Figure 21: true), the shooting control unit 460 (shooting control device) performs an automatic camera position operation to bring the camera 231 closer to the linear movement allowable range (S2028) when a second manual pan direction operation is performed to bring the pan direction closer to the target pan direction (S2027: true). This makes it possible to prevent the shooting range of the camera 231 from extending, for example, outside the stadium 90 (outside the shooting target area) when the pan direction of the camera 231 is brought closer to the target pan direction while the automatic restriction release operation (S2027) is being performed.

[0133] In the second dolly shooting mode of this embodiment, the shooting control unit 460 (shooting control device) performs a second automatic pan direction operation to bring the pan direction closer to the target pan direction when the amount of further movement of the camera 231 decreases (in other words, when the camera 231 returns from outside the linear movement tolerance range toward the linear movement tolerance range) (S2026 in Figure 21). As a result, when returning the shooting direction, which was restricted by exceeding the linear movement tolerance range, back to the linear movement tolerance range, the pan direction can be automatically adjusted to loosen the restriction.

[0134] In this embodiment, when the camera 231 enters the target field, the shooting control unit 460 (shooting control device) automatically adjusts the shooting direction, camera position, or zoom amount if the target field is not within the shooting range of the camera 231 so that the target field is within the shooting range of the camera 231 (S201 in Figure 7). This makes it possible to start shooting the target field smoothly.

[0135] In this embodiment, the shooting system 10 further includes a display unit 330 (notification means) that notifies the user of the permissible range of shooting direction and zoom amount (Figures 4 and 5). The display unit 330 notifies the user by display when the shooting direction or zoom amount is restricted (S2006, S2008 in Figure 16). This makes it possible to notify the user that the shooting direction or zoom amount is restricted.

[0136] In this embodiment, the imaging system 10 further includes a display unit 330 (notification means) that notifies the user of the allowable ranges of the imaging direction and the zoom amount (FIGS. 4 and 5). The display unit 330 displays information regarding the allowable range of the imaging direction or the zoom amount corresponding to the position of the drone 20 (camera position) (S2006, S2008 in FIG. 16). Thereby, it becomes easier for the user to perform camera operations.

[0137] <B. Modified Example> Note that the present invention is not limited to the above embodiment, and it is of course possible to adopt various configurations based on the description in this specification. For example, the following configurations can be adopted.

[0138] [[ID=\\(]] [B-1. Configuration] (B-1-1. Imaging System 10) The imaging system 10 of the above embodiment was for imaging a competition (such as soccer, tennis, etc.) being held in the stadium 90 (FIG. 8). However, for example, focusing on the restriction of the imaging direction or the zoom amount of the camera 231 using an allowable range changed according to the position of the camera 231, it is not limited to this. For example, the imaging target is not limited to the above competition, and can also be applied to other events (such as concerts, ceremonies, etc.) where people gather.

[0139] In the above embodiment, the imaging system 10 had the configuration shown in FIG. 1. However, for example, focusing on the restriction of the imaging direction or the zoom amount of the camera 231 using an allowable range changed according to the position of the camera 231, it is not limited to this.

[0140] Figure 22 is an overall configuration diagram of a shooting system 10A (hereinafter also referred to as "system 10A") according to the first modified example of the present invention. In system 10A shown in Figure 22, the drone 20, the control device 30, the server 40, and the base station 52 are all connected to each other so as to be able to communicate with one another via a communication network 50 such as an internet line. Here, unlike system 10 in Figure 1, the drone 20 communicates wirelessly with the internet line 50 directly using a communication method such as LTE without going through the control device 30. Therefore, the drone 20, the control device 30, and the base station 52 do not need to communicate wirelessly directly, and it is sufficient for them to be able to connect to the internet line 50 at remote locations. Therefore, this system configuration is suitable when the drone 20 and the control device 30 are located at remote locations (for example, when an operator remotely controls them).

[0141] Figure 23 is an overall configuration diagram of a shooting system 10B (hereinafter also referred to as "system 10B") according to a second modified example of the present invention. In the example of system 10B shown in Figure 23, the drone 20, the control device 30, the base station 52, and the server 40 are each connected to each other so as to be able to communicate with one another via a communication network 50 such as an internet line, and the drone 20 and the base station 52 are connected to the internet line 50 via satellite communication through a satellite 54.

[0142] Figure 24 is an overall configuration diagram of a shooting system 10C (hereinafter also referred to as "system 10C") according to a third modified example of the present invention. In the example of system 10C shown in Figure 24, an example of system redundancy is shown in which multiple servers 40 are connected to one drone 20 via multiple internet lines 50. In this case, even if a server 40 or internet line 50 malfunctions, the operation of system 10C can continue by the other redundant servers 40 and internet lines 50, thereby improving the reliability of system 10C. In addition, although the drone 20 and control device 30 in Figures 22 and 23 can be controlled even when they are in a remote location, making them suitable for remote operation, the system is not limited to this and can also be applied to visual flight where the operator manually controls the drone 20 while viewing it.

[0143] The device described in the above embodiment may be implemented as a single device, or it may be implemented as a plurality of devices (e.g., a cloud server 40, a drone 20, a control device 30) that are partially or entirely connected by a communication network 50. For example, each functional unit and storage unit of the server 40 may be implemented by being mounted on different servers 40, drones 20, and control devices 30 that are connected to each other by a communication network 50.

[0144] (B-1-2. Drone 20) In the above embodiment, the basic behavioral control of the drone 20 during shooting was performed based on operations on the control device 30 (Figure 5). However, if we focus on limiting the shooting direction or zoom amount of the camera 231 using an allowable range that is changed according to the position of the camera 231, for example, the basic behavioral control of the drone 20 may be automatic control (without manual operation) by the drone 20 itself or based on commands from the server 40.

[0145] In the above embodiment, a drone 20 was used as an example of a mobile aerial photography device, but the invention is not limited to this, for example, by focusing on the limitation of the shooting direction or zoom amount of the camera 231 using an allowable range that is changed according to the position of the camera 231. The mobile device may be, for example, a camera system that can move along a wire placed in the air within the stadium 90. Alternatively, the mobile device may be a camera system that is placed on the ground and can move (including one that moves on a rail).

[0146] (B-1-3. Control device 30) The control device 30 in the above embodiment had the configuration shown in Figures 4 and 5. However, it is not limited to this configuration, for example, by focusing on the limitation of the shooting direction or zoom amount of the camera 231 using an allowable range that is changed according to the position of the camera 231. With respect to the operation input unit 320, for example, the number and arrangement of input sticks, the number, shape and arrangement of buttons can be changed as appropriate. Alternatively, the configuration realized in the operation input unit 320 of Figure 5 can be replaced with a touch panel.

[0147] The control input unit 320 may have a takeoff button and a landing button for instructing automatic takeoff and landing, or it may have a flight start button for instructing the aircraft to automatically fly to a predetermined position and hover there, a home button for performing a return operation to the starting position, a mode switch button for switching flight modes, etc.

[0148] [B-2. Control] The series of processes described in relation to the above embodiment may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the server 40 according to this embodiment can be created and implemented on a PC or the like. A computer-readable recording medium on which such a computer program is stored can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a flash memory. Alternatively, the above computer program may be distributed without using a recording medium, for example, via a communication network 50.

[0149] The flowchart used in the above embodiment does not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0150] In the above embodiment, limiting ranges were set for both the shooting direction and zoom amount of the camera 231 (Figures 7, 16, 17, and 21). However, if we focus on limiting the shooting direction or zoom amount of the camera 231 using an allowable range that is changed according to the position of the camera 231, for example, it is also possible to set a limiting range for only one of the shooting direction or zoom amount. Furthermore, regarding the shooting direction, limiting ranges were set for both the pan direction (horizontal direction) and the tilt direction (vertical direction), but it is also possible to use only one of them.

[0151] The operation restriction control of the above embodiment enables a normal shooting mode (Figure 16), a first dolly shooting mode (Figure 17), and a second dolly shooting mode (Figure 21). However, this is not limited to this, for example, if we focus on limiting the shooting direction or zoom amount of the camera 231 using a tolerance range that is changed according to the position of the camera 231. For example, it is possible to use only one or two of the above three shooting modes. Alternatively, it is possible to use another shooting mode in addition to or instead of the above three shooting modes.

[0152] In the above embodiment, the limit ranges for shooting direction and zoom amount were set in advance for each position (camera position) of the drone 20 (in other words, for each position coordinate) (S103 in Figure 7, Figures 12 to 15). However, if we focus on limiting the shooting direction or zoom amount of the camera 231 using an allowable range that is changed according to the position of the camera 231, for example, another method can be used. For example, the relationship between the image captured by the camera 231 and the no-shooting area or marker indicating the no-shooting area may be determined for each position (camera position) of the drone 20 to determine the limit range for shooting direction or zoom amount.

[0153] Figure 25 is an explanatory diagram relating to the restriction of the shooting direction according to the first modification of the present invention. In the first modification, when setting the shooting restriction (S103 in Figure 7), the user sets the area to be photographed. The area to be photographed is set, for example, in relation to the field to be photographed. As described above, the field to be photographed may mean a two-dimensional location to be photographed (for example, the contour that defines the stadium 90 (for example, a frame consisting of the touchlines and goal lines of a soccer field)).

[0154] Next, when actually taking images using the drone 20 (S202 in Figure 7), the server 40 estimates the shooting area of ​​the camera 231 based on the current position (camera position) and orientation of the drone 20, as well as the operating state and zoom amount of the camera holder 232 (camera actuator). Then, if the estimated shooting area of ​​the camera 231 is likely to include an area other than the target area (for example, a no-shooting area), the server 40 restricts the shooting direction (pan direction and tilt direction) and zoom amount of the camera 231. In other words, it does not permit shooting directions and zoom amounts that would cause areas other than the target area to enter the camera's shooting area. Alternatively, if the ratio of areas other than the target area included in the estimated shooting area of ​​the camera 231 exceeds a predetermined value, the shooting direction and zoom amount of the camera 231 may be restricted.

[0155] Figure 25 shows a screen 66 as an example of the screen displayed on the display unit 330 of the control device 30 when the drone 20 is actually taking pictures. Screen 66 includes a stadium image 600, a drone image 661, a direction of travel image 662, an estimated shooting area image 663, and a captured image 664. The drone image 661 in Figure 25 has the same shape as the drone image 621a in Figure 12, etc., but moves within screen 66 according to the current position (camera position) of the drone 20.

[0156] The direction of travel image 662 has a shape similar to the direction of travel image 631a in Figure 13, but shows the actual direction of travel of the drone 20 (and the drone image 661). The estimated shooting area image 663 shows the shooting area of ​​the camera 231 (particularly the shooting area in the panning direction) estimated according to the position of the drone 20 (and the image 661).

[0157] The estimated captured images 634, 644, and 654 in Figures 13 to 15 were estimated captured images (simulated images). In contrast, the captured image 664 in Figure 25 is an actual image captured by camera 231. A boundary image 665 indicating the boundary of the area to be captured is superimposed on the captured image 644. The boundary of the area to be captured is determined, for example, by detecting features of the stadium 90 (goal line, touchline, etc.) from the captured image 644 and determining it based on the position, angle, etc., of these features in the captured image 644.

[0158] The shooting control unit 460 of the server 40 then calculates the ratio R1 between the area to be photographed and the other area in the captured image 664, and determines that the shooting direction has exceeded the restricted range when the ratio R1 exceeds a predetermined threshold (first ratio threshold). It then displays a message indicating that the shooting direction has exceeded the restricted range on the display unit 330 of the control device 30. In addition to or instead of notification via the display unit 330, the notification may also be given by sound from a speaker (not shown) or vibration from a vibration generator.

[0159] Furthermore, the determination of whether the shooting direction exceeds the restricted range may be made by a method other than using the ratio R1 of the target area to the other area in the captured image 664. For example, the determination may be made based on the outer edge information of the target area entered by the user. With regard to the outer edge information of the target area referred to herein, for example, if the position of the outer edge of the target area falls within a predetermined area in the captured image 664, or if the outer edge of the target area exists within the captured image 664, it may be determined that the shooting direction has exceeded the restricted range. Alternatively, if the area value or ratio of the target area in the captured image 665 falls below a predetermined threshold, or if the area value or ratio of the area other than the target area in the captured image 665 exceeds a predetermined threshold, it may be determined that the shooting direction has exceeded the restricted range.

[0160] Figure 26 is an explanatory diagram relating to the restriction of the shooting direction according to a second modification of the present invention. In the second modification, when setting the shooting restriction (S103 in Figure 7), the user sets a no-shooting area. The no-shooting area is defined, for example, using markers. As such markers, for example, contours defining the stadium 90 (touchlines, goal lines), corner flags, goalposts, colored marks, or 3D codes can be used.

[0161] Next, when actually taking images using the drone 20 (S202 in Figure 7), the server 40 determines whether or not the marker is present in the image captured by the camera 231. If a marker is detected in the image, the server 40 restricts the shooting direction (pan and tilt directions) and zoom amount of the camera 231. In other words, if a marker is detected, further changes in the shooting direction and reductions in the zoom amount are not permitted.

[0162] Figure 26 shows a screen 67 as an example of the screen displayed on the display unit 330 of the control device 30 when actually taking pictures using the drone 20. Screen 67 includes a stadium image 600, a drone image 661, a direction of travel image 662, an estimated shooting area image 663, a no-shooting area image 671, and a captured image 672. The drone image 661, direction of travel image 662, and estimated shooting area image 663 in Figure 26 are the same as those in Figure 25.

[0163] The no-photography zone image 671 indicates the no-photography zone. The no-photography zone is set by the user, for example, in step S103 in Figure 7. Similar to the captured image 664 in Figure 25, the captured image 672 in Figure 26 is an actual image captured by the camera 231. A boundary image 673 indicating the boundary of the no-photography zone is superimposed on the captured image 672. The boundary of the no-photography zone is determined, for example, by detecting features of the stadium 90 (goal line, touchline, etc.) from the captured image 644 and determining the boundary based on the position, angle, etc., of those features in the captured image 672.

[0164] The shooting control unit 460 of the server 40 calculates the ratio R2 of the prohibited shooting area to the entire captured image 664, and determines that the shooting direction has exceeded the restricted range when the ratio R2 exceeds a predetermined threshold (second ratio threshold). It then displays a message indicating that the shooting direction has exceeded the restricted range on the display unit 330 of the control device 30. In addition to or instead of notification via the display unit 330, the notification may also be given by sound from a speaker (not shown) or vibration from a vibration generator. The determination of whether or not the shooting direction has exceeded the restricted range may be made by a method other than using the ratio R2 of the prohibited shooting area to the entire captured image 664, as in the first modified example (Figure 25). [Explanation of Symbols]

[0165] 10, 10A, 10B, 10C… Shooting System 20… Drone (unmanned aerial vehicle) 90… Stadium (area of ​​focus for photography) 231... Camera for filming (mobile camera) 232...Camera holder (camera actuator) 330...Display section (notification means) 460... Shooting control unit (shooting control device)

Claims

1. A mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting system comprising, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, the shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. Furthermore, the aforementioned shooting control device, In the dolly shooting mode, a linear movement tolerance range is set, which is the range in which the camera's horizontal linear movement is permitted. At least when the panning direction is the target panning direction, the camera is restricted from moving outside the allowable linear movement range. A shooting system characterized by the following features.

2. In the imaging system according to claim 1, In the dolly shooting mode, when a predetermined operation is performed while the camera is at one end of the linear movement allowable range, the shooting control device allows the camera to move beyond the linear movement allowable range in the direction of linear movement. The aforementioned predetermined operation is, A first manual operation of the pan direction to bring the pan direction closer to the center of the allowable linear movement range, or Manual zoom-out operation to zoom out the aforementioned camera A shooting system characterized by the following:

3. A mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting system comprising, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, the shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. In the dolly shooting mode, when the camera moves in the direction of linear movement beyond the allowable linear movement range, the shooting control device shall A first automatic pan direction operation to bring the pan direction closer to the center of the allowable linear movement range, or Automatic zoom-out operation to zoom out the aforementioned camera Perform A shooting system characterized by the following features.

4. In the imaging system described in claim 3, In the dolly shooting mode, when the camera is outside the linear movement allowable range, the shooting control device performs an automatic camera position operation to bring the camera closer to the linear movement allowable range if a second manual panning operation is performed to bring the panning direction closer to the target panning direction. A shooting system characterized by the following features.

5. In the imaging system described in claim 3, In the dolly shooting mode, the shooting control device performs a second automatic pan direction operation to bring the pan direction closer to the target pan direction when the camera returns from outside the linear movement allowable range toward the linear movement allowable range. A shooting system characterized by the following features.

6. A mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting system comprising, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, when the camera enters the target area, the shooting control device automatically adjusts the shooting direction, camera position, or zoom amount of the camera so that the target area is included in the camera's shooting range if the target area is not within the camera's shooting range. A shooting system characterized by the following features.

7. A mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting system comprising, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. The camera allows for manual or automatic adjustment of the zoom amount in addition to the shooting direction. The aforementioned imaging control device is The allowable range of the zoom amount is set according to the camera position or the tilt direction or the pan direction. The zoom amount is limited using the aforementioned allowable range of the zoom amount, or the user is notified of the aforementioned allowable range of the zoom amount. Furthermore, the aforementioned shooting control device, The ratio of the area to be photographed and the area outside the area to be photographed in the image captured by the camera, The outer edge information of the area to be photographed, entered by the user, The value or percentage of the area of ​​the target region in the captured image, or The area value or percentage of the region outside the target area in the aforementioned captured image. Based on at least one of the following pieces of information, the allowable range of the shooting direction or the zoom amount is set. A shooting system characterized by the following features.

8. A mobile camera whose shooting direction, including at least one of the pan or tilt directions, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting method using a shooting system equipped with, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, the shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. Furthermore, the aforementioned shooting control device, In the dolly shooting mode, a linear movement tolerance range is set, which is the range in which the camera's horizontal linear movement is permitted. At least when the panning direction is the target panning direction, the camera is restricted from moving outside the allowable linear movement range. A photographic method characterized by the following:

9. In the photographic method described in Claim 8, In the dolly shooting mode, when a predetermined operation is performed while the camera is at one end of the linear movement allowable range, the shooting control device allows the camera to move beyond the linear movement allowable range in the direction of linear movement. The aforementioned predetermined operation is, A first manual operation of the pan direction to bring the pan direction closer to the center of the allowable linear movement range, or Manual zoom-out operation to zoom out the aforementioned camera A photographic method characterized by the following:

10. A mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting method using a shooting system equipped with, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, the shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. In the dolly shooting mode, when the camera moves in the direction of linear movement beyond the allowable linear movement range, the shooting control device shall A first automatic pan direction operation to bring the pan direction closer to the center of the allowable linear movement range, or Automatic zoom-out operation to zoom out the aforementioned camera Perform A photographic method characterized by the following:

11. In the photographing method described in Claim 10, In the dolly shooting mode, when the camera is outside the linear movement allowable range, the shooting control device performs an automatic camera position operation to bring the camera closer to the linear movement allowable range if a second manual panning operation is performed to bring the panning direction closer to the target panning direction. A photographic method characterized by the following:

12. In the photographing method described in Claim 10, In the dolly shooting mode, the shooting control device performs a second automatic pan direction operation to bring the pan direction closer to the target pan direction when the camera returns from outside the linear movement allowable range toward the linear movement allowable range. A photographic method characterized by the following:

13. A mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting method using a shooting system equipped with, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, when the camera enters the target area, the shooting control device automatically adjusts the shooting direction, camera position, or zoom amount of the camera so that the target area is included in the camera's shooting range if the target area is not within the camera's shooting range. A photographic method characterized by the following:

14. A mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, A shooting control device that controls the shooting of the aforementioned camera and A shooting method using a shooting system equipped with, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. The camera allows for manual or automatic adjustment of the zoom amount in addition to the shooting direction. The aforementioned imaging control device is The allowable range of the zoom amount is set according to the camera position or the tilt direction or the pan direction. The zoom amount is limited using the aforementioned allowable range of the zoom amount, or the user is notified of the aforementioned allowable range of the zoom amount. Furthermore, the aforementioned shooting control device, The ratio of the area to be photographed and the area outside the area to be photographed in the image captured by the camera, The outer edge information of the area to be photographed, entered by the user, The value or percentage of the area of ​​the target region in the captured image, or The area value or percentage of the region outside the target area in the aforementioned captured image. Based on at least one of the following pieces of information, the allowable range of the shooting direction or the zoom amount is set. A photographic method characterized by the following:

15. A shooting control device for controlling the shooting of a mobile camera, in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, the shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. Furthermore, the aforementioned shooting control device, In the dolly shooting mode, a linear movement tolerance range is set, which is the range in which the camera's horizontal linear movement is permitted. At least when the panning direction is the target panning direction, the camera is restricted from moving outside the allowable linear movement range. A photographic control device characterized by the following:

16. In the photographic control device according to claim 15, In the dolly shooting mode, when a predetermined operation is performed while the camera is at one end of the linear movement allowable range, the shooting control device allows the camera to move beyond the linear movement allowable range in the direction of linear movement. The aforementioned predetermined operation is, A first manual operation of the pan direction to bring the pan direction closer to the center of the allowable linear movement range, or Manual zoom-out operation to zoom out the aforementioned camera A photographic control device characterized by the following:

17. A shooting control device for controlling the shooting of a mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, the shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. In the dolly shooting mode, when the camera moves in the direction of linear movement beyond the allowable linear movement range, the shooting control device shall A first automatic pan direction operation to bring the pan direction closer to the center of the allowable linear movement range, or Automatic zoom-out operation to zoom out the aforementioned camera Perform A photographic control device characterized by the following:

18. In the photographic control device according to claim 17, In the dolly shooting mode, when the camera is outside the linear movement allowable range, the shooting control device performs an automatic camera position operation to bring the camera closer to the linear movement allowable range if a second manual panning operation is performed to bring the panning direction closer to the target panning direction. A photographic control device characterized by the following:

19. In the photographic control device according to claim 17, In the dolly shooting mode, the shooting control device performs a second automatic pan direction operation to bring the pan direction closer to the target pan direction when the camera returns from outside the linear movement allowable range toward the linear movement allowable range. A photographic control device characterized by the following:

20. A shooting control device for controlling the shooting of a mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. Furthermore, when the camera enters the target area, the shooting control device automatically adjusts the shooting direction, camera position, or zoom amount of the camera so that the target area is included in the camera's shooting range if the target area is not within the camera's shooting range. A photographic control device characterized by the following:

21. A shooting control device for controlling the shooting of a mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, The aforementioned imaging control device is The allowable range of the shooting direction is changed according to the absolute position of the camera, which indicates the latitude and longitude of the camera, or the camera position, which is the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point. The shooting direction is restricted using the aforementioned tolerance range, or the user is notified of the aforementioned tolerance range for the shooting direction. The camera allows for manual or automatic adjustment of the zoom amount in addition to the shooting direction. The aforementioned imaging control device is The allowable range of the zoom amount is set according to the camera position or the tilt direction or the pan direction. The zoom amount is limited using the aforementioned allowable range of the zoom amount, or the user is notified of the aforementioned allowable range of the zoom amount. Furthermore, the aforementioned shooting control device, The ratio of the area to be photographed and the area outside the area to be photographed in the image captured by the camera, The outer edge information of the area to be photographed, entered by the user, The value or percentage of the area of ​​the target region in the captured image, or The area value or percentage of the region outside the target area in the aforementioned captured image. Based on at least one of the following pieces of information, the allowable range of the shooting direction or the zoom amount is set. A photographic control device characterized by the following:

22. A program executed in a shooting control device that controls the shooting of a mobile camera whose shooting direction, including at least one of the pan direction or tilt direction, can be manually or automatically adjusted by the user, In the aforementioned imaging control device, The steps include changing the allowable range of the shooting direction according to the camera position, which is the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point, A step of restricting the shooting direction using the aforementioned tolerance range, or a step of notifying the user of the aforementioned tolerance range for the shooting direction. Execute, The aforementioned shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. Furthermore, in the aforementioned shooting control device, The steps include setting a linear movement tolerance range, which is the range in which the camera's horizontal linear movement is permitted in the dolly shooting mode, The steps include: restricting the camera from moving outside the linear movement allowable range, at least when the panning direction is the target panning direction; A program characterized by executing [this].

23. In the program described in Claim 22, In the aforementioned shooting control device, when the dolly shooting mode is activated and a predetermined operation is performed while the camera is at one end of the linear movement tolerance range, the device performs a step that allows the camera to move beyond the linear movement tolerance range in the direction of the linear movement, The aforementioned predetermined operation is, A first manual operation of the pan direction to bring the pan direction closer to the center of the allowable linear movement range, or Manual zoom-out operation to zoom out the aforementioned camera A program characterized by the following:

24. A program executed in a shooting control device that controls shooting of a mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, In the aforementioned imaging control device, The steps include changing the allowable range of the shooting direction according to the camera position, which is the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point, A step of restricting the shooting direction using the aforementioned tolerance range, or a step of notifying the user of the aforementioned tolerance range for the shooting direction. Execute, The aforementioned shooting control device can set a dolly shooting mode that allows only linear horizontal movement while keeping the camera's altitude constant, and enables dolly shooting in which at least the pan direction is fixed to the target pan direction. In the aforementioned shooting control device, when the dolly shooting mode is activated, if the camera moves in the direction of linear movement beyond the allowable linear movement range, A first automatic pan direction operation to bring the pan direction closer to the center of the allowable linear movement range, or Automatic zoom-out operation to zoom out the aforementioned camera Perform the steps to do so. A program characterized by the following features.

25. In the program described in Claim 24, In the aforementioned shooting control device, when the dolly shooting mode is active and the camera is outside the linear movement allowable range, if a second manual panning operation is performed to bring the panning direction closer to the target panning direction, the device executes a step of performing an automatic camera position operation to bring the camera closer to the linear movement allowable range. A program characterized by the following features.

26. In the program described in claim 24, In the aforementioned shooting control device, in the dolly shooting mode, when the camera returns from outside the linear movement allowable range toward the linear movement allowable range, a second automatic pan direction operation is performed to bring the pan direction closer to the target pan direction. A program characterized by the following features.

27. ​​A program executed in a shooting control device that controls shooting of a mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, In the aforementioned imaging control device, The steps include changing the allowable range of the shooting direction according to the camera position, which is the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point, A step of restricting the shooting direction using the aforementioned tolerance range, or a step of notifying the user of the aforementioned tolerance range for the shooting direction, When the camera enters the area to be photographed, if the area to be photographed is not within the camera's shooting range, the camera automatically adjusts the shooting direction, camera position, or zoom amount of the camera so that the area to be photographed is within the camera's shooting range. A program characterized by executing [this].

28. A program executed in a shooting control device that controls shooting of a mobile camera in which the shooting direction, including at least one of the pan direction or the tilt direction, can be manually or automatically adjusted by the user, In the aforementioned imaging control device, The steps include changing the allowable range of the shooting direction according to the camera position, which is the absolute position indicating the latitude and longitude of the camera or the relative position of the mobile camera with respect to a two-dimensional or three-dimensional target area or target point, A step of restricting the shooting direction using the aforementioned tolerance range, or a step of notifying the user of the aforementioned tolerance range for the shooting direction. Execute, The camera allows for manual or automatic adjustment of the zoom amount in addition to the shooting direction. Furthermore, in the aforementioned shooting control device, The steps include setting an allowable range for the zoom amount according to the camera position or the tilt direction or the pan direction, A step of limiting the zoom amount using the allowable range of the zoom amount, or a step of notifying the user of the allowable range of the zoom amount. Execute, Furthermore, in the aforementioned shooting control device, The ratio of the area to be photographed and the area outside the area to be photographed in the image captured by the camera, The outer edge information of the area to be photographed, entered by the user, The value or percentage of the area of ​​the target region in the captured image, or The area value or percentage of the region outside the target area in the aforementioned captured image. The step of setting the allowable range of the shooting direction or the zoom amount based on at least one of the following pieces of information is performed. A program characterized by the following features.

Images