Object imaging system, control device, control method in object imaging system, and computer program

Abstract

To enable aerial imaging with a simpler method.SOLUTION: An object imaging system comprises: a flying body 10 which has an imaging device 20 that performs imaging within a prescribed visual field to output imaging data, and can move to a prescribed position; a second communication device 50 which detects a subject position of a subject; and a remote controller 30 which controls movement to the prescribed position of the flying body 10. The remote controller 30 controls the movement of the flying body 10 in a plurality of mutually-different control modes in different time zones during the flight of the flying body 10, controls the flight of the flying body 10 so as to fly in a posture depending on the subject position in a first control mode of the plurality of control modes, and controls the flight of the flying body 10 so as to move in a prescribed posture that is independent of the subject position in a second control mode of the plurality of control modes.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an object photographing system, a control device, a control method in the object photographing system, and a computer program.

Background Art

[0002] The present invention relates to an object photographing system, a control method in the object photographing system, and a computer program, and preferably relates to an object photographing system using a flying object such as a drone, a control method in the object photographing system, and a computer program.

[0003] In this specification, the term "drone" is used to mean a moving object that can be operated without a passenger (for example, an aircraft, a vehicle, a watercraft, a submersible, etc. that can be remotely wirelessly operated or autonomously operated).

[0004] As technologies related to a system for photographing an object from the air with a camera mounted on a flying object, particularly a system for souvenir photography by travelers at a tourist destination, there are those described in Patent Documents 1 to 3.

[0005] Patent Document 1 discloses a system in which a moving object equipped with a photographing device moves to a user in response to an operation of a user terminal and performs souvenir photography. Patent Document 2 discloses a remote operation video recording device that photographs a user located at a photographing point by an operation terminal operated by a user and a drone with a camera. Patent Document 3 discloses a system that transmits various instruction data to a flying object via a user terminal and controls the flying object and devices mounted on the flying object according to those various instruction data.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

[0007] According to the technologies disclosed in Patent Documents 1 to 3, a user operates their own terminal device to control the movement of a flying object and the operation of a camera mounted on the flying object.

[0008] However, most ordinary people do not have the skills to appropriately control the operation of a flying object or a camera by operating their own terminal device. For example, when wanting to take commemorative photos for sightseeing as described in Patent Document 1, it is almost impossible for most tourists to operate the terminal by themselves to control the flying object and take good photos or videos.

[0009] On the other hand, if high-quality commemorative photography can be carried out in an extraordinary way such as aerial photography as professionals do, it will surely increase the pleasure of travel for tourists, and the taken videos and photos will become valuable souvenirs.

[0010] Therefore, one aspect of the object of the present invention is to enable aerial photography in a simpler way.

[0011] Another aspect of the object of the present invention is to enable commemorative photography for tourists using a moving object represented by a flying object in a simpler way.

[0012] Other objects will become apparent from the description of the embodiments below. [Means for Solving the Problems]

[0013] To solve the above problems, an object photographing system 1 according to one aspect of the present invention is an object photographing system that photographs an object using a moving body, and has an imaging device that images within a predetermined field of view and outputs imaging data, a moving body that can move to a predetermined position, a detection device that detects the object position of the object, and a control device that controls the movement of the moving body to the position. The control device controls the movement of the moving body in a plurality of different control modes at different time zones during the movement of the moving body. In the first control mode among the plurality of control modes, the movement of the moving body is controlled so as to move while taking a posture depending on the object position. In the second control mode among the plurality of control modes, the movement of the moving body is controlled so as to move while taking a predetermined posture independent of the object position.

Effect of the Invention

[0014] According to the present invention, it is possible to realize an object photographing system, a control device, a control method in the object photographing system, and a computer program that enable aerial photography in a simpler method.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and not all of the elements and combinations thereof described in the embodiments are essential for the solution means of the invention.

[0017] In the drawings for explaining the embodiments, the same reference numerals are given to portions having the same function, and repeated explanations thereof are omitted.

[0018] Also, in the following description, the expression "xxx data" may be used as an example of information, but the data structure of the information may be any kind. That is, in order to indicate that the information does not depend on the data structure, "xxx data" can be referred to as "xxx table". Furthermore, "xxx data" may simply be referred to as "xxx". And in the following description, the configuration of each information is an example, and the information may be divided and held, or combined and held.

[0019] Note that in the following description, the "program" may be used as the subject to explain the processing. However, the program is executed by a processor (e.g., CPU (Central Processing Unit)) to perform the defined processing while appropriately using storage resources (e.g., memory) and / or communication interface devices (e.g., ports). Therefore, the subject of the processing may be the program. The processing described with the program as the subject may also be the processing performed by a processor or a computer having that processor.

[0020] Note that in the following description, when the operating entity is described as "○○ unit is", it means that the processor reads the processing content of the ○○ unit which is a program from the memory, loads it into the memory, and then realizes the function of the ○○ unit (details will be described later).

[0021] <Overview of the Object Photographing System> FIG. 1 is a diagram for explaining the overview of the object photographing system according to the present embodiment.

[0022] In FIG. 1, the object photographing system 1 of the present embodiment generally includes an aircraft 10 having an imaging device 20, a remote controller (hereinafter simply referred to as a “remote control”) 30 that communicates with the aircraft 10 by means such as wireless communication to control the flight of the aircraft 10, a first communication device 40 provided so as to be able to exchange information with the remote control 30, and a second communication device 50 configured to be communicable with the first communication device 40.

[0023] The aircraft 10 is an example of a moving body in the object photographing system 1 of the present embodiment. It flies along a predetermined flight path (moving path) in the air, and causes the imaging device 20 to image a background, a landscape, and a subject (object) who is a service user possessing the second communication device 50, and sends the captured imaging data (either a still image or a moving image) to the remote control 30. The aircraft 10 is a drone as an example.

[0024] The imaging device 20 has an imaging element such as a CMOS and a photographing system such as a lens, images an object within a field of view in a predetermined field of view direction (more precisely, an object within the field of view), and outputs the result of imaging the field of view as imaging data. The imaging device 20 is a video camera as an example. The imaging device 20 is fixed to the aircraft 10 and is supported so as to be able to pitch up and down with respect to the aircraft 10 (arrow A in FIG. 1). The pitch angle of the imaging device 20 with respect to the aircraft 10 can be changed by a support / drive mechanism of the aircraft 10 that supports the imaging device 20.

[0025] The remote controller 30 causes the flying object 10 to fly along a predetermined flight path (that is, controls the flight of the flying object 10). Although details will be described later, the remote controller 30 has a flight plan regarding the flight path of the flying object 10 and controls the flight of the flying object 10 based on this flight plan. The flight plan has information for designating the above-described flight path. There is no particular limitation on the form of this information, but as an example, a plurality of points (that is, positions through which the flying object 10 should pass) are set on the flight path, and these points are sequentially transmitted to the flying object 10, and the flying object 10 collates its own position information and flies toward the points. Alternatively, as information for designating the flight path, the flight direction and the flight time are sequentially transmitted to the flying object 10, and the flying object 10 controls its own flight based on this information. Although there is no particular limitation on the details of the flight plan, in this embodiment, the following description will be made assuming that the flight plan has information regarding a plurality of points set on the flight path. As information regarding the points, there is generally information regarding latitude, longitude, and altitude, but there may also be information for designating the relative three-dimensional position from the departure point of the flying object 10.

[0026] In addition, the flight plan in the object photographing system 1 of this embodiment includes information for designating the nose direction of the flying object 10 and the pitch angle of the imaging device 20 with respect to the flying object 10. The information for designating the nose direction and the pitch angle may be designated in units of the above-described points or may be designated at predetermined time intervals. In the present embodiment described below, it is assumed that the information for designating the nose direction and the pitch angle is designated in units of points. When the imaging device 20 is attached to the flying object 10 via a so-called gimbal, the pitch angle is the angle of the gimbal.

[0027] In the object photographing system 1 of this embodiment, the flight plan has a landscape mode (second control mode) and a subject mode (first control mode). The landscape mode is a flight plan for flying the aircraft 10 along a predetermined flight path regardless of the position of the subject (independently of the position of the subject). The subject mode is a flight plan for flying the aircraft 10 along a flight path based on the flight position, the nose direction (the flight direction of the aircraft 10), and the pitch angle of the imaging device 20 that depends on the position of the subject.

[0028] The flight plan is created prior to the operation of flying the aircraft 10 while imaging with the imaging device 20 (hereinafter referred to as demonstration flight). The method of creating the flight plan is arbitrary. However, in the object photographing system 1 of this embodiment, the aircraft 10 acquires its own position information (latitude, longitude, altitude) while flying the aircraft 10 with the remote controller 30 (hereinafter referred to as learning flight), and the remote controller 30 receives this position information and creates a flight plan based on the position information. At this time, it is arbitrary whether the aircraft 10 sequentially transmits its own position information to the remote controller 30 or, alternatively, transmits the position information of the aircraft 10 to the remote controller 30 all at once after the learning flight is completed. Alternatively, there is also a method in which an operator who operates the remote controller 30 designates a flight path on a map such as an aeronautical chart regarding the area where the aircraft 10 flies, and the remote controller 30 creates a flight plan based on the designated flight path.

[0029] When performing learning flight, control is also performed from the remote controller 30 regarding the nose direction of the aircraft 10 and the pitch angle of the imaging device 20. Similar to the position information of the aircraft 10, information regarding the nose direction and pitch angle during actual learning flight is also transmitted to the remote controller 30. Also, when performing learning flight, it is specified from the remote controller 30 whether the flight plan is in the scenery mode or the subject mode in the current flight path, and information regarding which mode each point belongs to is also included in the flight plan by learning flight. Note that in the subject mode, since the actual nose direction of the aircraft 10 and the pitch angle of the imaging device 20 are determined depending on the position information of the subject (details will be described later), it is not necessary to specify information regarding the nose direction of the aircraft 10 and the pitch angle of the imaging device 20 in the subject mode.

[0030] In the flight plan of the object photographing system 1 of the present embodiment, the scenery mode and the subject mode are executed sequentially and continuously. Which of the scenery mode and the subject mode is executed first is arbitrary.

[0031] The first communication device 40 is an information processing terminal having a communication function such as a smartphone or a tablet terminal, and is possessed by the service provider P. The first communication device 40 can communicate with the remote controller 30 by wired or wireless communication (including wireless LAN, short-range communication, mobile communication). Also, the first communication device 40 is configured to be able to communicate with the second communication device 50 by wireless communication (including wireless LAN, mobile communication), and further, the first communication device 40 is connected to an external content server 60 by wireless communication (including mobile communication).

[0032] After the demonstration flight is completed, the remote controller 30 receives imaging data (still image data, moving image data) from the imaging device 20 of the aircraft 10 and sends this imaging data to the first communication device 40. The first communication device 40 sends the imaging data sent from the remote controller 30 to the content server 60, and the content server 60 stores this imaging data in itself. Also, the content server 60 sends this imaging data in response to a request from the user terminal 70 possessed by the subject.

[0033] The second communication device 50 is also an information processing terminal having a communication function such as a smartphone or a tablet terminal, and is possessed by the subject O who is a service user. The second communication device 50 is configured to be communicable with the first communication device 40 by wireless communication (including wireless LAN and mobile communication).

[0034] In the object photographing system 1 of this embodiment, the remote controller 30 holds a flight plan, and based on this flight plan, performs flight control (including control of the nose direction) of the flying object 10 and control of the pitch angle of the imaging device 20. However, the first communication device 40 has a function of performing such control, and the remote controller 30 is also configured to execute a function of sending information regarding the points of the flight path, information regarding the nose direction of the flying object 10, and information regarding the pitch angle of the imaging device 20 based on an instruction from the first communication device 40.

[0035] <Configuration of Object Photographing System> FIG. 2 is a block diagram showing a schematic configuration of the remote controller 30, the first communication device 40, and the second communication device 50 that constitute the object photographing system 1 of this embodiment.

[0036] The remote controller 30 is a device capable of various information processes, and as an example, is an information processing terminal having a computing function. The remote controller 30 has a processor 31 and a memory 32.

[0037] The processor 31 is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), etc. The memory 32 has, for example, a magnetic storage medium such as an HDD (Hard Disk Drive), a semiconductor storage medium such as a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), etc. Also, a combination of an optical disk such as a DVD (Digital Versatile Disk) and an optical disk drive is used as a memory. In addition, known storage media such as magnetic tape media are also used as memories.

[0038] The memory 32 stores programs such as firmware. When the remote controller 30 starts operating (for example, when the power is turned on), a program such as firmware is read from this memory 32 and executed to perform overall control of the remote controller 30. Also, in addition to programs, data etc. necessary for each process of the remote controller 30 are stored in the memory 32.

[0039] Also, the remote controller 30 has a display unit 33, an input unit 34, a first communication unit 35, and a second communication unit 36.

[0040] The display unit 33 is, for example, a liquid crystal (including organic EL) display, etc., and displays an operation screen etc. of the remote controller 30 based on a screen display control signal sent from the processor 31. The input unit 34 is, for example, a touch panel, a keyboard, a mouse, etc. provided overlaid on the display screen of the display unit 33, and accepts an operation input signal based on an operation by an operator of the remote controller 30. In addition, the input unit 34 includes a joystick, etc., and accepts an operation input signal based on an operation instruction in the three-dimensional flight direction by the operator of the remote controller 30 with respect to the aircraft 10.

[0041] The first communication unit 35 performs wireless communication with the flying object 10. The wireless communication between the first communication unit 35 and the flying object 10 is preferably based on a wireless communication standard commonly used for communication with drones, but there is no particular limitation, mainly using a wireless communication standard that does not require a license as a wireless station.

[0042] The second communication unit 36 performs data transmission and reception with the first communication device 40. There is no particular limitation on the means of data transmission and reception between the second communication unit 36 and the first communication device 40, and either a wired connection based on a data communication standard such as USB, or a wireless connection based on short-range communication such as Bluetooth (registered trademark) or wireless LAN such as WiFi (registered trademark) can be used.

[0043] The processor 31 of the remote controller 30 has a flight control unit 31a and a flight plan creation unit 31b as function realization units.

[0044] The flight control unit 31a controls the flying object 10 to fly according to the flight plan 32a (details will be described later) stored in the memory 32. That is, the flight control unit 31a controls the flying object 10 in the demonstration flight. The flight plan creation unit 31b controls the flying object 10 to fly based on an instruction from the input unit 34 of the remote controller 30. That is, the flight plan creation unit 31b controls the flying object 10 in the learning flight. Specific operations of the flight control unit 31a and the flight plan creation unit 31b will be described later with reference to the flowchart.

[0045] Also, the memory 32 of the remote controller 30 stores a flight plan 32a and subject position information 32d.

[0046] The flight plan 32a includes a route information table 32b storing information on the flight route of the flying object 10, and a point information table 32c storing information on points specifying individual flight routes (routes) stored in the route information table 32b. Details of the route information table 32b and the point information table 32c will be described later with reference to FIGS. 4 and 5.

[0047] The photographed object position information 32d is the position information (latitude, longitude, altitude) of the second communication device 50 acquired by the position acquisition unit 51 of the second communication device 50. Since the second communication device 50 is held by the photographed object who is the service user (see FIG. 1), the position information acquired by the position acquisition unit 51 may be regarded as the position information of the photographed object.

[0048] The first communication device 40 communicates with the second communication unit 36 of the remote controller 30 and the second communication device 50 to transmit and receive data.

[0049] The second communication device 50 communicates with the first communication device 40 to transmit and receive data. The second communication device 50 includes a position acquisition unit 51 and a display unit 52. The position acquisition unit 51 has position information acquisition means such as a GPS module, etc., acquires the position information (latitude, longitude, altitude) of the second communication device 50, and sends this position information to the first communication device 40 (and the remote controller 30 via the first communication device 40). The transmission frequency of the position information of the second communication device 50 is arbitrary, but in the object photographing system 1 of this embodiment, after communication with the first communication device 40 is established, the position information of the second communication device 50 in real time is sent to the first communication device 40 at a short time interval (for example, 0.2 seconds interval) such that the remote controller 30 can acquire it. The display unit 52 is, for example, a liquid crystal (including organic EL) display, etc., and displays an operation screen, etc. based on a screen display control signal sent from a processor (not shown).

[0050] FIG. 3 is a block diagram showing the schematic configuration of the flying object 10 constituting the object photographing system 1 of this embodiment.

[0051] The flying object 10 includes a drive unit 11, a drive control unit 12, a position acquisition unit 13, an imaging device control unit 14, an imaging device attitude control unit 15, and a communication unit 16.

[0052] The driving unit 11 moves the flying object 10 to a predetermined position (latitude, longitude, altitude) in the air. As an example, the driving unit 11 includes a plurality (preferably four or more) of propellers and a propeller driving mechanism that rotates the propellers to impart a predetermined buoyancy and propulsion force to the flying object 10. Naturally, the configuration of the driving unit 11 may be appropriately selected from well-known ones according to the flight capabilities required of the flying object 10.

[0053] The drive control unit 12 performs drive control of the drive unit 11 based on information for performing flight control of the flying object 10 (for example, position information of points provided on the flight path of the flying object 10) sent from the remote controller 30, and flies the flying object 10 to a position based on an instruction from the remote controller 30. Further, the drive control unit 12 performs drive control of the drive unit 11 based on information regarding the nose direction of the flying object 10 sent from the remote controller 30, and controls the nose direction of the flying object 10.

[0054] The position acquisition unit 13 has position information acquisition means such as a GPS module as an example, acquires the position information (latitude, longitude, altitude) of the flying object 10, and sends this position information to the remote controller 30 via the communication unit 16. The frequency of sending the position information of the flying object 10 by the position acquisition unit 13 is preferably a short time interval such that the remote controller 30 can grasp the position of the flying object 10 in real time when the flying object 10 is performing a demonstration flight. On the other hand, when the flying object 10 is performing a planned flight, it is sufficient to store the position information of the flying object 10 in a memory (not shown) at predetermined time intervals and send it to the remote controller 30 after the planned flight is completed. Naturally, even during planned flight, the position information of the flying object 10 may be sent at a short time interval such that the remote controller 30 can grasp the position of the flying object 10 in real time.

[0055] The imaging device control unit 14 causes the imaging device 20 to perform an imaging operation based on an instruction from the remote controller 30, and acquires imaging data from the imaging device 20. Then, the imaging device control unit 14 sends the acquired imaging data to the remote controller 30 via the communication unit 16.

[0056] The imaging device attitude control unit 15 performs control to set the rotation angle and pitch angle of the imaging device 20 to predetermined angles based on information regarding the pitch angle of the imaging device 20 sent from the remote controller 30.

[0057] The communication unit 16 performs data transmission and reception with the remote controller 30. The wireless communication between the remote controller 30 (its first communication unit 35) and the communication unit 16 is preferably based on a wireless communication standard commonly used for communication with drones, but there is no particular limitation, mainly using a wireless communication standard that does not require a license as a wireless station.

[0058] FIG. 4 is a diagram showing an example of the route information table 32b stored in the remote controller 30 of the object photographing system 1 of the present embodiment.

[0059] As shown in FIG. 4, the route information table 32b has, as entries, a route ID 320, a route name 321, an estimated flight time 322, a registration date and time 323, a departure point latitude 324, a departure point longitude 325, and a departure point altitude 326.

[0060] The route ID 320 is an ID for specifying each route when a plurality of flight plans (routes) are registered in the remote controller 30. The route name 321 is a name assigned to each route. The estimated flight time 322 is the estimated flight time when flying the flying object 10 by demonstration flight. As an example, the flight time when flying the flying object 10 by planned flight is stored. The registration date and time 323 is the date and time when the route was registered by flying the flying object 10 by planned flight. The departure point latitude 324 is the latitude of the point where the flying object 10 departs during planned flight. The departure point longitude 325 is the longitude of the point where the flying object 10 departs during planned flight. The departure point altitude 326 is the altitude of the point where the flying object 10 departs during planned flight.

[0061] Each entry in the route information table 32b stores respective information when registering a route by flying the aircraft 10 by planned flight. Also, the departure point latitude 324, longitude 325, and altitude 326 are used to determine from which position to depart the aircraft 10 during demonstration flight.

[0062] FIG. 5 is a diagram showing an example of a point information table 32c stored in the remote controller 30 of the object photographing system 1 of the present embodiment.

[0063] As shown in FIG. 5, the point information table 32c has, as entries, a route ID 330, a point ID 331, a time 332, a latitude 333, a longitude 334, an altitude 335, a nose direction 336, a pitch angle 337, and a photographing mode 338.

[0064] Route ID 330 is an entry common to the route ID 320 in the route information table 32b. Point ID 331 is an ID for specifying a point provided on the route. Time 332 is the elapsed time since takeoff when the aircraft 10 is flown by a planned flight. Latitude 333 is the latitude of the point, longitude 334 is the longitude of the point, and altitude 335 is the altitude of the point. At this time, the altitude 335 may be a relative value from the takeoff point altitude 326. The nose direction 336 is an angle indicating the nose direction of the aircraft 10, and as an example, it is an angle specified clockwise with the north direction as 0° (as an example, the east is 90°). The pitch angle 337 is the vertical angle of the imaging device 20 with respect to the aircraft 10. The nose direction 336 and the pitch angle 337 determine the field of view of the imaging device 20 (in other words, the imaging direction). The shooting mode 338 is information for specifying the control mode at each point of the flight plan, that is, the landscape mode or the subject mode. As an example, 0 is stored when the point is in the landscape mode, and 1 is stored when it is in the subject mode. In the case of the subject mode, since the nose direction of the aircraft 10 and the pitch angle of the imaging device 20 are determined based on the position information of the subject, information regarding the nose direction 336 and the pitch angle 337 is unnecessary if the point is in the subject mode. Therefore, invalid values (for example, 0) are input for the nose direction 336 and the pitch angle 337 in the subject mode.

[0065] <Overview of the operation of the object shooting system> FIG. 6 is a diagram showing an overview of the object shooting operation by the object shooting system 1 of the present embodiment.

[0066] In FIG. 6, the aircraft 10 takes off from the takeoff point t0 and flies in the air along the flight path until it reaches the landing point t3. At this time, at the takeoff point t0 to point t1 and point t2 to landing point t3, the aircraft 10 flies in the landscape mode, and at point t1 to t2, the aircraft 10 flies in the subject mode.

[0067] Therefore, between the takeoff point t0 and the point t1, and at the point t2 to the landing point t3, which are in the landscape mode, the attitude of the flying object 10 (especially the nose direction) and the pitch angle of the imaging device 20 are controlled so that the imaging device 20 faces the imaging directions C1 and C4 independent of the position of the subject O, who is the service user. Thereby, the imaging device 20 images the background and scenery around the takeoff position and the landing position, and acquires imaging data.

[0068] On the other hand, between the points t1 and t2, which are in the subject mode, the attitude of the flying object 10 (especially the nose direction) and the pitch angle of the imaging device 20 are controlled so that the imaging device 20 faces the imaging directions C2 and C3 dependent on the position of the subject O. Thereby, the imaging device 20 images the subject and its surroundings, and acquires imaging data.

[0069] Figs. 7 to 9 are diagrams showing an example of the setting of the landscape mode and the subject mode in the object imaging system 1 of the present embodiment.

[0070] In the example shown in Fig. 7, the flying object 10 ascends almost vertically from the takeoff point to a predetermined altitude (10 m to 15 m) as the landscape mode, then descends while the imaging device 20 images the background and scenery. After that, it shifts to the subject mode and approaches to about 5 m in front of the subject O while horizontally flying at the eye level of the subject O, and the imaging device 20 images the subject O. Then, while retreating and ascending, it continues the subject mode and the imaging device 20 images the subject O. And when it reaches the predetermined altitude, it returns to the landscape mode and the imaging device 20 images the background and scenery to end a series of flights.

[0071] Also, in the example shown in FIG. 8, the flying object 10, in the scenery mode, ascends from the takeoff point to a predetermined altitude and then moves the nose direction left and right (in the direction perpendicular to the paper surface in the figure) (that is, shakes the head) at that point, and the imaging device 20 captures the background and scenery. After that, it shifts to the subject mode, and while advancing and descending about 5 m in front at the height of the eyes of the subject O, the imaging device 20 captures the subject O. Then, while continuing to advance and ascend, it continues to capture the subject O with the imaging device 20 in the subject mode. When it reaches almost directly above the subject O, it returns to the scenery mode and the imaging device 20 captures the background and scenery, ending a series of flights.

[0072] Furthermore, in the example shown in FIG. 9, the flying object 10, in the scenery mode, ascends from the takeoff point to a predetermined altitude and then moves the nose direction left and right (in the direction perpendicular to the paper surface in the figure) (that is, shakes the head) at that point, and the imaging device 20 captures the background and scenery. Furthermore, while descending, the imaging device 20 captures the background and scenery. After that, it shifts to the subject mode and approaches to about 5 m in front of the subject O while flying horizontally at the height of the eyes of the subject O, and the imaging device 20 captures the subject O. Then, after retreating and ascending to a predetermined altitude (5 m to 7 m), it continues to capture the subject O with the imaging device 20 in the subject mode while flying on the circumference of a circle with a radius of 5 m to 7 m above the subject O. And after flying one round on the circumference, it ascends to a predetermined altitude (10 m to 15 m), and at this point, it returns to the scenery mode and the imaging device 20 captures the background and scenery, ending a series of flights.

[0073] <Operation of the Object Photographing System> Next, with reference to the flowcharts of FIGS. 10 to 12 and the screen examples of FIGS. 13 to 17, the operation of the object photographing system 1 of the present embodiment will be described.

[0074] First, with reference to the screen examples of FIGS. 13 to 15, the operation of the planned flight in the object photographing system 1 of the present embodiment will be described.

[0075] FIG. 13(a) shows an example of a screen displayed on the display unit 33 of the remote controller 30 at the start of the operation of the object photographing system 1 of the present embodiment. When an operator (service provider) of the object photographing system 1 performs an operation input by touching an icon 1301 at the upper right of this screen with a finger, the screen transitions to the one shown in FIG. 13(b). Therefore, when an operator who executes planned flight performs an operation input by touching the "flight route registration" button 1302 displayed on the screen with a finger, the planned flight operation starts.

[0076] First, the operator places the flying object 10 at a predetermined takeoff point and then operates the input unit 34 of the remote controller 30 to raise the flying object 10 to a predetermined height (10 m to 15 m) and stop the horizontal position while maintaining the altitude at this point (i.e., hover). As shown in FIG. 14(a), imaging data captured by the imaging device 20 provided on the flying object 10 is displayed on the display unit 33 of the remote controller 30. At this time, when the operator performs an operation input by touching the "registration start" button 1401 displayed on the screen of the display unit 33 with a finger, the flight route registration starts.

[0077] Thereafter, while the operator operates the input unit 34 of the remote controller 30 to fly the flying object 10 along a predetermined flight route, the operator appropriately changes the nose direction of the flying object 10 and the pitch angle of the imaging device 20 to capture a desired subject (including background, scenery) by the imaging device 20. The position information of the flying object 10, the information regarding the nose direction, and the information regarding the pitch angle of the imaging device 20 are sequentially transmitted to the remote controller 30 and stored in the memory 32. Note that a button 1402 for specifying the shooting mode is displayed on the screen of the display unit 33, and the shooting mode at that point (point) can be specified during flight route registration. Information regarding the shooting mode is also sequentially transmitted to the remote controller 30 and stored in the memory 32.

[0078] During the planned flight of the aircraft 10, as shown in Fig. 14(b), the flight path (route) 1403 of the aircraft 10 from the moment the "Registration Start" button 1401 is pressed until that point is displayed on the screen. Then, when the operator performs an operation input by touching the "Registration End" button 1404 displayed on the screen of the display unit 33 with a finger or the like, the planned flight by the aircraft 10 ends.

[0079] After that, the aircraft 10 descends to the landing point and ends the flight. The flight control of the aircraft 10 to the landing point may be performed by the operator operating the input unit 34 of the remote controller 30, or may be performed by the aircraft 10 controlling itself.

[0080] When the planned flight by the aircraft 10 ends, a screen as shown in Fig. 15 is displayed on the display unit 33 of the remote controller 30. The flight route 1501 of the planned flight is displayed on the screen. The operator operates the input unit 34 of the remote controller 30 to input the name of the flight route into the input field 1502, and performs an operation input by touching the "Route Registration" button 1503 with a finger or the like. Thereby, the flight plan creation unit 31b of the remote controller 30 inputs predetermined information into the route information table 32b and the point information table 32c that constitute the flight plan 32a.

[0081] Next, with reference to the flowcharts of Figs. 10 to 12 and the screen examples of Figs. 16 and 17, the operation of the demonstration flight in the object photographing system 1 of the present embodiment will be described.

[0082] Fig. 10 is a sequence diagram for explaining the communication procedure between the remote controller 30 and the second communication device 50. Although Fig. 10 shows communication between the remote controller 30 and the second communication device 50, it may be configured such that the first communication device 40 controls the communication procedure between the second communication device 50 and sends the result to the remote controller 30.

[0083] First, communication between the remote controller 30 and the second communication device 50 starts (S1000, S1001). When the second communication device 50 discovers the remote controller 30 (S1002), it transmits a connection request message to the remote controller 30 (S1003). The remote controller 30 that has received the connection request message (S1004) transmits a connection start message to the second communication device 50 (S1005). The second communication device 50 that has received the connection start message (S1006) transmits the position information of the second communication device 50 (i.e., the subject position information) acquired by the position acquisition unit 51 to the remote controller 30 (S1007). The remote controller 30 that has received the position information (S1008) transmits information regarding the flight state of the aircraft 10 at that time (S1009) and updates the subject position information 32d in the memory 32 with the transmitted subject position information (S1010). The information regarding the flight state of the aircraft 10 includes, for example, information indicating standby, in flight, and flight completion respectively. On the other hand, the second communication device 50 receives the information regarding the flight state of the aircraft transmitted from the remote controller 30 (S1011) and updates the screen (S1012).

[0084] Thereafter, the steps shown in S1007 to S1011 are repeatedly performed between the remote controller 30 and the second communication device 50. The repetition frequency is, for example, at 0.2 - second intervals.

[0085] The communication procedure shown in the sequence diagram of FIG. 10 is based on the WiFi Aware standard as an example. Of course, it is not limited to communication according to this standard, and communication via a WiFi router, communication based on the WiFi Direct standard, etc. are also possible.

[0086] FIG. 17 is a diagram showing an example of a screen displayed on the display unit 52 of the second communication device 50. First, when the subject O, who is a service user, touches the "Connect" button 1701 displayed on the screen of the display unit 52 of the second communication device 50 he / she has with a finger or performs an operation input in some other way, communication with the remote controller 30 starts (S1001 in FIG. 10). After this, based on the information transmitted from the remote controller 30, the screen displayed on the display unit 52 of the second communication device 50 changes. As an example, in FIG. 17(b), it is displayed that it is waiting for connection with the remote controller 30, and in FIG. 17(c), it is displayed that the flying object 10 is preparing for shooting.

[0087] In a state where communication between the remote controller 30 and the second communication device 50 is established, the operator, who is a service provider, places the flying object 10 at a predetermined takeoff position. Further, when an operation input is performed by touching the "Route Flight" button 1303 with a finger or the like on the screen of the display unit 33 of the remote controller 30 shown in FIG. 13(a), the flight control unit 31a of the remote controller 30 reads out the route information table 32b of the flight plan 32a, and displays a list of route names 321 in this route information table 32b on a schematic screen, thereby displaying a list of flight routes (flight paths) on the screen of the display unit of the remote controller 30. Therefore, when the operator selects a flight route for the demonstration flight, the procedure shown in the flowchart of FIG. 11 starts.

[0088] First, the flight control unit 31a of the remote controller 30 reads out the point information table 32c corresponding to the selected flight route. Then, the flight control unit 31a reads out various information of the start point of the flight route, that is, the information described in the latitude 333, longitude 334, altitude 335, nose direction 336, and pitch angle 337 of the point with the smallest point ID 331 in the point information table 32c, and transmits an instruction to move the flying object 10 to this start point to the flying object 10 (S1100).

[0089] When the aircraft 10 moves to the starting point, a screen as shown in Fig. 16(a) is displayed on the screen of the display unit 33 of the remote controller 30. In this state, when the operator performs an operation input by touching the "Start" button 1601 with a finger or the like, the flight control unit 31a instructs the aircraft 10 to perform a point flight operation from the starting point to the final point (S1101~S1103). The details of the point flight operation will be described later with reference to the flowchart of Fig. 12. During the point flight operation, a screen based on the imaging data captured by the imaging device 20 is displayed on the screen of the display unit 33 of the remote controller 30 as shown in Fig. 16(b).

[0090] Then, when the aircraft 10 moves to the final point, the flight control unit 31a instructs the aircraft 10 to move to the landing point (landing location) (S1104).

[0091] Fig. 12 is a flowchart for explaining the point flight operation in the object imaging system 1 of the present embodiment.

[0092] First, the flight control unit 31a determines whether the aircraft 10 has reached the horizontal point, that is, the point at which the aircraft 10 should hover (S1200). If it is determined that the horizontal point has been reached (YES in S1200), the flight control unit 31a calculates the horizontal movement instruction amount of the aircraft 10 (S1204). The horizontal movement instruction amount in S1204 is 0, that is, an instruction that the horizontal movement of the aircraft 10 is not performed.

[0093] On the other hand, if it is determined that the horizontal point has not been reached (NO in S1200), the flight control unit 31a determines whether the current point is the starting point (S1201). If it is determined that it is the starting point (YES in S1201), the flight control unit 31a calculates the distance and direction from the current position (latitude, longitude) of the aircraft 10 to the latitude and longitude of the point to be advanced, that is, the horizontal movement instruction amount, based on the current position information of the aircraft 10 (S1202), and further calculates the distance from the current position (altitude) of the aircraft 10 to the altitude of the point to be advanced, that is, the vertical movement instruction amount (S1203). On the other hand, if it is determined that it is not the starting point (NO in S1201), only S1203 is executed.

[0094] Next, the flight control unit 31a determines whether the shooting mode at the point to be advanced is the landscape mode or the subject mode based on the information described in the shooting mode 338 of the point information table 32c (S1205).

[0095] If it is determined that it is the landscape mode, the flight control unit 31a calculates the rotation direction of the aircraft 10 based on the current nose direction of the aircraft 10 and the nose direction 336 described in the point information table 32c, and further calculates the pitch angle of the imaging device 20 at the point to be advanced based on the pitch angle 337 described in the point information table 32c (S1207).

[0096] On the other hand, if it is determined that it is the subject mode, the flight control unit 31a calculates the rotation direction of the aircraft 10 based on the current nose direction of the aircraft 10 and the nose direction 336 described in the point information table 32c, and further assumes that the height of the subject is a predetermined height (for example, 1 m), and calculates the pitch angle of the imaging device 20 from the height of this subject and the current altitude of the aircraft 10 (S1206).

[0097] Then, the flight control unit 31a sends the horizontal movement instruction amount, vertical movement instruction amount calculated in S1202 and S1203, the rotational movement instruction amount calculated in S1206 or S1207, and the pitch angle to the flying object 10, and instructs the flying object 10 to fly based on these pieces of information (S1208).

[0098] As described in detail above, according to the object photographing system 1 of the present embodiment, aerial photography can be performed by the imaging device 20 mounted on the flying object 10 based on the flight plan 32a stored in the remote controller 30. Thereby, even a service provider who does not have the skill to fly a flying object 10 such as a drone can easily perform aerial photography.

[0099] Furthermore, according to the object photographing system 1 of the present embodiment, a scenery mode and a subject mode are provided on the flight path. In the subject mode, the nose direction of the flying object 10 and the pitch angle of the imaging device 20 are changed and controlled based on the position information of the subject who is the service user. Therefore, using the flying object 10 (and the imaging device 20), commemorative photography for tourists using the flying object 10 can be performed in a simpler method. In particular, according to the object photographing system 1 of the present embodiment, not only the imaging data of the subject is acquired, but also the imaging data obtained by photographing from the air the scenery around the shooting location, such as famous temples, which is usually difficult for tourists to photograph, can be acquired, and the imaging data with a unique charm that cannot be obtained only by the imaging data of the tourist (subject) can be acquired.

[0100] Note that the above-described embodiments are those in which the configuration has been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Also, a part of the configuration of each embodiment can be added to, deleted from, or replaced with other configurations.

[0101] As an example, in this embodiment, the subject is assumed to be stationary at a designated point in principle. However, since the subject position information 32d is acquired by the remote controller 30 sequentially, by appropriately changing the nose direction of the aircraft 10 and the elevation angle of the imaging device 20 in the subject mode based on the subject position information 32d, imaging by the imaging device 20 can be performed so as to follow the moving subject.

[0102] Also, some or all of the above-described components, functions, processing units, processing means, etc. may be realized in hardware by designing them, for example, in an integrated circuit. Further, the present invention can also be realized by a program code of software that realizes the functions of the embodiments. In this case, a storage medium storing the program code is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing it constitute the present invention. As a storage medium for supplying such a program code, for example, a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, an SSD (Solid State Drive), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, etc. are used.

[0103] Also, the program code for realizing the functions described in this embodiment can be implemented in a wide range of programs or script languages such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), Python, etc.

[0104] Furthermore, all or part of the program code of the software that realizes the functions of each embodiment may be stored in the memory 32 in advance, or may be stored in the memory 32 from a non-temporary storage device of another device connected to the network or from a non-temporary storage medium via a schematic external I / F included in the remote controller 30 as needed.

[0105] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network, stored in a storage means such as a hard disk or memory of a computer or a storage medium such as a CD-RW or CD-R, and the processor included in the computer may read and execute the program code stored in the storage means or the storage medium.

[0106] In the above-described embodiments, the control lines and information lines show those considered necessary for the description, and not necessarily all the control lines and information lines on the product. All the components may be interconnected.

Explanation of Reference Numerals

[0107] 1... Object imaging system 10... Aircraft 11... Driving unit 12... Driving control unit 13... Position acquisition unit 14... Imaging device control unit 15... Imaging device attitude control unit 16... Communication unit 20... Imaging device 30... Remote control 31... Processor 31a... Flight control unit 31b... Flight plan creation unit 32... Memory 32a... Flight plan 32b... Route information table 32c... Point information table 32d... Subject position information 33... Display unit 34... Input unit 35... First communication unit 36... Second communication unit 40... First communication device 50... Second communication device 51... Position acquisition unit 60... Content server 70... User terminal

Claims

1. An object imaging system that uses a moving body to image an object, comprising an imaging device that images within a predetermined field of view and outputs imaging data, the moving body that can move to a predetermined position, a detection device that detects the position of the object based on the position information of a communication device held by the object, and a control device that controls the movement of the moving body to the position wherein the control device controls the movement of the moving body in a plurality of different control modes at different time periods during the movement of the moving body, in a first control mode among the plurality of control modes, the control device controls the movement of the moving body to move while taking a posture dependent on the object position, in a second control mode among the plurality of control modes, the control device controls the movement of the moving body to move while taking a predetermined posture independent of the object position characterizing the object imaging system.

2. The imaging device outputs the imaging data by imaging within the field of view at least in the first control mode and the second control mode, according to claim 1 of the object imaging system.

3. The control device sequentially and continuously executes the first control mode and the second control mode, according to claim 1 or 2 of the object imaging system.

4. The moving body is configured to be able to control the imaging direction of the imaging device, in the first control mode, the control device controls the movement of the moving body while controlling the imaging direction of the imaging device so that the object is located within the field of view of the imaging device based on the object position characterizing the object imaging system according to any one of claims 1 to 3.

5. In at least the second control mode, the control device controls the movement of the moving body along a movement path formed by arranging the plurality of positions of the moving body in time series, according to any one of claims 1 to 4 of the object imaging system.

6. In at least the first control mode and the second control mode, the control device controls the movement of the moving body along the movement path, according to claim 5 of the object imaging system.

7. The moving body is configured to be able to detect the moving body position, the control device sends out information regarding the plurality of positions constituting the movement path, The moving body moves the moving body along the moving path based on the information regarding the plurality of positions sent from the control device and the moving body position. The object photographing system according to claim 5, characterized in that.

8. The control device acquires moving image data, which is the imaging data output from the imaging device, at least in the first control mode and the second control mode, and sends the moving image data to an external server. The object photographing system according to any one of claims 1 to 7, characterized in that.

9. The moving body is configured to be able to detect the moving body position. The control device moves the moving body along a planned moving path formed by arranging a plurality of planned positions of the moving body in time series, acquires the moving body position detected by the moving body when the moving body is moved along the planned moving path, and controls the movement of the moving body at least in the second control mode based on the acquired moving body position. The object photographing system according to any one of claims 1 to 8, characterized in that.

10. The object photographing system according to any one of claims 1 to 9, characterized in that the moving body is an aircraft.

11. An imaging device that images within a predetermined field of view and outputs imaging data, a moving body that can move to a predetermined position, A detection device that detects the object position of the object based on the position information of the communication device held by the object, A control device that controls the movement of the moving body to the position of the moving body A control device used in an object photographing system having, The control device controls the movement of the moving body in a plurality of different control modes at different time zones during the movement of the moving body. In the first control mode among the plurality of control modes, the control device controls the movement of the moving body so as to move while taking a posture depending on the object position. In the second control mode among the plurality of control modes, the control device controls the movement of the moving body so as to move while taking a predetermined posture independent of the object position. A control device used in an object photographing system, characterized in that.

12. An imaging device that images within a predetermined field of view and outputs imaging data, a moving body that can move to a predetermined position, A detection device that detects the object position of the object based on the position information of the communication device held by the object, A control device that controls the movement of the moving body to the position of the moving body A control method in an object photographing system having [specific feature not provided in the original, so it's left as is], controlling the movement of the moving body in a plurality of different control modes at different time zones during the movement of the moving body, in a first control mode among the plurality of control modes, controlling the movement of the moving body to move while taking a posture dependent on the object position, in a second control mode among the plurality of control modes, controlling the movement of the moving body to move while taking a predetermined posture independent of the object position characterized by a control method in an object photographing system.

13. having an imaging device that images within a predetermined field of view and outputs imaging data, a moving body that can move to a predetermined position, a detection device that detects the object position of the object based on the position information of a communication device held by the object, and a computer that controls the movement of the moving body to the position A computer program in an object photographing system having [specific feature not provided in the original, so it's left as is], the computer program causes the computer to control the movement of the moving body in a plurality of different control modes at different time zones during the movement of the moving body, in a first control mode among the plurality of control modes, control the movement of the moving body to move while taking a posture dependent on the object position, in a second control mode among the plurality of control modes, control the movement of the moving body to move while taking a predetermined posture independent of the object position characterized by a computer program.

Images