Information processing device, control method, and program

Abstract

To provide an information processing device, a control method for the same, and a program capable of recording information on maneuvering instructions for movable subjects accurately in real time for captured image data captured by a movable imaging device.SOLUTION: An imaging movable apparatus 100 is equipped with an imaging apparatus 130 configured to capture a moving subject 150 involving maneuvering, thereby generating image data. The imaging movable apparatus 100 includes a communication unit 118 for acquiring maneuvering instruction information of the moving subject 150, an information generating unit 201 for generating information concerning maneuvering instructions corresponding to the maneuvering instruction information, and a recording unit 202 for associating and recording the imaging data and maneuvering instruction-related information generated by the information generating unit 201 each time maneuvering instruction information is obtained by the communication unit 118.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a control method thereof, and a program, and particularly to an information processing apparatus for controlling shooting and recording in an image-capturable mobile device, a control method thereof, and a program.

Background Art

[0002] In recent years, a style of shooting the state of driving and making it into a video work as a driving VLOG has attracted attention. In VLOGs, various video cuts are required, so users who use image-capturable mobile devices such as drones to produce driving VLOGs are starting to appear.

[0003] In order to obtain a higher-quality driving VLOG, users may want to perform editing such as adding sound to the video according to the operation of the vehicle, or adding a transition between scenes of the captured video. For example, editing to add engine sound in a scene where the accelerator is depressed, or editing to add a rotation transition between the last scene where the steering wheel is turned and the next scene.

[0004] On the other hand, a technique for recording image data captured by a drone and information related to the image data is disclosed. For example, Patent Document 1 discloses a technique of recording image data captured by a camera provided in a drone and generating specific information for specifying the recorded image data.

[0005] Also, Patent Document 2 discloses a technique of acquiring additional information characterizing an image captured by a camera, determining the necessity of an image characterized by the additional information by an image management apparatus based on the additional information, requesting and receiving an image to be determined, and storing it in association with the additional information.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

[0007] However, both the specific information of Patent Document 1 and the additional information of Patent Document 2 are information such as the shooting time of the image taken by the camera, and are not information on the subject side photographed by the camera.

[0008] For example, when the subject involving movement is a car, information on how the user driving the car presses the accelerator can theoretically be obtained by analyzing the acceleration of the car from the image data already recorded by the camera, but it is difficult to obtain it accurately. Also, when such an analysis is performed, there is a problem that information regarding the operation instruction of the subject involving movement cannot be recorded in association with the photographed image in real time.

[0009] In addition, in VLOG production, a work is produced while combining various shooting data consisting of image data and audio data obtained by shooting with a drone. Therefore, when only the image to be determined is recorded together with additional information as in Patent Document 2, there is a problem that the shooting data required by the user may not be recorded during editing.

[0010] Therefore, an object of the present invention is to provide an information processing apparatus, a control method thereof, and a program that can accurately and in real time associate information regarding an operation instruction of a subject involving movement with shooting data taken by an imaging-capable mobile device and record the information. [Means for Solving the Problems]

[0011] In order to solve the above problems, an information processing apparatus according to claim 1 of the present invention acquires steering instruction information of a moving subject that involves steering, and generates information related to a steering instruction corresponding to the steering instruction information. Each time the imaging data is acquired from an imaging unit that is mounted on an imaging-capable moving device and that captures a moving subject that involves steering and generates the imaging data, the apparatus includes a recording unit that records the imaging data and the information related to the steering instruction generated by the generating unit in association with each other.

Effect of the Invention

[0012] According to the present invention, it is possible to accurately and in real time record information related to a steering instruction of a subject that involves movement in association with the imaging data acquired by the imaging-capable moving device.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0015] In the present invention, a desired video is captured by changing the position and orientation of the image-capturable mobile device and the shooting conditions according to the operation instruction information of the moving subject accompanied by operation. In the present invention, an example in which the image-capturable mobile device moves by flying in the air by a propeller or the like will be described, but the moving method is not limited to this. For example, it may move on the ground by wheels or move in water by a screw.

[0016] Hereinafter, an image-capturable mobile device 100 including a video recording unit 135 as an information processing device according to an embodiment of the present invention, and a moving subject 150 accompanied by operation that is wirelessly communicably connected to the image-capturable mobile device 100 and is its shooting target will be described in detail with reference to the accompanying drawings.

[0017] FIG. 1 is a block diagram showing an example of the hardware configuration of each of the image-capturable mobile device 100 and the moving subject 150.

[0018] In FIG. 1, the image-capturable mobile device 100 includes an imaging device 130 (shooting unit) that communicably connects video data, operation instruction information, camera work information, etc., which will be described later, to each other via a bus, and a mobile device 110 on which the imaging device 130 is mounted. The moving subject 150 is a subject photographed by the imaging device 130 and moves in response to an operation instruction.

[0019] First, a configuration example of the mobile device 110 included in the imaging-capable mobile device 100 will be described with reference to FIG. 1.

[0020] The mobile device 110 includes a control unit 111, a ROM 112, a RAM 113, a position and attitude detection unit 114, an image processing unit 115, a shooting scenario list recording unit 116, a movement mechanism unit 117, and a communication unit 118. These blocks are connected to each other via a bus so as to be communicable.

[0021] The control unit 111 is, for example, a CPU. It reads out a control program for each block included in the mobile device 110 and each block described later included in the imaging device 130 from the ROM 112, expands it in the RAM 113, and executes it. Thereby, the control unit 111 controls the operations of each block included in the imaging-capable mobile device 100. Further, the control unit 111 receives operation instruction information and position information from the moving subject 150 via the communication unit 118. The operation instruction information and the position information will be described later.

[0022] The ROM 112 is an electrically erasable and recordable non-volatile memory, and stores operation programs for each block included in the imaging-capable mobile device 100, as well as parameters and the like necessary for the operations of each block.

[0023] The RAM 113 is a rewritable volatile memory, and is used for expanding programs executed by the control unit 111 and temporarily storing data generated by the operations of each block included in the imaging-capable mobile device 100.

[0024] The position and attitude detection unit 114 is a sensor that detects various information necessary for position and attitude control of the mobile device 110, and outputs the detected position and attitude information to the control unit 111. Examples of this sensor include a GPS sensor for detecting position, a gyro sensor for detecting angular velocity, an acceleration sensor for detecting speed change, a magnetic sensor for detecting direction, a pressure sensor for detecting altitude, and an ultrasonic sensor for detecting the distance to surrounding objects such as a subject. The control unit 111 performs calculations according to the information detected by the position and attitude detection unit 114, and controls the position and attitude of the mobile device 110.

[0025] The image processing unit 115 performs various image processes related to the position and orientation of the mobile device 110 by analyzing the video captured by the imaging device 130 and the information necessary for position and orientation control acquired by the position and orientation detection unit 114. For example, the image processing unit 115 performs image processing to recognize a subject (e.g., the moving subject 150) included in the video captured by the imaging device 130, and the control unit 111 controls the movement mechanism unit 117 to track the subject using the recognition result. Regarding the subject recognition method and the tracking method, known methods are used and detailed explanations are omitted.

[0026] The shooting scenario list recording unit 116 records a shooting scenario list received from an external information processing device (not shown) via the communication unit 118. Note that at the time of initial setting, a shooting scenario list may be pre-recorded in the shooting scenario list recording unit 116.

[0027] The movement mechanism unit 117 is composed of a buoyancy generation mechanism such as a motor and a propeller. The control unit 111 controls the position and orientation of the mobile device 110 by driving the movement mechanism unit 117 according to the information detected by the position and orientation detection unit 114 and the recognition result of the subject by the image processing unit 115. As a result, it is possible to perform tracking shooting in which the angle of view is controlled so that the moving subject 150 is included in the field of view captured by the imaging device 130.

[0028] The communication unit 118 communicates with the moving subject 150 based on a communication method defined by a standard such as a wireless LAN.

[0029] Next, a configuration example of the imaging device 130 included in the imaging-capable mobile device 100 will be described with reference to FIG. 1.

[0030] The imaging device 130 includes an optical system 131, an imaging unit 132, an A / D conversion unit 133, an image processing unit 134, a video recording unit 135, and a display unit 136. These blocks are connected to each other via a bus so as to be communicable.

[0031] The optical system 131 is composed of a lens group including a zoom lens and a focus lens and a diaphragm mechanism, and forms an object image on the imaging surface of the imaging unit 132.

[0032] The imaging unit 132 is an imaging device such as a CCD or a CMOS sensor, photoelectrically converts the optical image formed on the imaging surface of the imaging unit 132 by the optical system 131, and outputs the obtained analog image signal to the A / D conversion unit 133.

[0033] The A / D conversion unit 133 converts the input analog image signal into digital image data and outputs it. The digital image data output from the A / D conversion unit 133 is temporarily stored in the RAM 113.

[0034] The image processing unit 134 performs various image processes on the image data stored in the RAM 113. Specifically, it performs various image processes for developing, displaying, and recording digital image data, such as demosaicking processing, noise reduction processing, white balance correction processing, and gamma processing. This image processing also includes a process of generating video data from the image data stored in the RAM 113 in time series.

[0035] The video recording unit 135 records data including the video data generated by the image processing unit 134 on the built-in recording medium. Further, the video recording unit 135 acquires camera work information indicating the movement of the moving device 110 during shooting of the imaging device 130, such as operation instruction information received via the communication unit 118 and the rotation of the camera, and generates and records information related to the operation instruction. The processing of the video recording unit 135 will be described later.

[0036] The display unit 136 includes a display device such as an LCD, and displays the image data stored in the RAM 113 and the video data stored in the video recording unit 135 on the display device.

[0037] Next, a configuration example of the moving subject 150 will be described with reference to FIG. 1. In the present invention, the imaging-capable moving device 100 captures the moving subject 150 with the imaging device 130 and generates image data. In the present invention, an automobile is described as an example of the moving subject 150, but the moving subject 150 is not limited thereto. For example, the moving subject 150 may be an object such as a motorcycle, a ship, an airplane, or a train that is operated and moved by a passenger. Note that the moving subject 150 may be a device that does not require a passenger to operate and that recognizes the surrounding situation and performs automatic control. Further, the moving subject 150 may be a device that is not operated by a passenger but is remotely controlled like a radio-controlled car. The method of automatic control and the method of control by remote operation use known control methods, and detailed description thereof is omitted.

[0038] The moving subject 150 includes a control unit 151, a ROM 152, a RAM 153, a position detection unit 154, a control mechanism unit 155, an operation mechanism unit 156, and a control communication unit 157. These blocks are connected to each other via a bus so as to be communicable.

[0039] The control unit 151 is, for example, a CPU, reads a control program for each block included in the moving subject 150 from the ROM 152, expands it in the RAM 153, and executes it. Thereby, the control unit 151 controls the operations of the respective blocks included in the moving subject 150. Further, the control unit 151 transmits control instruction information and position information to the moving device 110 via the control communication unit 157. The control instruction information and the position information will be described later.

[0040] The ROM 152 is an electrically erasable and recordable non-volatile memory, and stores operation programs for the respective blocks included in the moving subject 150, as well as parameters and the like necessary for the operations of the respective blocks.

[0041] The RAM 153 is a rewritable volatile memory, and is used for expanding programs executed by the control unit 151 and the like, and temporarily storing data generated by the operations of the respective blocks included in the moving subject 150.

[0042] The position detection unit 154 has a plurality of sensors that detect the three-dimensional space position, and outputs the information on the three-dimensional space position calculated from the information detected by each sensor to the operation control unit 151. Examples of the plurality of sensors include a GPS sensor that detects a two-dimensional plane position and a barometric pressure sensor that detects altitude. The position detection unit 154 calculates the three-dimensional position information of the moving object 150 by using these sensors. Note that a GPS sensor may be used as the sensor for detecting altitude.

[0043] The steering mechanism unit 155 is a mechanism for a passenger to give a steering instruction to the moving object 150. Specifically, the steering mechanism unit 155 is composed of movable mechanisms such as a steering wheel, an accelerator pedal, a brake pedal, a turn signal lever, a wiper lever, and a light switch. Each time the passenger moves and gives an instruction to the steering mechanism unit 155, steering instruction information is output to the operation mechanism unit 156.

[0044] The operation mechanism unit 156 drives various mechanisms (movable mechanisms such as front wheels, engines, and headlights, and notification mechanisms such as turn signal lights, brake lights, and horns) (not shown) in the moving object 150 according to the steering instruction information output from the steering mechanism unit 155. Specifically, when the steering angle of the steering wheel is output as steering instruction information from the steering mechanism unit 155, the operation mechanism unit 156 transmits this steering instruction information to an actuator that controls the traveling direction of the moving object 150, and changes the direction of the front wheels. Also, when the degree of depression of the accelerator pedal is output as steering instruction information from the steering mechanism unit 155, the operation mechanism unit 156 transmits this steering instruction information to an actuator that controls the acceleration of the moving object 150, and increases the engine speed. Also, there are cases where operations such as a light switch operation, a turn signal lever operation, a depression of the brake pedal, and a horn operation are output as steering instruction information from the steering mechanism unit 155. In this case, the operation mechanism unit 156 lights the headlights, turn signal lights, and brake lights according to each steering instruction information to give a visual notification, and also sounds the horn to give an audible notification.

[0045] The control communication unit 157 communicates with the communication unit 118 based on a communication method defined by a standard such as a wireless LAN.

[0046] Next, with reference to FIG. 2, the software configuration of the video recording unit 135 in FIG. 1 will be described.

[0047] The feature of the video recording unit 135 of this embodiment is that information related to the operation instruction is generated from the operation instruction information acquired from the moving subject 150 via the communication unit 118, and the generated information related to the operation instruction is recorded in association with the image data captured by the imaging device 130.

[0048] The video recording unit 135 in this embodiment includes an information generation unit 201 and a recording unit 202, as shown in FIG. 2.

[0049] When the information generation unit 201 (generation means) acquires the operation instruction information via the communication unit 118, it generates information related to the corresponding operation instruction and transmits it to the recording unit 202.

[0050] Each time the recording unit 202 (recording means) acquires the image data captured by the imaging device 130 from the imaging device 130 via the bus, it associates the acquired image data with the information related to the operation instruction transmitted from the information generation unit 201. Then, the recording unit 202 transmits the associated image data and the information related to the operation instruction to the RAM 113 for recording and holding.

[0051] Here, the input data from the imaging device 130 to the information generation unit 201 was the image data captured by the imaging unit 132, but it is not limited to this as long as it is the imaging data generated by the imaging device 130. For example, the imaging data input from the imaging device 130 to the information generation unit 201 may be not image data but sound data acquired by a sound acquisition unit (not shown) of the imaging device 130. Thereby, it is possible to easily obtain the material data in the case of separately shooting video data and sound data in a VLOG or the like and then combining them later. For example, by recording the engine sound acquired by the sound acquisition unit in association with the accelerator depression state (operation instruction information) at the time of acquisition, it is possible to obtain sound material data with the volume emphasized according to the operation. Further, the imaging data input from the imaging device 130 to the information generation unit 201 may be data including image data and sound data. Thereby, the video recording unit 135 can record the information regarding the operation instruction generated for the sound in association with the video data.

[0052] Next, the recording operation process executed by the video recording unit 135 will be described using the flowchart of FIG. 3. This process is executed each time the communication unit 118 acquires operation instruction information from the moving subject 150.

[0053] In step S301, when the operation instruction information is input from the communication unit 118, the information generation unit 201 generates information regarding the operation instruction from the operation instruction information and outputs it to the recording unit 202. Here, the operation instruction information is a flag indicating the content of the operation instruction such as depressing the accelerator, turning the steering wheel, opening the window, etc., and as shown in FIG. 4(a), it is expressed by a number indicating the operation instruction information set in advance. Further, the operation instruction information may be a flag expressed by multiple values that are distinguished according to the accelerator depression state, as shown in FIG. 4(b) for example. Also, in this embodiment, information regarding the operation instruction is generated and output from the operation instruction information, but the operation instruction information itself may be output as the information regarding the operation instruction. The method of generating information regarding the operation instruction from the operation instruction information will be described in detail later.

[0054] In step S302, when the image data captured by the imaging device 130 is input, the recording unit 202 records the input image data while associating it with the information regarding the operation instruction generated in step S301. Then, this process ends.

[0055] The above is the processing content of the video recording unit 135 according to this embodiment.

[0056] Next, a method for generating information regarding the operation instruction, which is performed in step S301, will be described in detail.

[0057] In step S301, the information generation unit 201 generates an editing point flag that is pre-associated with the input operation instruction information as information regarding the operation instruction. Here, the editing point flag refers to a flag indicating an in point or an out point for video, an in point or an out point for audio, etc. For example, when operation instruction information indicating an operation instruction content of stepping on the accelerator ((flag 1-1 in FIG. 4(a)) is input, according to the table shown in FIG. 5, an editing point flag 2-3 indicating the corresponding in point for audio is generated as information regarding the operation instruction. Then, the editing point flag 2-3 generated as information regarding the operation instruction is recorded in association with the image data in step S302. As a result, the user can grasp the image data that becomes a candidate for the timing of adding the audio data of the engine sound when editing the video in which the moving subject 150 is being photographed while the accelerator is being stepped on and the vehicle is accelerating. In this embodiment, the case where an editing point flag pre-associated with the input operation instruction information exists as information regarding the operation instruction has been described, but it may not exist. In this case, as information regarding the operation instruction, either at least one of flag 2-0 (a flag indicating no information regarding the corresponding operation instruction) and the operation instruction information is output, or nothing is output.

[0058] Also, in step S301, the information generation unit 201 may generate, as information regarding the operation instruction, a transition editing point flag or a recommended transition flag that is pre-associated with the input operation instruction information. Here, the transition editing point flag refers to a flag indicating in-points, out-points, etc. for video transitions such as the table shown in Fig. 6(a), a flag indicating in-points and out-points for audio transitions, etc. Also, the recommended transition flag refers to a flag (recommended transition flag) indicating a recommended transition that is pre-associated with each operation instruction information such as the table shown in Fig. 6(b). Multiple associations may be made for the information regarding the operation instruction. For example, there may be a case where two pieces of operation instruction information (flag 1-2 in Fig. 4(a)) indicating the content of the operation instruction of turning the steering wheel are input. In this case, for one of them, a transition editing point flag (flag 3-1 in Fig. 5(a)) indicating the in-point for video transitions is generated as information regarding the operation instruction. Also, for the other, a recommended transition flag (flag 4-1 in Fig. 5(b)) indicating a rotation transition is generated as information regarding the operation instruction. Then, the flags 3-1 and 4-1 generated as information regarding the operation instruction are associated with the image data and recorded in step S302. Thereby, when editing the video in which the moving subject 150 during turning is captured while the steering wheel is turned, the user can grasp the image data that is a candidate for the timing of inserting a rotation transition for rotating the video by image processing without checking all the videos. In this embodiment, the case where a transition editing point flag or a recommended transition flag that is pre-associated with the input operation instruction information exists as information regarding the operation instruction has been described, but it may not exist. In this case, as information regarding the operation instruction, either the flag 4-0 (a flag indicating no information regarding the corresponding operation instruction) and at least one of the operation instruction information is output, or nothing is output.

[0059] Also, in step S301, the information generation unit 201 may generate a captured transition flag that has been previously associated with the operation instruction information and the camera work information acquired by the video recording unit 135 as information regarding the operation instruction. Here, the camera work information is a flag indicating the content of the position and orientation control (camera work) of the moving device 110 by the control unit 111 during shooting by the imaging device 130, and as shown in FIG. 7, it is expressed by a number indicating the content of the camera work set in advance. Also, the captured transition flag here refers to a flag that indicates a combination of the operation instruction content and the type of camera work. For example, as shown in the table in FIG. 7, there may be a case where a flag 1-4 indicating the operation instruction content of turning the steering wheel is input, and at the same time, a flag 5-1 indicating the content of camera work of video rotation is input. In this case, a flag 6-1, which is a captured transition flag indicating that shooting was performed while rotating when turning the steering wheel and corresponds to flags 1-4 and 5-1, is generated as information regarding the operation instruction. Then, the flag 6-1 generated as information regarding the operation instruction is recorded in association with the image data in step S302. Thereby, even without checking all the videos, it becomes possible for the user to grasp that the transition has been captured during imaging. In this embodiment, the case where a captured transition flag previously associated with the input operation instruction information and camera work information exists as information regarding the operation instruction has been described, but it may not exist. In this case, as information regarding the operation instruction, at least one of a flag 6-0 (a flag indicating no information regarding the corresponding operation instruction), the operation instruction information, and the camera work information is output, or nothing is output. Also, when the input operation instruction information and camera work information are not previously associated, as information regarding the operation instruction, at least one of the flag 6-0 and the operation instruction information is output, or nothing is output.

[0060] Also, in step S301, the information generation unit 201 may perform weighting on the input operation instruction information and generate the weighting result as information regarding the operation instruction. For example, as shown in the table of FIG. 8, when operation instruction information indicating the degree of depression of the accelerator is input, flags 7-20, 7-40, and 7-60 indicating the weighting result are respectively generated as information regarding the operation instruction. Thereafter, the flags 7-20, 7-40, and 7-60 generated as information regarding the operation instruction are recorded in association with the corresponding image data in step S302. Thereby, when editing the video, it becomes possible to perform an edit such that the volume of the engine sound added according to the degree of depression of the accelerator is changed.

[0061] Also, in step S301, the information generation unit 201 may generate an editing point flag associated in advance with a combination of the operation instruction information and the surrounding space information acquired by the video recording unit 135, as information regarding the operation instruction. The surrounding space information here is not particularly limited as long as it is information regarding the surrounding space of the moving subject 150 (vehicle body) accompanied by operation. For example, information regarding the operation instructions of different moving subjects (other vehicle bodies) accompanied by operation in the surrounding space, information indicating the environment around the moving subject 150 such as in a tunnel or in the city, information indicating the weather around the moving subject 150 such as rain or snow, etc. may be mentioned. The surrounding space information is acquired by a surrounding space information acquisition device (not shown) and received by the information generation unit 201 via the communication unit 118. For example, when another vehicle body enters the captured video during a lane change, as shown in the table in FIG. 9, the information generation unit 201 may acquire the flag 1-1 from the moving subject 150 and at the same time acquire operation instruction information (flag 8-1) such as the blinking instruction of the turn signal of the other vehicle body. In this case, an editing point flag 9-1 corresponding to the combination of the flags 1-1 and 8-1 is generated as information regarding the operation instruction and recorded in association with the image data in step S302. Thereby, when editing the captured video, it becomes easier for the user to find a scene in which another vehicle body may be reflected in the video. In this embodiment, the case where an editing point flag associated in advance with the input operation instruction information and the surrounding space information exists as information regarding the operation instruction has been described, but it may not exist. In this case, as information regarding the operation instruction, at least one of the flag 9-0 (a flag indicating no information regarding the corresponding operation instruction), the operation instruction information, and the surrounding space information is output, or nothing is output. Also, when the input operation instruction information and the surrounding space information are not associated in advance, as information regarding the operation instruction, at least one of the flag 9-0 and the operation instruction information is output, or nothing is output.

[0062] Also, in step S301, when the information generation unit 201 acquires a plurality of pieces of operation instruction information indicating the same operation instruction content during shooting by the imaging unit 132, the plurality of acquired pieces of the same operation instruction information may be updated to a combination of associated sets.

[0063] For example, when two different image data are images captured when a manipulation instruction to turn a handle is made at the moving subject 150, the information generation unit 201 will acquire two identical manipulation instruction information (flags 1-4). In this case, one of the two acquired identical manipulation instruction information (flags 1-4) is updated to flag 1-4-1, and the other is updated to flag 1-4-2, and these are set as a set combination. For flag 1-4-1, a flag indicating the out point for video transition, which is an editing point flag associated in advance, is generated as information regarding the manipulation instruction. Also, for flag 1-4-2, a flag indicating the in point for video transition, which is an editing point flag associated in advance, is generated as information regarding the manipulation instruction. Thereafter, each flag generated as information regarding the manipulation instruction is recorded in association with the corresponding image data in step S302. Thereby, when editing a video, it becomes easier for the user to find the timing for inserting a transition and the candidates for the video to be connected by the transition.

[0064] Also, when three or more different image data are images captured when a steering instruction to turn the handle is made at the moving subject 150, the information generation unit 201 will acquire three or more pieces of the same steering instruction information (flags 1-4). In this case, among the three or more pieces of the same steering instruction information (flags 1-4) acquired, the flag that the information generation unit 201 acquired earliest is set as flag 1-4-11 (the first steering instruction information). On the other hand, the remaining flags are updated in the order in which the information generation unit 201 acquired them, as flag 1-4-12 (the second steering instruction information), flag 1-4-13 (the third steering instruction information), ···. Thus, flags 1-4-11, 1-4-12, 1-4-13 ··· are set as a set combination. For flag 1-4-11, a flag indicating the out point for video transition, which is an editing point flag associated in advance, is generated as information regarding the steering instruction. Also, for flags 1-4-12, 1-4-13 ···, a flag indicating the in point for video transition, which is an editing point flag associated in advance, is generated as information regarding the steering instruction. Thereafter, each flag generated as information regarding the steering instruction is recorded in association with the corresponding image data in step S302. This makes it easier for the user to find candidates for videos to be connected by transitions when editing videos.

[0065] In this embodiment, the case where the information generation unit 201 acquires a plurality of pieces of steering instruction information indicating the same steering instruction content and updates the plurality of acquired pieces of the same steering instruction information as a set combination has been described, but it is not limited to this. For example, when there is no steering instruction information to be updated as a set combination, information regarding the corresponding steering instruction is output without updating the steering instruction information.

[0066] The above is the content of the process for generating information regarding the steering instruction.

[0067] With the above configuration, it becomes possible for the user to easily perform video editing according to the information regarding the steering instruction while referring to the image data.

[0068] In this embodiment, the case where the video recording unit 135 as the information processing device is inside the imaging-capable mobile device 100 has been described. However, it may also be inside a server (not shown) that can communicate with the imaging-capable mobile device 100 and the moving subject 150. In this case, the control instruction information from the moving subject 150, the image data captured by the imaging device 130, the camera work information by the control unit 111, etc. are transmitted to the video recording unit 135 in the server.

[0069] Also, in this embodiment, the video recording unit 135 as the information processing device may be inside the moving subject 150. In this case, the image data captured by the imaging device 130, the camera work information by the control unit 111, etc. are transmitted to the moving subject 150.

[0070] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0071] The invention is not limited to the above-described embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, the claims are attached to disclose the scope of the invention.

[0072] As described above, the preferred embodiments of the present invention have been described. However, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist thereof.

[0073] The disclosure of this embodiment includes the following configurations, methods, and programs. An information processing apparatus, comprising: generation means for acquiring steering instruction information of a moving subject accompanied by steering and generating information regarding a steering instruction corresponding to the steering instruction information; and recording means for, each time acquiring the shooting data from a shooting unit mounted on an imaging-capable moving device, which shoots a moving subject accompanied by steering and generates the shooting data, associating and recording the shooting data and the information regarding the steering instruction generated by the generation means. The information processing apparatus according to Configuration 1, wherein the information regarding the steering instruction is the steering instruction information. The information processing apparatus according to Configuration 1, wherein the information regarding the steering instruction is a flag indicating either one of editing points for video and audio. The information processing apparatus according to Configuration 1, wherein the information related to the steering instruction is a flag indicating either one of an editing point for video transition and an editing point for audio transition. The information processing apparatus according to Configuration 1, wherein the information regarding the steering instruction is a flag indicating either one of a recommended transition for video transition and a recommended transition for audio transition. The information processing apparatus according to Configuration 1, wherein the steering instruction information is a flag expressed by multiple values. The information processing apparatus according to Configuration 1, wherein the information regarding the steering instruction is a flag corresponding to a combination of the steering instruction information and information regarding camera work in the imaging-capable moving device. The information processing apparatus according to Configuration 1, wherein the information regarding the steering instruction is a flag corresponding to a combination of the steering instruction information and peripheral space information of the moving subject. The information processing apparatus according to Configuration 8, wherein the peripheral space information of the moving subject is steering instruction information of another subject different from the moving subject. The information processing apparatus according to Configuration 8, wherein the peripheral space information of the moving subject is information indicating the environment around the moving subject. The information processing apparatus according to configuration 8, wherein the peripheral space information of the moving subject is information indicating the weather around the moving subject. (Configuration 12) The photographing unit includes an imaging unit and a sound acquisition unit, and the photographing data is at least one of the image data photographed by the imaging unit or the sound data acquired by the sound acquisition unit. The information processing apparatus according to any one of configurations 1 to 11. (Configuration 13) When the generation means acquires a plurality of pieces of the same information as the operation instruction information, the generation means updates the plurality of acquired pieces of the same operation instruction information to information on a combination of associated sets. The information processing apparatus according to any one of configurations 1 to 12. (Configuration 14) When two pieces of the same information are acquired as the operation instruction information, the generation means updates one of the two acquired pieces of the same operation instruction information to operation instruction information associated with information on an in-point of an editing point, and the other to operation instruction information associated with information on an out-point of the editing point. The information processing apparatus according to configuration 13. (Configuration 15) When three pieces of the same information are acquired as the operation instruction information, among the three acquired pieces of the same operation instruction information, for the operation instruction information acquired earliest, the generation means updates it to first operation instruction information associated with information on an in-point of an editing point, and for the remaining information, in the order of acquisition, it updates them to second operation instruction information and third operation instruction information associated with information on an out-point of the editing point, respectively. The information processing apparatus according to configuration 13. (Configuration 16) The information processing apparatus according to any one of configurations 1 to 15, wherein it is inside the movable device capable of imaging. (Configuration 17) The information processing apparatus according to any one of configurations 1 to 15, wherein it is inside the moving subject. (Configuration 18) The information processing apparatus according to any one of configurations 1 to 15, wherein it is inside a server communicably connected to the movable device capable of imaging and the moving subject. (Method 1) A control method for an information processing apparatus, comprising: an acquisition step of acquiring steering instruction information of a moving subject accompanied by steering and generating information related to a steering instruction corresponding to the steering instruction information; and a recording step of associating and recording the imaging data and the information related to the steering instruction generated in the generation step each time the imaging data is acquired from an imaging unit mounted on a movable imaging device that captures a moving subject accompanied by steering and generates the imaging data. The control method is characterized by the above steps. (Program 1) A program for causing a computer to function as each means of the information processing apparatus according to any one of Configurations 1 to 18.

Explanation of Signs

[0074] 100 Movable imaging device 110 Moving device 111 Control unit 112 ROM 113 RAM 114 Position and orientation detection unit 115 Image processing unit 116 Shooting scenario list recording unit 117 Moving mechanism unit 118 Communication unit 130 Imaging device 131 Optical system 132 Imaging unit 133 A / D conversion unit 134 Image processing unit 135 Video recording unit 136 Display unit 150 Moving subject 151 Steering control unit 152 ROM 153 RAM 154 Position detection unit 155 Steering mechanism unit 156 Operating mechanism unit 157 Steering communication unit

Claims

1. generating means for acquiring steering instruction information of a moving subject accompanied by steering and generating information regarding a steering instruction corresponding to the steering instruction information; recording means for associating and recording the imaging data and the information regarding the steering instruction generated by the generating means each time the imaging data is acquired from an imaging unit mounted on an imaging-capable moving device that captures a moving subject accompanied by steering and generates the imaging data; An information processing apparatus comprising the same.

2. The information processing apparatus according to claim 1, wherein the information regarding the steering instruction is the steering instruction information.

3. The information processing apparatus according to claim 1, wherein the information regarding the steering instruction is a flag indicating either one of editing points for video and audio.

4. The information processing apparatus according to claim 1, wherein the information related to the steering instruction is a flag indicating either one of an editing point for video transition and an editing point for audio transition.

5. The information processing apparatus according to claim 1, wherein the information regarding the steering instruction is a flag indicating either one of a recommended transition for video transition and a recommended transition for audio transition.

6. The information processing apparatus according to claim 1, wherein the steering instruction information is a flag expressed in multiple values.

7. The information processing apparatus according to claim 1, wherein the information regarding the steering instruction is a flag corresponding to a combination of the steering instruction information and information regarding camera work in the imaging-capable moving device.

8. The information processing apparatus according to claim 1, wherein the information regarding the steering instruction is a flag corresponding to a combination of the steering instruction information and peripheral space information of the moving subject.

9. The information processing apparatus according to claim 8, wherein the peripheral space information of the moving subject is steering instruction information of another subject different from the moving subject.

10. The information processing apparatus according to claim 8, wherein the peripheral space information of the moving subject is information indicating the environment around the moving subject.

11. The information processing apparatus according to claim 8, wherein the peripheral space information of the moving subject is information indicating the weather around the moving subject.

12. The imaging unit includes an imaging section and a sound acquisition section. The information processing apparatus according to claim 1, wherein the captured data is at least one of image data captured by the imaging unit or sound data acquired by the sound acquisition unit.

13. The information processing apparatus according to claim 1, wherein when the generation means acquires a plurality of pieces of the same information as the operation instruction information, the generation means updates the plurality of acquired pieces of the same operation instruction information to information on a combination of associated sets.

14. The information processing apparatus according to claim 13, wherein when two pieces of the same information are acquired as the operation instruction information, one of the two acquired pieces of the same operation instruction information is updated to operation instruction information associated with operation instruction information indicating an in point of an editing point, and the other is updated to operation instruction information associated with operation instruction information indicating an out point of the editing point.

15. The information processing apparatus according to claim 13, wherein when three pieces of the same information are acquired as the operation instruction information, among the three acquired pieces of the same operation instruction information, for the operation instruction information acquired earliest, it is updated to first operation instruction information associated with operation instruction information indicating an in point of an editing point, and for the remaining information, in the order of acquisition, it is updated to second operation instruction information and third operation instruction information associated with operation instruction information indicating an out point of the editing point, respectively.

16. The information processing apparatus according to claim 1, wherein it is inside the imaging-capable mobile device.

17. The information processing apparatus according to claim 1, wherein it is inside the moving subject.

18. The information processing apparatus according to claim 1, wherein it is inside a server communicably connected to the imaging-capable mobile device and the moving subject.

19. A control method for an information processing apparatus, comprising: an acquisition step of acquiring operation instruction information of a moving subject involving operation and generating information on an operation instruction corresponding to the operation instruction information; and a recording step of associating and recording the captured data and the information on the operation instruction generated in the generation step each time the captured data is acquired from a photographing unit mounted on an imaging-capable mobile device that photographs a moving subject involving operation and generates the captured data. The control method is characterized by having the above steps.

20. A program for causing a computer to function as each means of the information processing apparatus according to claim 1.

Images