Information processing apparatus, method of controlling the same, information processing system, and storage medium

Abstract

To collect sound information for enhancing presence when photographing a moving image of a moving object.SOLUTION: The moving object 150 that moves by manipulation includes a plurality of microphones 211 to 218 arranged at predetermined positions, and a manipulation control unit 151 that adjusts a sound volume level for sound collection of the microphones 211 to 218 according to manipulation instruction information, and the manipulation control unit 151 collects sound information by increasing the sound volume level of a microphone arranged at or near a portion that generates sound according to the manipulation instruction information among the microphones 211 to 218 and decreasing the sound volume level of the other microphones.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus, a control method thereof, an information processing system, and a program. [Background technology]

[0002] VLOGs (Videologs), which involve filming lifestyles and everyday events, editing the videos, and posting them publicly on video streaming sites or social media, are becoming increasingly popular. One category of VLOG is called "Drive VLOGs," which are primarily shot while driving in a car, and these are generally shot using drone cameras.

[0003] As a technology that can be used to shoot driving VLOGs, for example, Patent Document 1 discloses a technology that detects the position of a vehicle (specifically, an automobile) to be filmed and controls an unmanned aerial vehicle based on the detected vehicle position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-56903 Summary of the Invention [Problem to be solved by the invention]

[0005] With the technology disclosed in Patent Document 1, when the distance between the drone and the vehicle is long, it is difficult to capture realistic sounds such as the engine noise and tire noise of the vehicle, and as a result, the video may lack a sense of realism when viewed.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device that makes it possible to collect sound information that enhances the sense of realism when shooting video of a moving subject using a mobile device. [Means for solving the problem]

[0007] The information processing device of the present invention is an information processing device that has a sound information collection means for collecting sound information and controls a moving subject that moves by operating it, and is characterized by having an acquisition means for acquiring operation instruction information for the moving subject and a control means for controlling the sound information collection means based on the operation instruction information. [Effects of the Invention]

[0008] According to the present invention, when a moving image of a moving subject is captured by a mobile device, sound information that enhances the sense of realism can be collected. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of an imaging system according to an embodiment. [Figure 2] FIG. 10 is a diagram showing an example of the location of a sound information collecting unit in a moving subject (car). [Figure 3] 10 is a flowchart illustrating control executed by a steering control unit for a moving subject. [Figure 4] 3A and 3B are diagrams showing an example of a time series of audio signals collected by a sound information collecting unit. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In this embodiment, an information processing system according to the present invention is an imaging system that captures an image of a subject that moves by maneuvering (hereinafter referred to as a "moving subject") using a mobile device (hereinafter simply referred to as a "mobile device") that has an imaging function.

[0011] In summary, this photography system, when photographing a moving subject from a mobile device, collects and records sound information (audio) generated by the moving subject in accordance with the operation or manipulation of the moving subject, thereby enhancing the sense of realism when combined with the video.

[0012] 1 is a block diagram showing a schematic configuration of an imaging system according to an embodiment. The imaging system is generally composed of a mobile device 100 and a moving subject 150. The mobile device 100 is generally composed of a mobile device body 110 and an imaging device 130.

[0013] First, a description will be given of the mobile device main body 110 that constitutes the mobile device 100. The mobile device main body 110 has a control unit 111, a ROM 112, a RAM 113, a position and orientation detection unit 114, an image processing unit 115, a movement mechanism unit 117, and a communication unit 118.

[0014] The control unit 111 has a CPU, and performs overall control of the mobile device main body 110 by reading out a control program for each block constituting the mobile device main body 110 from a ROM 112 and developing the program in a RAM 113 .

[0015] The ROM 112 is a non-volatile memory that can be electrically erased and stored, and stores parameters and the like necessary for the operation of each block in addition to programs for controlling the operation of each block provided in the mobile device main body 110. The RAM 113 is a rewritable volatile memory, and is used for expanding programs executed by the control unit 111 and the like, and for temporarily storing data generated by the operation of each block provided in the mobile device main body 110.

[0016] The position and orientation detection unit 114 is a sensor that detects various information necessary for controlling the position and orientation of the mobile device main body 110. The position and orientation detection unit 114 includes a GPS sensor that detects position, a gyro sensor that detects angular velocity, and an acceleration sensor that detects changes in velocity. The position and orientation detection unit 114 also includes a magnetic sensor that detects direction, an atmospheric pressure sensor that detects altitude, and a distance measurement sensor that detects the distance to the subject to be photographed (in this embodiment, this refers to the moving subject 150) and surrounding objects using, for example, ultrasound.

[0017] The image processing unit 115 performs various image processing operations on the video image acquired from the imaging device 130 via the communication unit 118 to control the position and attitude of the mobile device main body 110. Specifically, the image processing unit 115 detects and tracks the position of the subject being photographed relative to the position of the mobile device main body 110 in three dimensions, that is, in space.

[0018] When the mobile device main body 110 is a drone that flies and moves through the air, the mobile mechanism unit 117 is composed of a propeller, a motor that rotates the propeller, a power source that drives the motor, etc., and generates buoyancy and propulsion. Note that the mobile device main body 110 is not limited to one that flies through the air, and may be one that moves on land by rotating wheels or one that moves underwater by rotating a screw.

[0019] The communication unit 118 communicates with the image capture device 130 and the moving subject 150 using a communication method defined by a predetermined standard, such as wireless LAN.

[0020] The control unit 111 controls the movement mechanism unit 117 and the imaging device 130 so that the moving device 100 moves following the movement of the moving subject 150 and so that the moving subject 150 fits within the imaging range of the imaging device 130. At this time, the tracking result of the subject to be imaged by the image processing unit 115, the position information and orientation information of the moving device main body 110 by the position and orientation detection unit 114, and the steering instruction information and position information of the moving subject 150 transmitted from the moving subject 150 are used. The steering instruction information and position information will be described in detail later.

[0021] Next, a description will be given of the imaging device 130 that constitutes the mobile device 100. The imaging device 130 has an imaging control unit 131, a ROM 132, a RAM 133, an optical system 134, an imaging unit 135, an A / D conversion unit 136, an image processing unit 137, a video storage unit 138, a display unit 139, and an imaging communication unit 140.

[0022] The imaging control unit 131 is, for example, a CPU, and performs overall control of the imaging device 130 by reading out control programs for each block that constitutes the imaging device 130 from the ROM 132 and expanding them in the RAM 133 in accordance with command signals from the control unit 111.

[0023] The ROM 132 is a non-volatile memory that can be electrically erased and stored, and stores parameters and the like necessary for the operation of each block in addition to programs for controlling the operation of each block provided in the imaging device 130. The RAM 133 is a rewritable volatile memory, and is used for expanding programs executed by the imaging control unit 131 and the like, and for temporarily storing data generated by the operation of each block provided in the imaging device 130.

[0024] The optical system 134 has a lens group including a zoom lens and a focus lens, and an aperture mechanism, and forms an optical image of incident light on the imaging surface of an imaging element that constitutes the imaging unit 135. The imaging unit 135 has an imaging element such as a CCD sensor or a CMOS sensor, and the optical image formed on the imaging surface of the imaging element by the optical system 134 is converted into an analog image signal by photoelectric conversion and output to the A / D conversion unit 136. The A / D conversion unit 136 converts the analog image signal input from the imaging unit 135 into a digital image signal and outputs it to the RAM 133, and the RAM 133 temporarily stores the transmitted digital image signal.

[0025] The image processing unit 137 performs various image processing, such as demosaicing processing, noise reduction processing, white balance correction processing, gamma processing, etc., on the digital image signal temporarily stored in the RAM 133 depending on the purpose of display, storage (saving), etc. The image processing unit 137 stores the image data (video data) generated by performing predetermined image processing (development processing) in the video storage unit 138.

[0026] Note that image processing unit 137 generates video data for image processing unit 115 of mobile device main body 110 to perform tracking processing of moving subject 150, and transmits the generated video data to mobile device main body 110 via imaging and communication unit 140. Furthermore, when audio data is transmitted from moving subject 150 via communication unit 118 and imaging and communication unit 140, image processing unit 137 synthesizes the audio data with the generated video data by aligning the time axis. Mobile device 100 may be configured to have a single image processing unit that combines the functions of image processing unit 115 and image processing unit 137.

[0027] Video storage unit 138 has a storage medium such as a media card that stores various data including the video data and video data generated by image processing unit 137. When audio data is transmitted from moving subject 150, video storage unit 138 can also store the audio data alone.

[0028] The display unit 139 is configured with a display device such as a liquid crystal monitor, and displays images stored in the RAM 133 and images stored in the video storage unit 138 .

[0029] The imaging communication unit 140 communicates with the mobile device main body 110 using a communication method defined by a predetermined standard, such as wireless LAN. The imaging control unit 131 acquires various setting conditions of the imaging device 130 from the mobile device main body 110 via the imaging communication unit 140, and performs various settings of the imaging device 130 based on the acquired setting conditions. Note that, since the imaging device 130 is mounted on the mobile device main body 110, they may be configured to be able to communicate with each other via a wired connection using a USB cable or the like, or via connection between connectors.

[0030] The photography system may be configured so that the video captured by the imaging device 130 is transmitted to an external terminal (not shown) having a monitor via the imaging communication unit 140 and displayed on the monitor of the external terminal. This allows the user of the photography system to check the video captured by the imaging device 130 in real time. Here, the user of the photography system refers to the passengers (driver and passengers) of the moving subject 150, and to those involved in video shooting with the photography system who are not on board the moving subject 150.

[0031] Next, the configuration of moving subject 150 will be described. Moving subject 150 is something that moves according to control, and is, for example, a car, a motorcycle, a boat, an airplane, a train, etc. Moving subject 150 is not limited to something that moves by being directly controlled by a person on board, but may also be something that moves by remote control (a radio-controlled car, a radio-controlled airplane, a radio-controlled board, etc.). Furthermore, moving subject 150 is not limited to something that moves by being directly controlled by a person on board, but may also be something that moves by automatic control, which automatically controls itself while recognizing the surrounding situation even if there is a driver, or something that travels unmanned by automatic driving along a predetermined route. It is assumed that a known control method is used for the automatic control method, and a description thereof will be omitted here.

[0032] The moving subject 150 has a steering control unit 151 , a ROM 152 , a RAM 153 , a position detection unit 154 , a steering mechanism unit 155 , an operation mechanism unit 156 , a steering communication unit 157 , a sound information collection unit 158 ​​, and a sound information processing unit 159 .

[0033] Steering control unit 151 is, for example, a CPU, and is an information processing device (microcomputer) that performs overall control of moving subject 150 by reading out a control program for each block of moving subject 150 from ROM 152 and expanding it in RAM 153. Steering control unit 151 transmits steering instruction information and position information to moving device main body 110 via steering communication unit 157.

[0034] ROM 152 is a non-volatile memory that can be electrically erased and stored, and stores parameters and the like necessary for the operation of each block in addition to programs for controlling the operation of each block provided in moving subject 150. RAM 153 is a rewritable volatile memory, and is used for expanding programs executed by steering control unit 151 and the like, and for temporarily storing data generated by the operation of each block provided in moving subject 150.

[0035] Position detection unit 154 is a sensor that detects the spatial position of moving subject 150, and includes, for example, a GPS sensor, a barometric pressure sensor, etc. Position detection unit 154 can detect the spatial position of moving subject 150 by acquiring ground position information (planar position information) of moving subject 150 using the GPS sensor and acquiring height information using the barometric pressure sensor. Note that the altitude of moving subject 150 can also be detected using a GPS sensor.

[0036] Steering mechanism unit 155 is operated when moving subject 150 is operated (driven), and for example, when moving subject 150 is an automobile, steering mechanism unit 155 may include a steering wheel, accelerator pedal, brake pedal, blinker lever, wiper lever, light switch, etc. Specific examples of steering mechanism unit 155 for vehicles other than automobiles will not be described here. Note that steering mechanism unit 155 is operated by a driver aboard moving subject 150, or by automatic steering by steering control unit 151.

[0037] The steering control unit 151 detects steering or operation of the steering mechanism unit 155, generates steering instruction information corresponding to the detected steering or operation, and outputs the generated steering instruction information to the operation mechanism unit 156. The operation mechanism unit 156 operates a predetermined mechanism according to the steering instruction information. For example, if the moving subject 150 is an automobile, the steering angle is transmitted to an actuator as steering instruction information, thereby changing the direction of the front wheels. Furthermore, steering instruction information corresponding to the degree of depression of the accelerator pedal is transmitted to the actuator, which adjusts the rotation speed of the engine (or motor). Furthermore, headlights are turned on / off, blinking / off of blinker lights, and on / off of brake lights are performed according to the steering instruction information corresponding to the light switch operation, the turn signal lever operation, and the degree of depression of the brake pedal.

[0038] The steering communication unit 157 communicates with the mobile device main body 110 using a communication method defined by a predetermined standard, such as wireless LAN. The steering instruction information generated by the steering control unit 151 is transmitted to the steering communication unit 157 together with position information detected by the position detection unit 154, and is further transmitted from the steering communication unit 157 to the control unit 111 via the communication unit 118 of the mobile device main body 110. When tracking the moving subject 150 by image processing, the control unit 111 controls the movement mechanism unit 117 so that the moving subject 150 is within the shooting range of the imaging device 130, using the position information of the mobile device main body 110 and the steering instruction information and position information transmitted from the moving subject 150.

[0039] Sound information collecting unit 158 ​​is specifically microphones arranged at a plurality of predetermined positions on moving subject 150, and each microphone collects sound information (audio signals) emitted by moving subject 150. If moving subject 150 is an automobile, the microphones can be arranged near, for example, the steering wheel, accelerator pedal, brake pedal, turn signal lever, wiper lever, light switch, wheels, etc., which are steering mechanism unit 155. Note that, in addition to microphones, sound information collecting unit 158 ​​can also use microphones on the smartphone of a passenger of moving subject 150, or on a drive recorder or car navigation system equipped on moving subject 150, if these devices are equipped with microphones.

[0040] FIG. 2 is a diagram showing an example of the arrangement positions of multiple microphones that are sound information collecting unit 158 ​​when moving subject 150 is an automobile. Eight microphones 211 to 218 that are sound information collecting unit 158 ​​are arranged in automobile 201 as shown in the figure. Here, microphone 211 is arranged in the center of the interior of the automobile, microphone 212 near the window, microphone 213 near the steering wheel, microphones 214, 215, 216, and 217 near the tires, and microphone 218 near the muffler (exhaust pipe). However, the number and arrangement positions of the microphones are not limited to the embodiment shown in FIG. 2. As will be described later with reference to the flowchart of FIG. 3, steering control unit 151 adjusts the volume levels for sound collection by microphones 211 to 218 based on steering instruction information generated in response to operation of steering mechanism unit 155.

[0041] The sound information processing unit 159 performs various signal processing on the audio signals collected by the sound information collecting unit 158 ​​in response to instructions from the steering control unit 151. Specifically, it performs noise cancellation, synthesis of audio signals collected by each microphone, sound enhancement processing of a specific frequency band, etc. The sound information processing unit 159 has a storage medium (e.g., a media card) and can store the audio signals that have undergone the aforementioned predetermined processing as audio data. In addition, the steering control unit 151 can transmit the audio data generated by the sound information processing unit 159 to the mobile device main body 110 via the steering communication unit 157.

[0042] Next, a processing flow when photographing moving subject 150 by mobile device 100 will be described. Fig. 3 is a flowchart explaining the control executed by operation control unit 151 of moving subject 150 in a series of controls executed in the photographing system. Each process (step) indicated by an S number in this flowchart is realized by operation control unit 151 executing a predetermined program and controlling the operation of each block of moving subject 150. Note that here, a case will be described in which moving subject 150 is an automobile and mobile device main body 110 is a drone.

[0043] Assume that the start of capturing the moving subject 150 is performed, for example, as follows. That is, when the engine (or motor) of the moving subject 150 (automobile) starts, operation instruction information indicating the start of the engine (or motor) and position information are transmitted from the moving subject 150 to the mobile device 100 (drone). The control unit 111 of the mobile device main body 110 starts position control and drive control of the mobile device 100 based on this information and its own position information. The control unit 111 of the mobile device main body 110 also instructs the imaging control unit 131 of the imaging device 130 to start capturing the moving subject 150, that is, to start recording video, which causes the imaging device 130 to start capturing the moving subject 150. After the start of capturing, the image processing unit 115 of the mobile device main body 110 detects and tracks the moving subject 150 as described above using the captured image transmitted from the imaging device 130. Note that the method for starting capturing is not limited to this, and for example, the user may manually perform the capturing at any timing when or before the moving subject 150 starts to capture the moving subject 150.

[0044] In S301, the steering control unit 151 determines whether or not it has detected an operation on the steering mechanism unit 155. If it determines that it has not detected an operation (NO in S301), the steering control unit 151 executes the processing of S302.

[0045] In S302, the steering control unit 151 increases the volume level (sound collection sensitivity) of the microphone 211 arranged in the center of the vehicle interior to collect sound information (collect sound) inside the vehicle. Then, in S303, the steering control unit 151 lowers the volume levels of the other microphones 212 to 218. This is because, when collecting sound using multiple microphones, if multiple sounds are combined at the same volume level, it may become difficult to determine what sound is the main purpose of the sound being collected; in other words, the sound may sound like noise. Therefore, in S202 to S203, sound is mainly collected using the microphone 211 until steering or operation is detected. The steering control unit 151 executes the process of S314 after the process of S303.

[0046] Although the degree of detection of steering or operation differs depending on the mechanisms of the steering mechanism unit 155, if a minute movement is detected, it will be detected as an operation occurring for almost the entire period. Therefore, taking the steering operation of an automobile as an example, it is desirable not to detect steering operation when the vehicle is being driven at a substantially constant speed or on a straight road.

[0047] If the operation control unit 151 determines in S301 that an operation has been detected (YES in S301), it executes the process of S304.

[0048] In S304, the steering control unit 151 lowers the volume level of the microphone 211 located in the center of the vehicle. Then, in S305 and thereafter, the volume level of a predetermined microphone other than the microphone 211 is increased in accordance with the detected steering or operation. As a result, similar to the above description regarding S302 to S303, it is possible to mainly pick up sounds generated in accordance with the detected steering or operation, and to prevent the picked up voice from sounding like noise.

[0049] If the microphone has a noise cancellation function, that function may be used. For example, as disclosed in Japanese Patent Application Laid-Open No. 2000-242277, the audio signal picked up by the microphone whose volume level you want to lower is considered a noise signal, and a noise cancellation signal that cancels out the audio signal is generated and then emitted. This makes it possible to suppress the sound picked up by the microphone whose volume level you want to lower.

[0050] In S305, the operation control unit 151 determines whether the operation or manipulation detected in S301 is a window opening / closing manipulation. If the operation control unit 151 determines that the operation is a window opening / closing manipulation (YES in S305), it executes the process of S306, and if the operation is not a window opening / closing manipulation (NO in S305), it executes the process of S307.

[0051] In S306, the steering control unit 151 increases the volume level of the microphone 212 disposed near the window, and then executes the process of S308. On the other hand, in S307, the steering control unit 151 decreases the volume level of the microphone 212 disposed near the window, and then executes the process of S308.

[0052] The processing of S306 and S307 will now be described with reference to Fig. 4. Fig. 4 is a diagram showing, in time series, an example of an audio signal 400 picked up by microphone 211 in the center of the vehicle interior and another audio signal 410 picked up by microphone 212 near the window. During a period 421 in which moving subject 150 (automobile) is not being steered or operated, the volume level of microphone 211 in the center of the vehicle interior is increased, while the volume level of microphone 212 located near the window is decreased. Although not shown, the volume levels of microphones 213 to 218 are also decreased to match the volume level of microphone 212.

[0053] During period 422 when the window was opened or closed, the volume level of main microphone 212 was increased, while the volume level of microphone 211 was decreased. The volume levels of microphones 213 to 218 are maintained at the decreased level during period 422, as they were during period 421. During period 423, there is again no steering or operation, so the volume levels of microphones 211 to 218 are set to the same level as during period 421.

[0054] Returning to the explanation of the flowchart in Fig. 3, in S308, the steering control unit 151 determines whether the steering or operation detected in S301 is a steering operation. If the steering control unit 151 determines that it is a steering operation (YES in S308), it executes the process of S309, and if it determines that it is not a steering operation (NO in S308), it executes the process of S310.

[0055] In S309, the steering control unit 151 increases the volume level of the microphone 213 disposed near the steering wheel, and then executes the process of S311. In S310, the steering control unit 151 decreases the volume level of the microphone 213 disposed near the steering wheel, and then executes the process of S311.

[0056] In S311, the steering control unit 151 determines whether the steering or operation detected in S301 is an accelerator / brake operation. If the steering control unit 151 determines that the steering or operation is an accelerator / brake operation (YES in S311), it executes the process of S312, and if it determines that the steering or operation is not an accelerator / brake operation (NO in S311), it executes the process of S313.

[0057] In S312, the steering control unit 151 increases the volume levels of the microphones 214 to 217 arranged near the tires and the microphone 218 arranged near the muffler, and then executes the process of S314. In S313, the steering control unit 151 decreases the sound collection levels of the microphones 214 to 218, and then executes the process of S314.

[0058] Note that the frequency of the audio to be picked up may be adjusted when the accelerator or brake is operated. In the case of engine-powered vehicles, the engine sound generally becomes higher in pitch in proportion to the engine speed and number of cylinders, as expressed by the relationship: engine sound [Hz] = {engine speed [rpm] × number of cylinders} / 60. Furthermore, since the number of cylinders is a fixed value for each vehicle, the engine sound is essentially determined by the engine speed. Therefore, when the accelerator is pressed and the engine speed increases, high-frequency sounds are emitted from the vehicle. Therefore, by not only increasing the pickup level of microphones 214 to 318 but also performing processing to emphasize the frequency band calculated from the engine speed and the above formula when the accelerator is pressed, it is possible to not only enhance the sense of realism but also capture sounds that are comfortable to drive.

[0059] In S314, the steering control unit 151 synthesizes the sounds collected by the microphones 211 to 218 to create audio data. The audio synthesis method is not particularly limited, and may be a known method (for example, averaging in real space and frequency space). The steering control unit 151 may transmit the synthesized audio data to the imaging device 130 and / or store it in the sound information processing unit 159. When audio data is transmitted to the imaging device 130 via the mobile apparatus main body 110, as described above, the image processing unit 137 can synthesize the video data and audio data to generate and store video data. When the video data and audio data are stored separately, video data can be generated by synthesizing the data using, for example, a personal computer after shooting is completed.

[0060] In S315, the steering control unit 151 determines whether or not to end the shooting. For example, the steering control unit 151 determines to end the shooting when it detects that the shift lever of the moving subject 150 (automobile) is put into park or that the engine is turned off. In this case, the control unit 111 of the moving apparatus main body 110 ends the shooting by the imaging device 130 in accordance with the operation instruction information indicating that the shift lever of the moving subject 150 is put into park or that the engine is turned off. If the steering control unit 151 determines to end the shooting (YES in S315), it ends this processing, and if it determines to continue the shooting (NO in S315), it executes the processing of S301.

[0061] As explained above, according to this embodiment, sound data is generated by collecting sound information of moving subject 150, mainly from areas where sound is generated in response to the operation or manipulation performed on moving subject 150. In this way, by synthesizing the generated sound data and video data in line with the time axis, it is possible to provide a highly realistic video in which the sound and video in response to the operation or manipulation are linked.

[0062] While the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0063] For example, in the above embodiment, steering control unit 151 provided in moving subject 150 determines the main microphone when collecting sound information generated by moving subject 150 based on steering instruction information. The configuration is not limited to this, and for example, a configuration may be adopted in which a server (not shown) collectively controls mobile device main body 110, imaging device 130, and moving subject 150, and changes the main part of moving subject 150 when collecting sound information based on steering instruction information. In this case, a configuration may be adopted in which the server generates video data by synthesizing video data obtained by mobile device 100 and audio data obtained by moving subject 150, aligning the time axes.

[0064] Furthermore, control unit 111 of mobile device main body 110 receives steering instruction information from moving subject 150. Therefore, it is also possible to configure control unit 111 of mobile device 100 to adjust the volume level based on the steering instruction information of multiple microphones that collect sound at moving subject 150 by steering control unit 151 (control unit 111 functions as the above-mentioned server).

[0065] Furthermore, an image processing unit may be provided in the moving subject 150, and this image processing unit may receive video data from the moving device 100, and the received video data may be synthesized with the audio signal generated by the audio information processing unit 159, aligning the time axes, to generate video data.

[0066] Also, the configuration has been described in which the mobile device 100 is configured to capture images by including the control unit 111 and the imaging control unit 131. However, the present invention is not limited to this, and the mobile device 100 may be configured to include a single control unit that controls the drive of the mobile device main body 110 and the imaging of the imaging device 130.

[0067] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0068] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) An information processing device that has a sound information collection means for collecting sound information and controls a moving subject that moves by operating it, characterized in that it has an acquisition means for acquiring operation instruction information for the moving subject, and a control means for controlling the sound information collection means based on the operation instruction information. (Configuration 2) The information processing device described in Configuration 1, characterized in that the sound information collection means is microphones arranged at multiple positions on the moving subject, and the control means adjusts the volume levels for sound collection by the multiple microphones in accordance with the steering instruction information. (Configuration 3) The information processing device according to Configuration 2, further comprising signal processing means for performing predetermined signal processing on the audio signal collected by the sound information collecting means. (Configuration 4) The information processing device described in Configuration 3, wherein the control means increases the volume level of a microphone among the plurality of microphones that is located at or near a location that generates sound in response to the steering instruction information, and decreases the volume levels of the other microphones. (Configuration 5) The information processing device according to Configuration 4, characterized in that the signal processing means regards the audio signal picked up by the other microphone as a noise signal, generates a noise cancellation signal that cancels out the noise signal, and emits the noise cancellation signal. (Configuration 6) The information processing device according to Configuration 5, wherein the signal processing means performs processing to emphasize an audio signal in a specific frequency band in accordance with the steering instruction information. (Configuration 7) The information processing device according to configuration 6, wherein the moving subject is equipped with an engine, and the signal processing means emphasizes an audio signal in a frequency band of the engine sound calculated from the rotation speed and number of cylinders of the engine. (Configuration 8) An information processing device according to any one of configurations 1 to 7, characterized in that it comprises a transmitting means for transmitting the audio signal collected by the sound information collecting means to a moving device photographing the moving subject by following the movement of the moving subject. (Configuration 9) An information processing device according to any one of configurations 1 to 7, characterized in that it comprises an image processing means for creating video data by synthesizing, along a time axis, the audio signal acquired by the sound information collecting means and video data acquired from a mobile device that follows the movement of the moving subject and photographs the moving subject. (Configuration 10) The information processing device described in any one of configurations 1 to 9, characterized in that the information processing device is a moving device that follows the movement of the moving subject and photographs the moving subject, or is mounted on the moving subject. (Configuration 11) The information processing device described in any one of configurations 1 to 9, characterized in that the information processing device is a mobile device that follows the movement of the moving subject and photographs the moving subject, and a server that is communicatively connected to the moving subject. (Configuration 12) A moving subject that moves by being controlled, comprising: microphones arranged at multiple predetermined positions; and a control means for adjusting the volume levels for sound pickup by the multiple microphones in accordance with control instruction information, wherein the control means increases the volume levels of the microphones arranged at or near the location that generates sound in accordance with the control instruction information, and decreases the volume levels of the other microphones. (Configuration 13) An information processing system having a mobile device equipped with an imaging means and a driving means for movement, and a moving subject that moves by being controlled, wherein the moving subject has microphones arranged at multiple predetermined positions, a first control means that controls sound collection from the multiple microphones based on steering instruction information generated by the moving subject, and a transmitting means that transmits the steering instruction information to the mobile device, and the mobile device has a receiving means that receives the steering instruction information, and a second control means that controls the imaging means and the driving means so that the moving subject is photographed by the imaging means while moving the mobile device to follow the movement of the moving subject based on the steering instruction information. (Method 1) A control method for an information processing device that controls a moving subject that moves by being controlled, the moving subject having a sound information collection means for collecting sound information generated by the moving subject, the control method for an information processing device comprising the steps of: acquiring control instruction information for the moving subject; and controlling the collection of sound information by the sound information collection means based on the control instruction information. (Program 1) A program for causing a computer to execute each step of the control method for an information processing device according to Method 1. [Explanation of symbols]

[0069] 100 Mobile Device 110 Mobile device body 111 Control Unit 115 Image processing section 117 Moving mechanism section 130 Imaging device 150 Moving subject 151 Steering control unit 158 Sound Information Collection Unit 159 Sound information processing section 155 Control mechanism 211~218 Mike

Claims

1. An information processing device that includes a sound information collecting means for collecting sound information and controls a moving subject that moves by operation, acquisition means for acquiring operation instruction information for the moving subject; and a control means for controlling the sound information collecting means based on the steering instruction information.

2. the sound information collecting means is microphones arranged at a plurality of positions on the moving subject, 2. The information processing apparatus according to claim 1, wherein the control means adjusts volume levels for collecting sounds from the plurality of microphones in accordance with the operation instruction information.

3. 3. The information processing apparatus according to claim 2, further comprising signal processing means for performing predetermined signal processing on the audio signals collected by said sound information collecting means.

4. 4. The information processing device according to claim 3, wherein the control means increases the volume level of a microphone among the plurality of microphones that is located at or near a location that generates sound in response to the operation instruction information, and decreases the volume levels of the other microphones.

5. 5. The information processing apparatus according to claim 4, wherein the signal processing means regards the audio signals picked up by the other microphones as noise signals, and generates and emits noise cancellation signals that cancel out the noise signals.

6. 6. The information processing apparatus according to claim 5, wherein the signal processing means performs processing to emphasize an audio signal in a specific frequency band in accordance with the operation instruction information.

7. the moving object has an engine; 7. The information processing apparatus according to claim 6, wherein said signal processing means enhances an audio signal in a frequency band of engine sound calculated from the rotation speed and number of cylinders of said engine.

8. 8. The information processing apparatus according to claim 1, further comprising a transmitting means for transmitting the audio signal collected by the sound information collecting means to a moving device that is photographing the moving subject by following the movement of the moving subject.

9. The information processing device according to any one of claims 1 to 7, characterized in that it comprises an image processing means for creating video data by synthesizing, along a time axis, the audio signal acquired by the sound information collecting means and video data acquired from a mobile device that follows the movement of the moving subject and photographs the moving subject.

10. 8. The information processing device according to claim 1, wherein the information processing device is mounted on a moving device that follows the movement of the moving subject and photographs the moving subject, or the information processing device is mounted on the moving subject.

11. 8. The information processing device according to claim 1, wherein the information processing device is a mobile device that follows the movement of the moving subject and photographs the moving subject, and a server that is communicably connected to the moving subject.

12. A moving subject that moves by being steered, a plurality of microphones arranged at predetermined positions; a control means for adjusting a volume level for collecting sound from the plurality of microphones in accordance with the operation instruction information; The control means increases the volume level of a microphone among the plurality of microphones that is located at or near a portion that generates sound in response to the steering instruction information, and decreases the volume levels of the other microphones.

13. a moving device including an imaging means and a driving means for moving; a moving subject that moves by operation, The moving subject is a plurality of microphones arranged at predetermined positions; a first control means for controlling sound collection from the plurality of microphones based on steering instruction information generated by the moving subject; a transmitting means for transmitting the steering instruction information to the mobile device, The moving device is a receiving means for receiving the steering instruction information; and a second control means for controlling the imaging means and the driving means so that the imaging means captures an image of the moving subject while moving the moving device in accordance with the movement of the moving subject based on the steering instruction information.

14. A control method for an information processing device that controls a moving subject that moves by operation and has sound information collecting means for collecting sound information generated by the moving subject, acquiring steering instruction information for the moving subject; and controlling the collection of sound information by said sound information collecting means based on said steering instruction information.

15. 15. A program for causing a computer to execute each step of the method for controlling an information processing apparatus according to claim 14.

Images