Audio processing device, control method, and program

Abstract

To provide a sound processing device capable of effectively reducing noise in a configuration where detachable microphones are connected.SOLUTION: A sound processing device includes: connection means for connecting first microphones to acquire environment sound; and a second microphone for acquiring sound generated in a noise source. The sound processing device includes: first transformation means for generating a first sound signal by performing Fourier transformation of a digital sound signal based on the sound acquired by the first microphones connected by the connection means; second transformation means for generating a second sound signal by performing Fourier transformation of the digital sound signal based on the sound acquired by the second microphone; noise data generation means for generating noise data obtained by correcting the second sound signal based on first transmission parameters related to noise transmission from the noise source to the first microphones and second transmission parameters related to noise transmission from the noise source to the second microphone; and subtraction means for subtracting noise data from the first sound signal.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an audio processing device capable of performing noise reduction processing on an audio signal. [Background technology]

[0002] A digital camera, which is an example of an audio processing device, can record surrounding audio when recording video data. Digital cameras also have an autofocus function that drives an optical lens to focus on a subject while recording video data. Digital cameras also have a zoom function that drives the optical lens while recording video.

[0003] In this way, when an optical lens is driven while recording a video, the driving sound of the optical lens may be included as noise in the audio recorded along with the video. Therefore, in conventional digital cameras, if sliding sounds generated when the optical lens is driven are picked up as noise, the noise is reduced and the surrounding audio is recorded. A related technique is proposed in Patent Document 1. In Patent Document 1, an imaging device having an external sound microphone for picking up external sounds and a mechanical sound microphone for picking up device mechanical sounds is equipped with a storage unit for storing transfer function information from the mechanical sound microphone to the external sound microphone, and a mechanical sound cancellation signal is generated by filtering based on the transfer function information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-110629 Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned Patent Document 1, when a detachable microphone such as an external microphone is connected, calibration is required to generate a transfer function by recording the sound generated by moving a drive unit in a quiet environment using a microphone for external sound and a microphone for mechanical sound. However, it is difficult for a typical user to prepare an ideal quiet environment such as an anechoic chamber and perform calibration. As such, if calibration cannot be performed in an ideal quiet environment, there is a risk that noise contained in the captured sound cannot be effectively reduced.

[0006] Therefore, an object of the present invention is to effectively reduce noise in a configuration in which a detachable microphone is connected. [Means for solving the problem]

[0007] In order to achieve the above object, the voice processing device of the present invention includes a connection means for connecting a first microphone for acquiring environmental sounds, and a second microphone for acquiring sounds generated by a noise source. and, a digital audio signal based on the audio captured by the first microphone connected by the connection means; of a first conversion means for performing a Fourier transform to generate a first audio signal; and a digital audio signal based on the audio captured by the second microphone. of second transforming means for performing a Fourier transform to generate a second audio signal; a first calculation means for calculating a first transfer parameter relating to the transmission of noise from the noise source to the first microphone based on transfer parameters of components arranged on the noise transmission path from the noise source to the first microphone; a second calculation means for calculating a second transfer parameter relating to the transmission of noise from the noise source to the second microphone based on transfer parameters of components arranged on the noise transmission path from the noise source to the second microphone; a first transfer parameter and The aforementioned a noise data generating means for generating noise data obtained by correcting the second audio signal based on a second transfer parameter; and a subtracting means for subtracting the noise data from the first audio signal. The first calculation means calculates the first transfer parameter by multiplying the transfer parameters of each component arranged on the noise propagation path from the noise source to the first microphone, and the second calculation means calculates the second transfer parameter by multiplying the transfer parameters of each component arranged on the noise propagation path from the noise source to the second microphone. It is characterized by the following. [Effects of the Invention]

[0008] According to the present invention, noise can be effectively reduced in a configuration in which a detachable microphone is connected. [Brief explanation of the drawings]

[0009] [Figure 1]1 is an external view of an imaging device as an audio processing device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing a schematic configuration of the imaging device of FIG. [Figure 3] 3 is a block diagram showing the configuration of an audio input unit and an external microphone unit in FIG. 2. FIG. [Figure 4] FIG. 3 is a diagram for explaining the locations of the L microphone, the R microphone, and the noise microphone in FIG. 2. [Figure 5] 2 is a diagram for explaining a configuration in which detachable parts are connected to the imaging device of FIG. 1. FIG. [Figure 6] FIG. 2 is a diagram showing an operation flow of the imaging device of FIG. [Figure 7] FIG. 4 is a diagram illustrating an example of a transfer function according to the present embodiment. [Figure 8] 10 is a diagram showing the relationship between a transfer function Ter, a transfer function Tnr, and a correction coefficient C. FIG. [Figure 9] 10A and 10B are diagrams for explaining noise reduction in the present embodiment. [Figure 10] FIG. 10 is an external view of an imaging device as an audio processing device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] 1A and 1B are external views of an image capturing device 100 as an audio processing device according to an embodiment of the present invention. Fig. 1A is an example of a front perspective view of the image capturing device 100. Fig. 1B is an example of a rear perspective view of the image capturing device 100.

[0012] The lens mount 101 is a mounting portion to which a lens unit can be attached. In this embodiment, for example, the optical lens unit 200 shown in FIG. 2 (described later) is attached to the lens mount 101. The display portion 102 is provided on the rear surface of the imaging device 100. The display portion 102 displays image data, text information, and the like. The extra-finder display portion 103 is a display portion provided on the top surface of the imaging device 100. The extra-finder display portion 103 displays settings for the imaging device 100, such as the shutter speed and aperture value. The eyepiece viewfinder 104 is a peer-type viewfinder. The user can check the focus and composition of the optical image of the subject by observing the focusing screen in the eyepiece viewfinder 104.

[0013] The release switch 105 is an operation member that allows the user to issue a shooting instruction. The mode selector switch 106 is an operation member that allows the user to switch between various modes. The main electronic dial 107 is a rotary operation member. The user can change the settings of the imaging device 100, such as the shutter speed and aperture value, by turning the main electronic dial 107. The release switch 105, mode selector switch 106, and main electronic dial 107 are included in an operation unit 211 in FIG. 2, which will be described later.

[0014] The power switch 108 is an operating member that switches the power of the imaging device 100 on and off. The sub electronic dial 109 is a rotary operating member. The user can use the sub electronic dial 109 to move a selection frame displayed on the display unit 102 and to advance images in playback mode. The cross key 110 is a cross key (four-way key) that can be pressed up, down, left, or right. The imaging device 100 executes processing according to the part (direction) of the cross key 110 that is pressed. The power switch 108, the sub electronic dial 109, and the cross key 110 are also included in an operation unit 211, which will be described later.

[0015] The SET button 111 is a push button. The SET button 111 is mainly used by the user to confirm a selection item displayed on the display unit 102, etc. The LV button 112 is a button used to switch live view (hereinafter referred to as "LV") on and off. The LV button 112 is used to start and stop video shooting (recording) in video recording mode. The enlarge button 113 is a push button for turning enlargement mode on and off and changing the magnification ratio in enlargement mode in live view display in shooting mode. The enlargement button 113 also functions as a button for increasing the magnification ratio of image data displayed on the display unit 102 in playback mode. The SET button 111, LV button 112, and enlargement button 113 are also included in the operation unit 211, which will be described later. The reduce button 114 is a button for decreasing the magnification ratio of image data enlarged and displayed on the display unit 102. The playback button 115 is an operation button for switching between shooting mode and playback mode. When the user presses the playback button 115 during the shooting mode, the imaging device 100 transitions to the playback mode, and the image data recorded on the recording medium 209 is displayed on the display unit 102. The reduction button 114 and the playback button 115 are also included in the operation unit 211, which will be described later.

[0016] The quick-return mirror (hereinafter simply referred to as "mirror") 116 is a mirror that switches the light beam incident from the optical lens unit 200 attached to the imaging device 100 so that it is incident either on the eyepiece finder 104 side or on the imaging unit 201 side (described later in FIG. 2). The control unit 210 controls an actuator (not shown) to raise and lower the mirror 116 during exposure, live view shooting, and video shooting. The mirror 116 is normally positioned to direct the light beam toward the eyepiece finder 104. When shooting or displaying live view, the mirror 116 flips up (mirror up) to direct the light beam toward the imaging unit 201. The center of the mirror 116 is a half mirror. A portion of the light beam that passes through the center of the mirror 116 is incident on a focus detection unit (not shown) that performs focus detection.

[0017] In this embodiment, a configuration in which the imaging device 100 includes a mirror 116 will be described, but the present invention is not limited to this configuration, and a so-called mirrorless configuration without the mirror 116 may also be used. In a mirrorless configuration, the light beam incident from the optical lens unit 200 (described later) is always incident on the imaging unit 201, and the imaging unit 201 not only converts the optical image into an electronic signal but also simultaneously performs focus detection, thereby eliminating the need for the mirror 116 or focus detection unit. Furthermore, in a mirrorless configuration, the eyepiece finder 104 is not an optical finder that displays the light beam incident from the optical lens unit 200 (described later), but an electronic finder that displays an image converted into an electrical signal by the imaging unit 201.

[0018] The communication terminal 117 is a communication terminal through which the imaging device 100 communicates with an optical lens unit 200 (described later) attached to the lens mount 101. The terminal cover 118 is a cover for protecting a connector (not shown) for connecting an external device to the imaging device 100. The lid 119 is a lid for a slot that stores a recording medium 209 (described later in FIG. 2). The built-in L microphone 120a and the built-in R microphone 120b are microphones for picking up the user's voice and the like. When viewed from the rear of the imaging device 100, the built-in L microphone 120a is located on the left side and the built-in R microphone 120b is located on the right side. The accessory shoe 121 is a mounting portion to which an external device can be attached. In this embodiment, for example, an external microphone unit 215 (described later in FIG. 2) is mounted on the accessory shoe 121.

[0019] FIG. 2 is a block diagram showing a schematic configuration of the imaging device 100 of FIG.

[0020] The optical lens unit 200 is a lens unit that is detachable from the imaging device 100, and includes an optical lens (not shown), a motor (not shown) for driving the optical lens, and a communication unit (not shown) that communicates with a lens control unit 202 (described later). For example, the optical lens unit 200 is a zoom lens or a varifocal lens. The optical lens unit 200 performs focusing and zooming on a subject and corrects camera shake by moving the optical lens using the motor based on a control signal received by the communication unit.

[0021] The imaging unit 201 includes an imaging element for converting an optical image of a subject formed on an imaging surface via the optical lens unit 200 into an electrical signal, and an image processing unit (not shown) for generating and outputting image data or video data from the electrical signal generated by the imaging element. The imaging element is, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). In this embodiment, a series of processes for generating image data including still image data and video data in the imaging unit 201 and outputting the image data from the imaging unit 201 is referred to as "photographing." In the imaging device 100, the image data is recorded on a recording medium 209 (described later) in accordance with the DCF (Design rule for Camera File system) standard.

[0022] The lens control unit 202 controls the optical lens unit 200 by transmitting a control signal to the optical lens unit 200 via the communication terminal 117 based on data output from the imaging unit 201 and a control signal output from a control unit 210 (described later). The information acquisition unit 203 detects the tilt of the imaging device 100 and the temperature inside the housing of the imaging device 100. For example, the information acquisition unit 203 detects the tilt of the imaging device 100 using an acceleration sensor or gyro sensor (not shown). The information acquisition unit 203 also detects the temperature inside the housing of the imaging device 100 using a temperature sensor (not shown).

[0023] The audio input unit 204 includes a microphone. The detailed configuration of the audio input unit 204 will be described later. The audio input unit 204 acquires audio around the imaging device 100 using the microphone, and performs analog-to-digital conversion (A / D conversion) and various audio processes on the acquired audio to generate audio data. The volatile memory 205 temporarily stores image data generated by the imaging unit 201 and audio data generated by the audio input unit 204. The volatile memory 205 is also used as a temporary storage area for image data displayed on the display unit 102, a work area for the control unit 210, etc.

[0024] The display control unit 206 controls the display unit 102 to display image data output from the imaging unit 201, characters for interactive operations, menu screens, etc. Furthermore, when capturing still images and videos, the display control unit 206 controls the display unit 102 to sequentially display digital data output from the imaging unit 201, thereby allowing the display unit 102 to function as an electronic viewfinder. For example, the display unit 102 is a liquid crystal display or an organic EL display. Furthermore, the display control unit 206 can also control the image data and video data output from the imaging unit 201, characters for interactive operations, menu screens, etc. to be displayed on an external display via the external input / output unit 214, which will be described later.

[0025] The encoding processing unit 207 can encode the image data and audio data temporarily stored in the volatile memory 205. For example, the encoding processing unit 207 can generate moving image data by encoding and compressing image data according to the JPEG standard or a RAW image format. The encoding processing unit 207 can also generate moving image data by encoding and compressing video data according to the MPEG2 standard or the H.264 / MPEG4-AVC standard. The encoding processing unit 207 can also generate audio data by encoding and compressing audio data according to the AC3AAC standard, the ATRAC standard, or the ADPCM method. Note that the encoding processing unit 207 may also encode audio data without data compression according to, for example, the Linear PCM method.

[0026] The recording control unit 208 records data on the recording medium 209 and reads data from the recording medium 209. For example, the recording control unit 208 can record still image data, video data, and audio data generated by the encoding processing unit 207 on the recording medium 209 and read these data from the recording medium 209. The recording medium 209 is, for example, an SD card, a CF card, an XQD memory card, an HDD (magnetic disk), an optical disk, or a semiconductor memory. The recording medium 209 may be configured to be detachable from the imaging device 100, or may be built into the imaging device 100. That is, the recording control unit 208 only needs to have at least a means for accessing the recording medium 209.

[0027] The control unit 210 controls each component of the imaging device 100 via a data bus 216 in accordance with input signals and programs. The control unit 210 includes a CPU, ROM, and RAM (not shown) for executing various controls. Note that instead of the control unit 210 controlling the entire imaging device 100, multiple pieces of hardware may share the control of the entire imaging device 100. The ROM included in the control unit 210 stores programs for controlling each component. The RAM included in the control unit 210 is a volatile memory used for arithmetic processing and the like.

[0028] The operation unit 211 is a user interface for receiving instructions from the user for the imaging device 100. The operation unit 211 includes, for example, a power switch 108 for turning the power of the imaging device 100 on or off, a release switch 105 for issuing an instruction to capture an image, a playback button 115 for issuing an instruction to play image data or video data, and a mode switch 106. The operation unit 211 also includes a touch panel formed on the display unit 102. The operation unit 211 outputs a control signal to the control unit 210 in response to a user's operation. The release switch 105 includes SW1 and SW2. When the release switch 105 is pressed halfway, SW1 is turned on. This allows the operation unit 211 to receive preparation instructions for performing imaging preparation operations such as AF (autofocus) processing, AE (autoexposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing. When the release switch 105 is pressed fully, SW2 is turned on. By such a user operation, an imaging instruction for performing an imaging operation is accepted. Furthermore, the operation unit 211 includes an operation member (for example, a button) that can adjust the volume of audio data reproduced from a speaker 213, which will be described later.

[0029] The audio output unit 212 acquires audio data read from the recording medium 209 by the recording control unit 208, audio data output from the nonvolatile memory 217, and audio data output from the encoding processing unit. The audio output unit 212 outputs the acquired audio data to a speaker 213 and an external input / output unit 214. The speaker 213 is an electro-acoustic transducer capable of reproducing audio data. The external input / output unit 214 outputs image data, video data, audio data, etc. to an external device. The external input / output unit 214 also acquires various data from external devices. For example, the external input / output unit 214 acquires audio data from an external microphone unit 215 attached to the imaging device 100. The detailed configuration of the external microphone unit 215 will be described later. The external input / output unit 214 is composed of, for example, a video terminal, a microphone terminal, a headphone terminal, etc. The data bus 216 is a data bus for transmitting various data such as audio data, video data, image data, etc., and various control signals to each block of the imaging device 100. The nonvolatile memory 217 stores programs and the like executed by the control unit 210. In addition, sound data is recorded in the nonvolatile memory 217. This sound data is, for example, sound data of electronic sounds such as a focusing sound that is output when a subject is focused, an electronic shutter sound that is output when an instruction to shoot is given, and an operation sound that is output when the imaging device 100 is operated.

[0030] Next, we will explain the operation of the imaging device 100. When the user turns on the power supply of the imaging device 100 by operating the power switch 108, the imaging device 100 supplies power to each component of the imaging device from a power supply (not shown). For example, the power supply is a battery such as a lithium ion battery or an alkaline manganese dry battery.

[0031] When power is supplied, the control unit 210 determines in which of a plurality of modes, including a moving image recording mode or a playback mode, the operation mode will be based on the state of the mode selector switch 106. In the moving image recording mode, the control unit 210 records the moving image data output from the imaging unit 201 and the audio data output from the audio input unit 204 as one piece of moving image data with audio. In the playback mode, the control unit 210 controls the recording control unit 208 to read out the image data or moving image data recorded on the recording medium 209 and display it on the display unit 102.

[0032] First, the moving image recording mode will be described. In the moving image recording mode, the control unit 210 first transmits a control signal to each component of the imaging device 100 to transition the imaging device 100 to a shooting standby state. For example, the control unit 210 controls the imaging unit 201 and the audio input unit 204 to perform the following operations.

[0033] The imaging unit 201 converts the optical image of the subject formed on the imaging surface via the optical lens unit 200 into an electrical signal, and generates video data from the electrical signal generated by the imaging element. The imaging unit 201 then transmits the video data to the display control unit 206, which displays it on the display unit 102. The user can prepare for shooting while viewing the video data displayed on the display unit 102.

[0034] The audio input unit 204 performs A / D conversion on analog audio signals input from multiple microphones, respectively, to generate multiple digital audio signals. The audio input unit 204 then generates audio data for multiple channels from the multiple digital audio signals. The audio input unit 204 transmits the generated audio data to the audio output unit 212, which then plays the audio data from the speaker 213. While listening to the audio data played back from the speaker 213, the user can use the operation unit 211 to adjust the volume of the audio data recorded in the audio-accompanied video data.

[0035] Next, in response to the user pressing the LV button 112, the control unit 210 transmits an instruction signal to start shooting to each component of the image capturing device 100. For example, the control unit 210 controls the image capturing unit 201, the audio input unit 204, the encoding processing unit 207, and the recording control unit 208 to perform the following operations.

[0036] The imaging unit 201 converts an optical image of a subject formed on an imaging surface via the optical lens unit 200 into an electrical signal, and generates video data from the electrical signal generated by the imaging element. The imaging unit 201 then transmits the video data to the display control unit 206, which displays it on the display unit 102. The imaging unit 201 also transmits the generated video data to the volatile memory 205.

[0037] The audio input unit 204 performs A / D conversion on analog audio signals input from multiple microphones to generate multiple digital audio signals. The audio input unit 204 then generates multi-channel audio data from the multiple digital audio signals. The audio input unit 204 then transmits the generated audio data to the volatile memory 205.

[0038] The encoding processing unit 207 reads out and encodes the video data and audio data temporarily recorded in the volatile memory 205. The control unit 210 generates a data stream from the video data and audio data encoded by the encoding processing unit 207 and outputs the data stream to the recording control unit 208. The recording control unit 208 records the acquired data stream as video data with audio on the recording medium 209 in accordance with a file system such as UDF or FAT.

[0039] The components of the image capturing apparatus 100 continue to perform the above operations during video capture.

[0040] Then, in response to the user pressing the LV button 112, the control unit 210 transmits an instruction signal to end shooting to each component of the imaging device 100. For example, the control unit 210 controls the imaging unit 201, the audio input unit 204, the encoding processing unit 207, and the recording control unit 208 to perform the following operations.

[0041] The imaging unit 201 stops generating video data. The audio input unit 204 stops generating audio data. The encoding processing unit 207 reads and encodes the remaining video data and audio data recorded in the volatile memory 205. The control unit 210 generates a data stream from the video data and audio data encoded by the encoding processing unit 207, and outputs the data stream to the recording control unit 208.

[0042] The recording control unit 208 records the data stream as a file of audio-accompanying moving image data on the recording medium 209 in accordance with a file system such as UDF or FAT. Then, when the input of the data stream stops, the recording control unit 208 completes the audio-accompanying moving image data. Upon completion of the audio-accompanying moving image data, the recording operation of the imaging device 100 stops.

[0043] In response to the stop of the recording operation, the control unit 210 transmits a control signal to each component of the image capturing device 100 to transition to a shooting standby state. As a result, the control unit 210 controls the image capturing device 100 to return to the shooting standby state.

[0044] Next, the playback mode will be described. In the playback mode, the control unit 210 transmits a control signal to each component of the image capture device 100 to transition to a playback state. For example, the control unit 210 controls the encoding processing unit 207, the recording control unit 208, the display control unit 206, and the audio output unit 212 to perform the following operations.

[0045] The recording control unit 208 reads the audio-accompanied moving image data recorded on the recording medium 209 and transmits the read audio-accompanied moving image data to the encoding processing unit 207. The encoding processing unit 207 decodes the audio-accompanied moving image data into image data and audio data. The encoding processing unit 207 transmits the decoded moving image data to the display control unit 206 and the decoded audio data to the audio output unit 212. The display control unit 206 displays the decoded image data on the display unit 102. The audio output unit 212 plays the decoded audio data through a speaker 213.

[0046] As described above, the imaging device 100 can record and play back image data and audio data.

[0047] In this embodiment, the audio input unit 204 performs audio processing such as adjusting the level of an audio signal input from a microphone. In this embodiment, the audio input unit 204 performs this audio processing in response to the start of moving image recording. Note that this audio processing may be performed after the power of the imaging device 100 is turned on. Also, this audio processing may be performed in response to the selection of a shooting mode. Also, this audio processing may be performed in response to the selection of a mode related to audio recording, such as a moving image recording mode or an audio memo function. Also, this audio processing may be performed in response to the start of recording of an audio signal.

[0048] FIG. 3 is a block diagram showing the configuration of the voice input unit 204 and the external microphone unit 215 in FIG.

[0049] 3, the external microphone unit 215 includes an external L microphone 301a, an external R microphone 301b, an A / D conversion unit 302, and a transfer function storage unit 303. The external L microphone 301a and the external R microphone 301b are each an example of a first microphone. In this embodiment, the image capture device 100 collects environmental sounds using the external L microphone 301a and the external R microphone 301b and records the audio signals output from the external L microphone 301a and the external R microphone 301b in stereo. For example, the environmental sounds are sounds generated outside the housing of the external microphone unit 215, the housing of the image capture device 100, and the housing of the optical lens unit 200, such as a user's voice, animal cries, the sound of rain, and music. The external L microphone 301a and the external R microphone 301b generate analog audio signals from the acquired sounds and output the analog audio signals to the A / D conversion unit 302. 3, the audio signal output from the external L microphone 301a is represented as Lch, and the audio signal output from the external R microphone 301b is represented as Rch. The A / D conversion unit 302 converts the analog audio signals output from the external L microphone 301a and the external R microphone 301b into digital audio signals. The A / D conversion unit 302 outputs these digital audio signals to the audio input unit 204 via the external input / output unit 214. The transfer function storage unit 303 stores an external microphone transfer function Te, which will be described later.

[0050] 3, audio input unit 204 includes built-in L microphone 120a, built-in R microphone 120b, noise microphone 304, A / D conversion unit 305, FFT (Fast Fourier Transform) unit 306, noise data generation unit 307, transfer function storage unit 308, subtraction processing unit 309, iFFT unit 310, and audio processing unit 311. In this embodiment, when external microphone unit 215 is not connected to external input / output unit 214, imaging device 100 picks up environmental sounds using built-in L microphone 120a and built-in R microphone 120b.

[0051] The noise microphone 304 is an example of a second microphone. The noise microphone 304 is a microphone for acquiring noise, such as drive noise, from a predetermined noise source generated within the housing of the image capture device 100 and the housing of the optical lens unit 200. The noise source is, for example, a motor such as an ultrasonic motor (hereinafter referred to as "USM") or a stepper motor (hereinafter referred to as "STM"). The noise is, for example, vibration noise generated by driving a motor such as a USM or STM. For example, the motor is driven during AF processing to focus on a subject. The control unit 210 acquires noise, such as drive noise, generated within the housing of the image capture device 100 and the housing of the optical lens unit 200 using the noise microphone 304, and generates noise parameters (described later) using audio data of the acquired noise. The location of the noise microphone 304 will be described later with reference to FIG. 4.

[0052] Noise microphone 304 generates an analog audio signal from the acquired audio and outputs the analog audio signal to A / D conversion unit 305. In FIG. 3, the audio signal output from noise microphone 304 is represented as Nch. A / D conversion unit 305 converts the analog audio signal output from noise microphone 304 into a digital audio signal. A / D conversion unit 305 outputs this digital audio signal to FFT unit 306. In this embodiment, A / D conversion units 302 and 305 convert the analog audio signal into a digital audio signal by performing sampling processing with a sampling frequency of 48 kHz and a bit depth of 16 bits.

[0053] The FFT unit 306 performs fast Fourier transform processing on the time-domain digital audio signals output from the A / D conversion unit 305 and the external microphone unit 215 to convert them into frequency-domain digital audio signals. In this embodiment, the frequency-domain digital audio signals have a frequency spectrum of 1024 points in a frequency band from 0 [Hz] to 48 [kHz]. Furthermore, the frequency-domain digital audio signals have a frequency spectrum of 513 points in a frequency band from 0 [Hz] to 24 [kHz], which is the Nyquist frequency. In this embodiment, the imaging device 100 performs noise reduction processing using the 513-point frequency spectrum from 0 [Hz] to 24 [kHz] of the audio data output from the FFT unit 306.

[0054] Here, the frequency spectrum of the Lch subjected to the fast Fourier transform is represented by 513-point array data of Lch_Before(0) to Lch_Before(512). Hereinafter, these array data will be collectively referred to as "Lch_Before". Also, the frequency spectrum of the Rch subjected to the fast Fourier transform is represented by 513-point array data of Rch_Before(0) to Rch_Before(512). Hereinafter, these array data will be collectively referred to as "Rch_Before". Also, the frequency spectrum of the Nch subjected to the fast Fourier transform is represented by 513-point array data of Nch_Before(0) to Nch_Before(512). Hereinafter, these array data will be collectively referred to as "Nch_Before".

[0055] The noise data generation unit 307 generates noise data for reducing noise contained in Lch_Before and Rch_Before based on Nch_Before. In this embodiment, the noise data generation unit 307 generates array data NL(0) to NL(512), which is noise data for reducing noise contained in Lch_Before(0) to Lch_Before(512), respectively. The noise data generation unit 307 also generates array data NR(0) to NR(512), which is noise data for reducing noise contained in Rch_Before(0) to Rch_Before(512), respectively. The frequency points in the array data NL(0) to NL(512) are the same as the frequency points in the array data Lch_Before(0) to Lch_Before(512). Furthermore, the frequency points in the sequence data of NR(0) to NR(512) are the same as the frequency points in the sequence data of Rch_Before(0) to Rch_Before(512). Note that, hereinafter, the sequence data of NL(0) to NL(512) will be collectively referred to as "NL", and NR(0) to NR(512) will be collectively referred to as "NR".

[0056] The transfer function storage unit 308 stores transfer functions used by the noise data generation unit 307 in calculations to generate NL and NR from Nch_Before. In this embodiment, the transfer function storage unit 308 stores a body part transfer function Tnb and a body part transfer function Tab. The body part transfer function Tnb is a transfer function from the lens mount 101 to the noise microphone 304. The body part transfer function Tab is a transfer function from the lens mount 101 to the accessory shoe 121. Hereinafter, transfer functions for generating NL from Nch_Before will be collectively referred to as "TLx", and transfer functions for generating NR from Nch_Before will be collectively referred to as "TRx".

[0057] TLx has the same number of sequences as NL, and TRx has the same number of sequences as NR. For example, TL1 is sequence data of TL1(0) to TL1(512), and the frequency points of TL1 are the same as the frequency points of Lch_Before. Also, TR1 is sequence data of TR1(0) to TR1(512), and the frequency points of TR1 are the same as the frequency points of Rch_Before.

[0058] In this embodiment, the transfer function storage unit 308 stores all of the coefficients for each of the 513 frequency spectrum points as a transfer function. However, it is sufficient that the coefficients for at least the frequency points necessary for noise reduction are stored, rather than the coefficients for all 513 frequency points. For example, the transfer function storage unit 308 may store coefficients for each frequency spectrum from 20 Hz to 20 kHz, which are considered to be typical audible frequencies, but may not store coefficients for other frequency spectrums. Furthermore, for example, coefficients for frequency spectrums whose coefficient value is zero may not be stored in the transfer function storage unit 308.

[0059] The subtraction processing unit 309 subtracts NL from Lch_Before and subtracts NR from Rch_Before. For example, the subtraction processing unit 309 includes an L subtractor 309a and an R subtractor 309b. The L subtractor 309a subtracts NL from Lch_Before and outputs array data of 513 points from Lch_After(0) to Lch_After(512). The R subtractor 309b subtracts NR from Rch_Before and outputs array data of 513 points from Rch_After(0) to Rch_After(512). In this embodiment, the subtraction processing unit 309 performs subtraction processing using a spectral subtraction method.

[0060] The iFFT unit 310 performs an inverse fast Fourier transform (inverse Fourier transform) on the frequency-domain digital audio signal output from the subtraction processing unit 309 to convert it into a time-domain digital audio signal. The audio processing unit 311 performs audio processing on the time-domain digital audio signal, such as an equalizer, an auto-level controller, and stereo enhancement processing. The audio processing unit 311 outputs the processed audio data to the volatile memory 205.

[0061] In the present embodiment, a configuration in which the first microphones are external L microphone 301a and external R microphone 301b will be described, but the present invention is not limited to this configuration, and the first microphone may be only one microphone or three or more microphones. For example, when external microphone unit 215 has one microphone as the first microphone, audio data picked up by the one microphone is recorded in monaural format. When external microphone unit 215 has three or more microphones as the first microphone, audio data picked up by the three or more microphones is recorded in surround format.

[0062] Fig. 4 is a diagram for explaining the locations of the built-in L microphone 120a, the built-in R microphone 120b, and the noise microphone 304 in Fig. 2. Fig. 4 shows a cross-sectional view of the upper part of the image capture device 100 (area 400 indicated by the dotted line in Fig. 4). The upper part of the image capture device 100 is composed of an exterior part 401, a microphone bushing 402, and a fixing part 403.

[0063] The exterior unit 401 has holes (hereinafter referred to as "microphone holes") for inputting environmental sounds to the built-in L microphone 120a and the built-in R microphone 120b. In this embodiment, these microphone holes are formed above the built-in L microphone 120a and the built-in R microphone 120b. On the other hand, the noise microphone 304 is provided to acquire drive sounds generated within the housing of the image capture device 100 and the housing of the optical lens unit 200, and is not required to acquire environmental sounds. Therefore, in this embodiment, no microphone hole is formed above the noise microphone 304 in the exterior unit 401. Note that, although the shape of the microphone hole formed in the exterior unit 401 in FIG. 4 is circular, this is not limited thereto and may be other shapes such as elliptical or rectangular. Furthermore, the microphone hole on the built-in L microphone 120a may have a different shape from the microphone hole on the built-in R microphone 120b.

[0064] Microphone bushing 402 is a member for fixing built-in L microphone 120a, built-in R microphone 120b, and noise microphone 304. Fixing section 403 is a member for fixing microphone bushing 402 to exterior section 401. In this embodiment, exterior section 401 and fixing section 403 are made of a molded material such as PC (polycarbonate). Note that exterior section 401 and fixing section 403 may also be made of a metal material such as aluminum or stainless steel. Also, in this embodiment, microphone bushing 402 is made of a rubber material such as ethylene propylene diene rubber.

[0065] Incidentally, an optical lens unit 200 can be connected to the lens mount 101 of the imaging device 100, for example, as shown in FIG. 5(a). When the optical lens unit 200 is connected to the lens mount 101, an optical image of a subject captured by the optical lens unit 200 is formed on an image sensor of the imaging device 100. The control unit 210 communicates with the optical lens unit 200 connected to the lens mount 101 (hereinafter referred to as "lens communication"). The control unit 210 receives information about the optical lens unit 200 from the optical lens unit 200 through lens communication, and stores the received information about the optical lens unit 200 in the volatile memory 205. By having the imaging device 100 obtain the information about the optical lens unit 200, the control unit 210 can optimally control the optical lens unit 200 connected to the lens mount 101 among a wide variety of lens units available.

[0066] 5B, for example. When the external microphone unit 215 is connected to the accessory shoe 121 of the imaging device 100, an external microphone unit 215 can be connected. When the external microphone unit 215 is connected to the accessory shoe 121, an audio signal generated by the external microphone unit 215 is transmitted to the audio input unit 204. The control unit 210 communicates with the external microphone unit 215 connected to the accessory shoe 121 (hereinafter referred to as "accessory communication"). The control unit 210 receives information about the external microphone unit 215 from the external microphone unit 215 through the accessory communication, and stores the received information about the external microphone unit 215 in the volatile memory 205 of the imaging device 100. When the imaging device 100 obtains the information about the external microphone unit 215, the control unit 210 can optimally control the external microphone unit 215 connected to the accessory shoe 121 among the wide variety of external microphone units available.

[0067] In this embodiment, a transfer function Ter (first transfer parameter) described later relating to the transfer of noise from the noise source to each microphone of external microphone unit 215 is calculated based on the transfer functions of the components arranged on the noise propagation path from the noise source to each microphone of external microphone unit 215. Furthermore, a transfer function Tnr (second transfer parameter) described later relating to the transfer of noise from the noise source to noise microphone 304 is calculated based on the transfer functions of the components arranged on the noise propagation path from the noise source to noise microphone 304. NL and NR are generated based on transfer functions Ter and Tnr, and NL is subtracted from Lch_Before and NR is subtracted from Rch_Before. This makes it possible to effectively reduce noise contained in picked-up audio even when calibration cannot be performed in an ideal quiet environment.

[0068] Fig. 6 is a diagram showing an operation flow of the imaging device 100 of Fig. 1. In the present embodiment, as an example, a case will be described in which an AF motor (not shown) in the optical lens unit 200 is a noise source. The operation flow of the imaging device 100 of Fig. 6 is realized by the control unit 210 executing a program stored in the nonvolatile memory 217 or the like.

[0069] 6, in step S601, the control unit 210 determines whether or not a lens unit is connected to the lens mount 101. If the control unit 210 determines that, for example, the optical lens unit 200 is connected to the lens mount 101 (YES in step S601), the control unit 210 performs lens communication with the optical lens unit 200 via the lens control unit 202. The control unit 210 can determine the type of optical lens unit 200 connected to the lens mount 101 based on information about the optical lens unit 200 received from the optical lens unit 200 through lens communication. Next, the process of step S602 is executed.

[0070] In step S602, the control unit 210 acquires the lens unit transfer function Tr from the optical lens unit 200 via lens communication. Next, the process of step S603 is executed. The lens unit transfer function Tr is the transfer function of the optical lens unit 200, and is the transfer function from an AF motor (not shown) inside the optical lens unit 200 to a portion that connects to the lens mount 101 of the imaging device 100. The control unit 210 stores the received lens unit transfer function Tr in the transfer function storage unit 308 in the audio input unit 204. As described above, in this embodiment, each time a lens unit is connected to the lens mount 101, the lens unit transfer function Tr of the connected lens unit is stored in the transfer function storage unit 308. However, since the storage capacity of the transfer function storage unit 308 is limited, when the remaining storage capacity of the transfer function storage unit 308 falls below a predetermined value, the lens unit transfer function Tr stored oldest in the transfer function storage unit 308 is deleted. Alternatively, the lens unit transfer function Tr of the lens unit that has been connected to the lens mount 101 the least number of times is deleted. In the present embodiment, the lens transfer function Tr is stored in a storage unit (not shown) of the optical lens unit 200, but this is not limiting. For example, the lens transfer function Tr may be stored in advance in the transfer function storage unit 308 in the voice input unit 204.

[0071] Here, the transfer function in this embodiment is a function that represents a frequency response that indicates how vibration is transmitted. FIG. 7(a) is an example of a lens transfer function Tr. In FIG. 7(a), the vertical axis represents vibration transmissibility, and the horizontal axis represents frequency. When the vibration transmissibility is 1 at a certain frequency, the vibration amplitude of the vibration source and the vibration amplitude of the vibration destination are the same. When the vibration transmissibility is 1 or greater, the vibration amplitude of the vibration destination is greater than the vibration amplitude of the vibration source. For example, when the vibration transmissibility is 2, the vibration amplitude of the vibration destination is twice the vibration amplitude of the vibration source. In other words, when the vibration transmissibility is 2, the vibration is easily transmitted. When the vibration transmissibility is less than 1, the vibration amplitude of the vibration destination is smaller than the vibration amplitude of the vibration source. For example, when the vibration transmissibility is 0.5, the vibration amplitude of the vibration destination is 0.5 times the vibration amplitude of the vibration source. In other words, when the vibration transmissibility is 0.5, the vibration is not easily transmitted. In this embodiment, a function that represents how vibrations are transmitted is used as the transfer function, but the transfer function is not limited to this, and may be a function that represents how sound is transmitted. Also, the transfer function may be a function that uses other parameters.

[0072] In step S603, control unit 210 determines whether or not an external microphone unit is connected to accessory shoe 121. If control unit 210 determines that, for example, external microphone unit 215 is connected to accessory shoe 121 (YES in step S603), control unit 210 performs accessory communication with external microphone unit 215 via external input / output unit 214. Control unit 210 can determine the type of external microphone unit 215 connected to accessory shoe 121 based on information about external microphone unit 215 received from external microphone unit 215 through accessory communication. Next, the process of step S604 is executed.

[0073] In step S604, the control unit 210 acquires the external microphone unit transfer function Te from the external microphone unit 215 via accessory communication. The control unit 210 stores the acquired external microphone unit transfer function Te in the transfer function storage unit 308 in the audio input unit 204. Next, the process of step S605 is executed. The external microphone unit transfer function Te is the transfer function of the external microphone unit 215. For example, if the connected external microphone is a stereo microphone, the external microphone unit transfer function Te includes a transfer function Te(l) from the external left microphone 301a of the external microphone unit 215 to the portion connecting to the accessory shoe 121, and a transfer function Te(r) from the external right microphone 301b to the portion connecting to the accessory shoe 121. Note that if the transfer functions of the external left microphone 301a and the external right microphone 301b are considered to be substantially the same, such as when the external left microphone 301a and the external right microphone 301b are installed on the same component and are physically close to each other, the same transfer function may be used for the external left microphone 301a and the external right microphone 301b. In this embodiment, for ease of explanation, it is assumed that the two transfer functions can be considered to be substantially the same, and that Te=Te(l)=Te(r).

[0074] Furthermore, in the present embodiment, a case will be described in which external microphone transfer function Te is stored in transfer function storage unit 303 of external microphone unit 215, but this is not limiting. For example, external microphone transfer function Te may be stored in advance in transfer function storage unit 308 in audio input unit 204. Furthermore, as described above, since transfer function storage unit 308 has a limited storage capacity, when the remaining storage capacity of transfer function storage unit 308 falls below a predetermined value, the external microphone transfer function Te stored oldest in transfer function storage unit 308 is deleted. Alternatively, the external microphone transfer function Te of the external microphone unit that has been connected to accessory shoe 121 the fewest times is deleted.

[0075] In step S605, the control unit 210 reads out the body part transfer function Tnb stored in the transfer function storage unit 308. As described above, the body part transfer function Tnb is the transfer function from the lens mount 101 to the noise microphone 304. Next, in step S606, the control unit 210 reads out the body part transfer function Tab stored in the transfer function storage unit 308. As described above, the body part transfer function Tab is the transfer function from the lens mount 101 to the accessory shoe 121. Next, in step S607, the control unit 210 calculates the correction coefficient C when the optical lens unit 200 and the external microphone unit 215 are connected to the image capture device 100 (for example, the configuration shown in FIG. 5B). This correction coefficient C is a coefficient used to calculate the driving sound of the AF motor recorded in the external L microphone 301a and the external R microphone 301b, respectively, from the driving sound of the AF motor recorded in the noise microphone 304 mounted on the image capture device 100.

[0076] The AF motor drive sounds recorded by the external left microphone 301a and the external right microphone 301b are classified into the following two types. The first type is sound generated by vibrations generated by driving the AF motor, which propagate through the air (air propagation) and are directly recorded as drive sounds by the external left microphone 301a and the external right microphone 301b. The second type is sound generated by the vibrations propagating through the housing (vibration propagation), vibrating the diaphragms of the external left microphone 301a and the external right microphone 301b, which are converted into sound and recorded. It is known that the sound recorded by the external left microphone 301a and the external right microphone 301b due to the drive of the AF motor is predominantly due to the vibration propagation. Therefore, by reducing the AF motor drive sounds recorded due to vibration propagation, it is possible to reduce most of the AF motor drive sounds recorded by the external left microphone 301a and the external right microphone 301b.

[0077] The driving sounds of various motors, not just the AF motor, are often recorded as noise in video audio. Products reduce the noise caused by the driving sounds of various motors to a volume level that is not unpleasant to the user, but traditionally only the sound that reaches the user's ears through air transmission has been reduced. The vibrations caused by the driving of various motors that travel through the housing to the microphone have not been reduced, and these vibrations cause the microphone diaphragm to vibrate, resulting in noise being recorded as audio.

[0078] As such, vibration propagation is often the dominant cause of the drive sounds of various motors, not just the AF motor, being recorded as noise. Therefore, by applying the present invention, which focuses on reducing noise caused by vibration propagation, it is expected that noise will be reduced in the same way as the drive sound of the AF motor. In other words, by using a transfer function, noise caused by vibration propagation can be effectively reduced.

[0079] 7 and 8, the calculation of the correction coefficient C in step S607 will be described. Note that, as an example, the calculation of the correction coefficient C for the external L microphone 301a out of the external L microphone 301a and the external R microphone 301b will be described below.

[0080] FIG. 7 is a diagram showing the relationship between the lens unit transfer function Tr, the body unit transfer function Tnb, and the transfer function Tnr. In each graph in FIG. 7, the horizontal axis represents frequency from point 0 to point 512, and the vertical axis represents vibration transmissibility. FIG. 7(a) is an example of the lens unit transfer function Tr acquired in step S602. As described above, the lens unit transfer function Tr is a transfer function from an AF motor (not shown) inside the optical lens unit 200 to a portion connecting to the lens mount 101 of the imaging device 100. FIG. 7(b) is an example of the body unit transfer function Tnb acquired in step S605. As described above, the body unit transfer function Tnb is a transfer function from the lens mount 101 to the noise microphone 304. FIG. 7(c) is an example of the transfer function Tnr. The transfer function Tnr is a transfer function from an AF motor (noise source) (not shown) inside the optical lens unit 200 to the noise microphone 304. The transfer function Tnr is a coefficient of each frequency calculated by multiplying the vibration transmissibility of each frequency of the lens transfer function Tr by the vibration transmissibility of each frequency of the body transfer function Tnb.

[0081] That is, the transfer function Tnr is Tnr=Tr×Tnb In this manner, in this embodiment, the transfer function Tnr from the noise source to the noise microphone 304 is calculated based on the transfer function of each component placed on the noise propagation path from the noise source to the noise microphone 304.

[0082] FIG. 8 is a diagram showing the relationship between the transfer function Ter, the transfer function Tnr, and the correction coefficient C. In each graph in FIG. 8, the horizontal axis represents the frequency from point 0 to point 512, and the vertical axis represents the vibration transmissibility. FIG. 8(a) is an example of the transfer function Ter. The transfer function Ter is a transfer function from an AF motor (noise source) (not shown) inside the optical lens unit 200 to the external L microphone 301a. The transfer function Ter is a coefficient for each frequency calculated by multiplying the vibration transmissibility of each frequency of the lens unit transfer function Tr acquired in step S602, the vibration transmissibility of each frequency of the body unit transfer function Tab acquired in step S606, and the vibration transmissibility of each frequency of the external microphone unit transfer function Te acquired in step S604.

[0083] That is, the transfer function Ter is Ter = Tr × Tab × Te In this manner, in this embodiment, the transfer function Ter from the noise source to the external L microphone 301a is calculated based on the transfer functions of the components arranged on the noise propagation path from the noise source to the external L microphone 301a.

[0084] Fig. 8(b) shows the transfer function Tnr of Fig. 7(c). Fig. 8(c) shows an example of the correction coefficient C. The correction coefficient C is a coefficient for each frequency calculated by dividing the vibration transmissibility of each frequency of the transfer function Ter by the vibration transmissibility of each frequency of the transfer function Tnr.

[0085] That is, the correction coefficient C is C=Ter / Tnr In this manner, in this embodiment, the correction coefficient C for the external L microphone 301a is calculated using the transfer functions of the components acquired in steps S602, and S604 to S606. The correction coefficient C for the external R microphone 301b is also calculated in a similar manner. Once these correction coefficients C are calculated, the process of step S611, which will be described later, is executed.

[0086] On the other hand, if the control unit 210 determines in step S603 that an external microphone unit is not connected to the accessory shoe 121 (NO in step S603), the process proceeds to step S608. In step S608, the control unit 210 reads out the body part transfer function Tnb stored in the transfer function storage unit 308. Next, in step S609, the control unit 210 reads out the body part transfer function Tib. The body part transfer function Tib includes the transfer function from the lens mount 101 to the built-in L microphone 120a and the transfer function from the lens mount 101 to the built-in R microphone 120b.

[0087] Next, in step S610, control unit 210 calculates correction coefficient C for the case where an external microphone unit is not connected to image capture device 100 but optical lens unit 200 is connected (for example, the configuration of FIG. 5(a)). This correction coefficient C is a coefficient for calculating the AF motor drive sound recorded by built-in L microphone 120a and built-in R microphone 120b from the AF motor drive sound recorded by noise microphone 304. It is known that the AF motor drive sound recorded by built-in L microphone 120a and built-in R microphone 120b, respectively, is predominantly caused by vibration propagation, similar to the AF motor drive sound recorded by external L microphone 301a and external R microphone 301b described above. Therefore, by reducing the AF motor noise recorded due to vibration propagation, it is possible to reduce most of the AF motor drive sound recorded by built-in L microphone 120a and built-in R microphone 120b, respectively.

[0088] Here, the calculation of the correction coefficient C in step S610 will be described. Note that, as an example, the calculation of the correction coefficient C for the built-in L microphone 120a out of the built-in L microphone 120a and the built-in R microphone 120b will be described below. In calculating the correction coefficient C in step S610, first, a transfer function Tir is calculated using the lens unit transfer function Tr acquired in step S602 and the body unit transfer function Tib acquired in step S609. The transfer function Tir is a transfer function from an AF motor (not shown) inside the optical lens unit 200 to the built-in L microphone 120a.

[0089] The transfer function Tir is Tir=Tr×Tib is calculated as follows.

[0090] The correction coefficient C is a coefficient for each frequency calculated by dividing the vibration transmissibility of each frequency of the transfer function Tir by the vibration transmissibility of each frequency of the transfer function Tnr from the AF motor (not shown) inside the optical lens unit 200 to the noise microphone 304.

[0091] That is, the correction coefficient C is C=Tir / Tnr In this manner, in this embodiment, the correction coefficient C for the built-in L microphone 120a is calculated using the transfer functions of the components acquired in steps S602, S608, and S609. The correction coefficient C for the built-in R microphone 120b is also calculated in a similar manner. Once these correction coefficients C are calculated, the process of step S611 is executed.

[0092] In step S611, control unit 210 determines whether or not an instruction to start recording has been given. If control unit 210 determines that an instruction to start recording has been given (YES in step S611), the process proceeds to step S612. In step S612, control unit 210 calculates the amount of noise. Specifically, control unit 210 causes noise data generation unit 307 to calculate noise data.

[0093] FIG. 9 is a diagram illustrating noise reduction in this embodiment. Here, as an example, noise reduction using the correction coefficient C calculated in step S607, that is, the correction coefficient C when the optical lens unit 200 and the external microphone unit 215 are connected to the image capture device 100 (for example, the configuration of FIG. 5B), will be described. Also, as an example, noise reduction using the correction coefficient C for the external L microphone 301a out of the external L microphone 301a and the external R microphone 301b will be described. In each graph in FIG. 9, the horizontal axis represents the frequency from point 0 to point 512, and the vertical axis represents the amplitude of the frequency spectrum. FIG. 9A is an example of a frequency spectrum of Lch_Before when the driving sound of the AF motor is generated within the housing of the optical lens unit 200. FIG. 9B is an example of a frequency spectrum of Nch_Before when the driving sound of the AF motor is generated within the housing of the optical lens unit 200. FIG. 9C is an example of NL when the driving sound of the AF motor is generated within the housing of the optical lens unit 200. The noise data generation unit 307 generates NL by multiplying each frequency spectrum of Nch_Before by the value of each frequency of the correction coefficient C. As described above, this correction coefficient C is a coefficient for calculating the driving sound of the AF motor recorded in the external L microphone 301a from the driving sound of the AF motor recorded in the noise microphone 304 mounted on the imaging device 100.

[0094] That is, each frequency spectrum NL[N] of the driving sound of the AF motor recorded by the external L microphone 301a is expressed as follows: NL[N]=Nch_Before[N]×C[N] is calculated as follows.

[0095] where: Nch_Before[N]: Frequency spectrum of the noise microphone 304 C[N]: Correction coefficient value for each frequency is.

[0096] After calculating the NL, the control unit 210 performs noise subtraction processing in step S613.

[0097] 9(d) is an example of Lch_After when drive sound is generated inside the housing of the optical lens unit 200. The control unit 210 subtracts NL from Lch_Before using the subtraction processing unit 309 to generate the frequency spectrum Lch_After of the external L microphone 301a after the subtraction processing. In this manner, in this embodiment, noise in the audio recorded by the external L microphone 301a is reduced. Similarly, noise in the audio recorded by the external R microphone 301b is also reduced.

[0098] When the noise subtraction process is completed, the control unit 210 records the audio data in the nonvolatile memory 217 etc. in step S614, after which the process ends.

[0099] According to the above-described embodiment, a transfer function Ter related to the transmission of noise from the noise source to each microphone of the external microphone unit 215 is calculated based on the transfer functions of components arranged on the noise propagation path from the noise source to each microphone of the external microphone unit 215. Furthermore, a transfer function Tnr related to the transmission of noise from the noise source to the noise microphone 304 is calculated based on the transfer functions of components arranged on the noise propagation path from the noise source to the noise microphone 304. NL and NR are generated based on the transfer functions Ter and Tnr, and NL is subtracted from Lch_Before and NR is subtracted from Rch_Before. In other words, noise data for reducing noise is generated without performing calibration in an ideal quiet environment such as an anechoic chamber. This enables effective noise reduction in a configuration in which a detachable external microphone unit 215 is connected.

[0100] In the above-described embodiment, the transfer function is a parameter that indicates how easily vibrations are transmitted, thereby making it possible to effectively reduce noise caused by vibration propagation.

[0101] Furthermore, in the above-described embodiment, the transfer function is a parameter that indicates how easily sound propagates, thereby making it possible to effectively reduce noise caused by air propagation.

[0102] In the above-described embodiment, the transfer functions of a plurality of components are stored in the transfer function storage unit 308. This makes it possible to easily obtain the transfer function of each component for calculating the transfer function Ter and the transfer function Tnr.

[0103] Furthermore, in the above-described embodiment, when the remaining storage capacity of the transfer function storage unit 308 falls below a predetermined value, the transfer function of the part that was stored the oldest in the transfer function storage unit 308 is deleted. This makes it possible to avoid a situation in which the transfer functions of the parts connected to the image capture device 100 cannot be stored in the transfer function storage unit 308 and noise cannot be reduced due to unused transfer functions being held in the transfer function storage unit 308.

[0104] In the above-described embodiment, when the remaining storage capacity of the transfer function storage unit 308 falls below a predetermined value, the transfer function of the component that has been least frequently connected to the image capture device 100 is deleted. This makes it possible to avoid a situation in which the transfer function of the component connected to the image capture device 100 cannot be stored in the transfer function storage unit 308 and noise cannot be reduced because a transfer function that is used less frequently is held in the transfer function storage unit 308.

[0105] In the above-described embodiment, the first microphone is composed of the external L microphone 301a and the external R microphone 301b. This makes it possible to effectively reduce noise in a configuration in which the external microphone unit 215, which includes multiple microphones for capturing environmental sounds, is connected.

[0106] In the above-described embodiment, the noise microphone 304 is a microphone built into the image capture device 100. As a result, in a configuration in which the external microphone unit 215 for acquiring environmental sounds is located relatively far from the noise microphone 304 for acquiring sounds generated by noise sources, it is possible to effectively reduce noise without performing calibration in an ideal quiet environment such as an anechoic chamber.

[0107] In the above-described embodiment, the noise source is a member that is driven during imaging, which makes it possible to effectively reduce the driving noise of the member that is driven during imaging.

[0108] Although the present invention has been described using the above-mentioned embodiment, the present invention is not limited to the above-mentioned embodiment. For example, the transfer function to be used may be determined based on measured temperature information.

[0109] Fig. 10 is an external view of a modified example of the imaging device 100 of Fig. 1. The imaging device 1000, which is a modified example of the imaging device 100, includes an internal temperature sensor 1001 and an outside air temperature sensor 1002 (temperature measurement means) in addition to the components included in the imaging device 100.

[0110] The internal temperature sensor 1001 is a temperature sensor for measuring the temperature inside the image capture device 1000, and is disposed on an electronic board (not shown) inside the image capture device 1000. The internal temperature sensor 1001 measures the temperature of a device that operates at high speed and generates a lot of heat, such as a CPU (not shown) on the electronic board, and monitors the device to control it so that it does not exceed its guaranteed operating temperature. For example, if the guaranteed operating temperature of the device is 85°C, the control unit 210 stops the operation of the image capture device 1000 just before the internal temperature sensor 1001 reaches 85°C (for example, at 80°C), thereby controlling the device so that it does not exceed its guaranteed operating temperature.

[0111] The outside air temperature sensor 1002 is a temperature sensor for measuring the environmental temperature outside the imaging device 1000, and is disposed inside the imaging device 1000 near the grip of the imaging device 1000. The outside air temperature sensor 1002 is disposed in a location away from the electronic board and where a temperature rise due to the operation of each component of the imaging device 1000 is unlikely to occur. Even if the imaging device 1000 operates at high speed for a long period of time and the temperature of the electronic board rises, the temperature near the grip of the imaging device 1000 does not rise much, and the temperature measured by the outside air temperature sensor 1002 indicates a temperature close to the outside air temperature, which is the environmental temperature outside the imaging device 1000.

[0112] Meanwhile, the transfer functions of the imaging device 1000, the optical lens unit 200, and the external microphone unit 215 change with temperature. As an example, the transfer function Tib from the lens mount 101 to the built-in L microphone 120a will be described. As shown in FIG. 4, the built-in L microphone 120a is typically held in place by a microphone bushing 402, which is made of a rubber material such as ethylene propylene diene rubber. Depending on the rubber material, some materials have a modulus of rigidity that increases by approximately 500% at a low temperature of -20°C compared to a room temperature of 25°C. This causes the transfer coefficient of the rubber material to shift to the positive side (the direction in which vibrations are more easily transmitted). Therefore, when the temperature drops from room temperature, the transfer function Tib to the built-in L microphone 120a also changes, which affects the noise reduction effect achieved by the noise subtraction processing of the present invention.

[0113] In this way, in noise subtraction processing when the path of vibration propagation from the noise source to the microphone includes parts whose hardness, etc. changes with temperature and affects the transfer function, it is preferable to use a transfer function that corresponds to the temperature change for each part.

[0114] In contrast, in this embodiment, multiple transfer functions that differ for each temperature for each component are recorded in transfer function storage unit 308. Control unit 210 determines which transfer function to use from the multiple transfer functions based on the temperature measured by internal temperature sensor 1001 and outside air temperature sensor 1002. This makes it possible to perform appropriate noise subtraction processing in accordance with temperature changes, even if a component whose hardness, etc., changes with temperature and thus affects the transfer function is located on the noise propagation path.

[0115] Furthermore, even if the temperature of the electronic board rises, the optical lens unit 200 and the external microphone unit 215, which are located at a distance, are less susceptible to the effect of the temperature rise of the electronic board. For this reason, their temperatures become approximately the same as the outside air temperature, and in this embodiment, transfer functions corresponding to the temperature indicated by the outside air temperature sensor 1002 are selected as the transfer functions for the optical lens unit 200 and the external microphone unit 215. On the other hand, a transfer function corresponding to the temperature of the internal temperature sensor 1001 is selected as the transfer function for the inside of the imaging device 100, which is affected when the temperature of the electronic board rises.

[0116] The configuration shown in FIG. 10 is just an example, and a temperature sensor may be provided in each of the optical lens unit 200 and the external microphone unit 215. Alternatively, multiple temperature sensors may be provided within the imaging device 100, and the transfer function to be used may be determined by referring to the temperatures of the multiple temperature sensors.

[0117] In the above-described embodiment, the present invention has been described as being applied to a digital camera as an example of an audio processing device, but the present invention is not limited to digital cameras and may also be applied to devices capable of recording video with audio, such as smartphones and tablet terminals.

[0118] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a recording medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0119] The present invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0120] 100 Imaging device 200 Optical lens section 201 Imaging unit 210 Control Unit 215 External microphone section 301a External L Microphone 301b External R microphone 304 Noise Microphone 306 FFT section 307 Noise Data Generator 308 Transfer function memory section 309 Subtraction processing unit 1000 Imaging Device 1001 Internal temperature sensor 1002 Outside air temperature sensor

Claims

1. a connection means for connecting a first microphone for acquiring environmental sounds; a second microphone for capturing a sound generated by the noise source; a first conversion means for Fourier transforming a digital audio signal based on the audio captured by the first microphone connected by the connection means to generate a first audio signal; a second conversion means for Fourier transforming a digital audio signal based on the audio captured by the second microphone to generate a second audio signal; a first calculation means for calculating a first transfer parameter related to the transfer of noise from the noise source to the first microphone based on transfer parameters of components arranged on a noise propagation path from the noise source to the first microphone; a second calculation means for calculating a second transfer parameter related to the transfer of noise from the noise source to the second microphone based on transfer parameters of components arranged on a noise propagation path from the noise source to the second microphone; a noise data generating means for generating noise data obtained by correcting the second audio signal based on the first transfer parameter and the second transfer parameter; a subtraction means for subtracting the noise data from the first audio signal, the first calculation means calculates the first transfer parameter by multiplying transfer parameters of each component arranged on a noise propagation path from the noise source to the first microphone; The audio processing device, characterized in that the second calculation means calculates the second transfer parameter by multiplying the transfer parameters of each component arranged on a noise propagation path from the noise source to the second microphone.

2. 2. The audio processing device according to claim 1, wherein the transmission parameter of the component is a parameter related to transmission of vibration.

3. 2. The audio processing device according to claim 1, wherein the transmission parameters of the components are parameters related to the transmission of sound.

4. acquiring means for acquiring a transfer parameter relating to the first microphone from the first microphone; 4. The audio processing device according to claim 1, wherein the first calculation means calculates the first transfer parameter based on a transfer parameter related to the first microphone.

5. A lens mount to which a lens unit having a motor that is the noise source is attached, the first calculation means calculates the first transfer parameter by multiplying a transfer parameter of the lens unit attached to the lens mount, a body transfer parameter from the lens mount to the connection means, and a transfer parameter of the first microphone connected to the connection means; 5. The audio processing device according to claim 1, wherein the second calculation means calculates the second transfer parameter by multiplying a transfer parameter of the lens unit attached to the lens mount by a body transfer parameter from the lens mount to the second microphone.

6. A voice processing device, a connection means for connecting a first microphone for acquiring environmental sounds; a second microphone for capturing a sound generated by the noise source; a first conversion means for Fourier transforming a digital audio signal based on the audio picked up by the first microphone connected to the connection means to generate a first audio signal; a second conversion means for Fourier transforming a digital audio signal based on the audio captured by the second microphone to generate a second audio signal; a storage means detachably attached to the audio processing device for storing transmission parameters of a plurality of components including the first microphone; a noise data generating means for, when the first microphone is connected to the connecting means, calculating a first transfer parameter relating to the transfer of noise from the noise source to the first microphone using a transfer parameter of the first microphone stored in advance in the storage means, and generating noise data obtained by correcting the second audio signal based on the first transfer parameter and a second transfer parameter relating to the transfer of noise from the noise source to the second microphone; and a subtraction means for subtracting the noise data from the first audio signal.

7. A lens mount to which a lens unit including a motor that is the noise source is attached, the plurality of components includes the lens unit attached to the lens mount, The noise data generating means calculating the first transfer parameter by multiplying a transfer parameter of the lens unit attached to the lens mount, a body transfer parameter from the lens mount to the connection means, and a transfer parameter of the first microphone connected to the connection means; 7. The audio processing device according to claim 6, wherein the second transfer parameter is calculated by multiplying a transfer parameter of the lens unit attached to the lens mount by a body transfer parameter from the lens mount to the second microphone.

8. The audio processing device described in Claim 7, characterized in that the storage means pre-stores transmission parameters of the lens unit attached to the lens mount, body part transmission parameters from the lens mount to the connection means, and body part transmission parameters from the lens mount to the second microphone.

9. the first microphone is a stereo microphone, 9. The audio processing device according to claim 6, wherein the storage means stores transfer parameters of the R microphone and the L microphone that constitute the stereo microphone.

10. the second microphone is a microphone built into the audio processing device, The audio processing device according to claim 1 , wherein the exterior of the audio processing device does not have a hole for inputting the environmental sound into the second microphone.

11. An imaging means is provided, 11. The audio processing device according to claim 1, wherein the noise source is a member that is driven when the imaging means captures an image.

12. A third microphone, a third conversion means for Fourier transforming a digital audio signal based on the audio captured by the third microphone to generate a third audio signal; When the connection with the first microphone is cut off, the noise data generating means generates noise data by correcting the second audio signal based on the second transfer parameter; 12. The audio processing device according to claim 1, wherein the subtraction means subtracts the noise data obtained by correcting the second audio signal based on the second transfer parameter from the third audio signal.

13. An audio processing device according to claim 1, further comprising an inverse transform means for performing an inverse Fourier transform on the audio signal output from the subtraction means.

14. a connection means for connecting a first microphone for acquiring environmental sounds; a second microphone for acquiring a sound generated by a noise source, a first conversion step of Fourier transforming a digital audio signal based on the audio captured by the first microphone connected by the connection means to generate a first audio signal; a second conversion step of Fourier transforming a digital audio signal based on the audio captured by the second microphone to generate a second audio signal; a first calculation step of calculating a first transfer parameter related to the transfer of noise from the noise source to the first microphone based on transfer parameters of components arranged on a noise propagation path from the noise source to the first microphone; a second calculation step of calculating a second transfer parameter related to the transfer of noise from the noise source to the second microphone based on transfer parameters of components arranged on a noise propagation path from the noise source to the second microphone; a noise data generating step of generating noise data obtained by correcting the second audio signal based on the first transfer parameter and the second transfer parameter; a subtraction step of subtracting the noise data from the first audio signal, the first calculation step calculates the first transfer parameter by multiplying transfer parameters of each component arranged on a noise propagation path from the noise source to the first microphone; A control method for an audio processing device, characterized in that the second calculation step calculates the second transfer parameter by multiplying the transfer parameters of each component arranged on the noise propagation path from the noise source to the second microphone.

15. A connection means for connecting a first microphone for acquiring environmental sounds; a second microphone for capturing a sound generated by the noise source; a storage means detachably attached to the audio processing device for storing transmission parameters of a plurality of components including the first microphone, a first conversion step of Fourier transforming a digital audio signal based on the audio captured by the first microphone connected by the connection means to generate a first audio signal; a second conversion step of Fourier transforming a digital audio signal based on the audio captured by the second microphone to generate a second audio signal; a noise data generating step of calculating, when the first microphone is connected to the connection means, a first transfer parameter relating to the transfer of noise from the noise source to the first microphone using a transfer parameter of the first microphone stored in advance in the storage means, and generating noise data obtained by correcting the second audio signal based on the first transfer parameter and a second transfer parameter relating to the transfer of noise from the noise source to the second microphone; and a subtraction step of subtracting the noise data from the first audio signal.

16. A program for causing a computer to execute each means of the speech processing device according to any one of claims 1 to 13.

Images