Audio processing device, control method, and program

Abstract

To effectively reduce noise.SOLUTION: A sound processor comprises: a first microphone for acquiring environmental sounds; a second microphone for acquiring sounds from noise sources; means for performing Fourier transform on sound signals from the first microphone to generate first sound signals; means for performing Fourier transform on sound signals from the second microphone to generate second sound signals; generation means for generating noise data using the second sound signals and a first parameter related to noise of the noise sources; subtraction means for subtracting the noise data from the first sound signals; means for performing inverse Fourier transform on the sound signals from the subtraction means; and update means for updating the first parameter with the first sound signals and the second sound signals. When a first noise parameter is updated by the update means, the generation means generates noise data using the first noise parameter updated by the update means and the second sound signals.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to an audio processing device capable of reducing noise contained in audio data. [Background technology]

[0002] A digital camera, which is an example of an audio processing device, can record surrounding audio when recording video data. Digital cameras also have an autofocus function that drives an optical lens to focus on a subject while recording video data. Digital cameras also have a function to drive an optical lens to zoom while recording a video.

[0003] In this way, when an optical lens is driven while a video is being recorded, the driving sound of the optical lens may be included as noise in the audio recorded with the video. Conventionally, therefore, when a digital camera picks up a sliding sound generated when the optical lens is driven as noise, the noise can be reduced and the surrounding audio can be recorded. Patent Document 1 discloses a digital camera that reduces noise by a spectral subtraction method. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-205527 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, since the digital camera creates a noise pattern from noise collected by a microphone that records surrounding sounds, there is a possibility that an accurate noise pattern cannot be obtained from the sliding sound generated inside the housing of the optical lens. In this case, there is a risk that the digital camera cannot effectively reduce the noise contained in the collected sounds.

[0006] Therefore, an object of the present invention is to effectively reduce noise. [Means for solving the problem]

[0007] The audio processing device includes a first microphone for acquiring environmental sound, a second microphone for acquiring sound from a noise source, a first conversion means for Fourier transforming an audio signal from the first microphone to generate a first audio signal in a frequency domain, a second conversion means for Fourier transforming an audio signal from the second microphone to generate a second audio signal in the frequency domain, a generation means for calculating the second audio signal and a first parameter related to noise of the noise source to generate noise data, a subtraction means for subtracting the noise data from the first audio signal, and an inverse Fourier transform of the audio signal from the subtraction means. The noise data generating device has a third conversion means, and an update means for generating new parameters related to the noise of the noise source using the first audio signal and the second audio signal, and updating the first parameters using the newly generated parameters, wherein when the first parameters are updated by the update means, the generation means calculates the first parameters updated by the update means and the second audio signal to generate the noise data, and when the value of the newly generated parameters is not smaller than the value of the first parameters, the update means does not update the first parameters. Effect of the Invention

[0008] The audio processing device of the present invention can effectively reduce noise. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a perspective view of an imaging device according to a first embodiment. [Diagram 2] 1 is a block diagram showing a configuration of an imaging apparatus according to a first embodiment. [Diagram 3] 2 is a block diagram showing a configuration of a voice input unit of the imaging device according to the first embodiment. FIG. [Figure 4] 3 is a diagram showing an arrangement of microphones in a voice input unit of the imaging device according to the first embodiment. FIG. [Diagram 5] FIG. 4 is a diagram showing noise parameters in the first embodiment. [Figure 6] 13A and 13B are diagrams illustrating a frequency spectrum of an audio and a frequency spectrum of a noise parameter in a case where a drive sound occurs in a situation where it is considered that there is no environmental sound in the first embodiment. [Figure 7] 11 is a diagram showing a frequency spectrum of audio when a drive sound occurs in a situation where an environmental sound is present in the first embodiment; FIG. [Figure 8] FIG. 4 is a block diagram showing a configuration of a noise parameter selection unit in the first embodiment. [Figure 9] 4 is a timing chart relating to the audio noise reduction process in the first embodiment. [Figure 10] 11 is a diagram showing a frequency spectrum of audio when a drive sound occurs in a situation where an environmental sound is present in the first embodiment; FIG. [Figure 11] FIG. 4 is a block diagram showing a configuration of a noise parameter selection unit in the first embodiment. [Figure 12] 4 is a timing chart relating to a noise parameter updating method in the first embodiment. [Figure 13] 11 is a flowchart illustrating a noise parameter update process according to the first embodiment. [Figure 14] 5 is a timing chart relating to a noise parameter update process in the first embodiment. [Figure 15] 13 is a flowchart illustrating a noise parameter update process according to the second embodiment. [Figure 16] 10 is a timing chart relating to a noise parameter update process in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] [First Example] <External view of imaging device 100> 1(a) and (b) show an example of an external view of an image capture device 100 as an example of an audio processing device to which the present invention can be applied. Fig. 1(a) is an example of a front perspective view of the image capture device 100. Fig. 1(b) is an example of a rear perspective view of the image capture device 100. In Fig. 1, an optical lens (not shown) is attached to a lens mount 301.

[0012] The display unit 107 displays image data, text information, and the like. The display unit 107 is provided on the rear surface of the imaging device 100. The outside-finder display unit 43 is a display unit provided on the top surface of the imaging device 100. The outside-finder display unit 43 displays the settings of the imaging device 100, such as the shutter speed and aperture value. The eyepiece viewfinder 16 is a peer-type viewfinder. The user can check the focus and composition of the optical image of the subject by observing the focusing screen in the eyepiece viewfinder 16.

[0013] The release switch 61 is an operation member that allows the user to issue a shooting instruction. The mode changeover switch 60 is an operation member that allows the user to switch between various modes. The main electronic dial 71 is a rotating operation member. The user can change the settings of the imaging device 100, such as the shutter speed and aperture value, by turning this main electronic dial 71. The release switch 61, the mode changeover switch 60, and the main electronic dial 71 are included in the operation unit 112.

[0014] The power switch 72 is an operating member that switches the power of the imaging device 100 on and off. The sub electronic dial 73 is a rotary operating member. The user can use the sub electronic dial 73 to move the selection frame displayed on the display unit 107 and to advance images in playback mode. The cross key 74 is a cross key (four-way key) whose up, down, left, and right portions can be pressed. The imaging device 100 executes processing according to the portion (direction) of the cross key 74 that is pressed. The power switch 72, the sub electronic dial 73, and the cross key 74 are included in the operation unit 112.

[0015] The SET button 75 is a push button. The SET button 75 is mainly used by the user to confirm a selection item displayed on the display unit 107. The LV button 76 is a button used to switch live view (hereinafter, LV) on and off. In the video recording mode, the LV button 76 is used to instruct the start and stop of video shooting (recording). The enlargement button 77 is a push button for turning the enlargement mode on and off in the live view display in the shooting mode, and for changing the magnification ratio during the enlargement mode. The SET button 75, the LV button 76, and the enlargement button 77 are included in the operation unit 112.

[0016] In the playback mode, the enlargement button 77 functions as a button for increasing the magnification rate of image data displayed on the display unit 107. The reduction button 78 is a button for decreasing the magnification rate of image data enlarged and displayed on the display unit 107. The playback button 79 is an operation button for switching between the shooting mode and the playback mode. When the user presses the playback button 79 while the imaging device 100 is in the shooting mode, the imaging device 100 transitions to the playback mode, and the image data recorded on the recording medium 110 is displayed on the display unit 107. The reduction button 78 and the playback button 79 are included in the operation unit 112.

[0017] The quick return mirror 12 (hereinafter, mirror 12) is a mirror for switching the light beam incident from an optical lens attached to the imaging device 100 so that it is incident on either the eyepiece finder 16 side or the imaging unit 101 side. The mirror 12 is raised and lowered by the control unit 111 controlling an actuator (not shown) during exposure, live view shooting, and video shooting. The mirror 12 is normally arranged to allow the light beam to be incident on the eyepiece finder 16. When shooting or live view display is performed, the mirror 12 flips up (mirror up) so that the light beam is incident on the imaging unit 101. The center of the mirror 12 is a half mirror. A part of the light beam that passes through the center of the mirror 12 is incident on a focus detection unit (not shown) for focus detection.

[0018] The communication terminal 10 is a communication terminal for communication between the imaging device 100 and the optical lens 300 attached to the imaging device 100. The terminal cover 40 is a cover for protecting a connector (not shown) such as a connection cable that connects an external device to the imaging device 100. The lid 41 is a lid for a slot that stores the recording medium 110. The lens mount 301 is an attachment portion to which the optical lens 300 (not shown) can be attached.

[0019] The L microphone 201a and the R microphone 201b are microphones for picking up environmental sounds such as the user's voice, etc. When viewed from the rear of the imaging device 100, the L microphone 201a is disposed on the left side, and the R microphone 201b is disposed on the right side.

[0020] <Configuration of imaging device 100> FIG. 2 is a block diagram showing an example of the configuration of the imaging device 100 in this embodiment.

[0021] The optical lens 300 is a lens unit that is detachable from the imaging device 100. For example, the optical lens 300 is a zoom lens or a varifocal lens. The optical lens 300 has an optical lens, a motor for driving the optical lens, and a communication unit that communicates with a lens control unit 102 of the imaging device 100, which will be described later. The optical lens 300 can focus and zoom on a subject and correct camera shake by moving the optical lens using the motor based on a control signal received by the communication unit.

[0022] The imaging unit 101 has an imaging element for converting an optical image of a subject formed on an imaging surface via an optical lens 300 into an electrical signal, and an image processing unit for generating and outputting image data or video data from the electrical signal generated by the imaging element. The imaging element is, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). In this embodiment, a series of processes for generating image data including still image data and video data in the imaging unit 101 and outputting the image data from the imaging unit 101 is called "shooting". In the imaging device 100, the image data is recorded on a recording medium 110 (described later) in accordance with the DCF (Design rule for Camera File system) standard.

[0023] The lens control unit 102 transmits a control signal to the optical lens 300 via the communication terminal 10 based on data output from the imaging unit 101 and a control signal output from a control unit 111 (described later) to control the optical lens 300. The lens control unit 102 also receives lens information from the optical lens 300 attached to the imaging device 100. The lens information includes, for example, the type of lens, the lens model number, the zoom magnification, and the type of noise source.

[0024] The information acquisition unit 103 detects the inclination of the imaging device 100 and the temperature inside the housing of the imaging device 100. For example, the information acquisition unit 103 detects the inclination of the imaging device 100 by an acceleration sensor or a gyro sensor. Also, for example, the information acquisition unit 103 detects the temperature inside the housing of the imaging device 100 by a temperature sensor.

[0025] The audio input unit 104 generates audio data from audio acquired by a microphone. The audio input unit 104 acquires audio around the imaging device 100 by the microphone, performs analog-to-digital conversion (A / D conversion) on the acquired audio, and various audio processes to generate audio data. In this embodiment, the audio input unit 104 has a microphone. A detailed configuration example of the audio input unit 104 will be described later.

[0026] The volatile memory 105 temporarily records image data generated by the imaging unit 101 and audio data generated by the audio input unit 104. The volatile memory 105 is also used as a temporary recording area for image data displayed on the display unit 107, a working area for the control unit 111, etc.

[0027] The display control unit 106 controls the display unit 107 to display image data output from the imaging unit 101, characters for interactive operation, a menu screen, and the like. Furthermore, the display control unit 106 can control the display unit 107 to sequentially display digital data output from the imaging unit 101 during still image shooting and video shooting, thereby allowing the display unit 107 to function as an electronic viewfinder. For example, the display unit 107 is a liquid crystal display or an organic EL display. Furthermore, the display control unit 106 can also control the image data and video data output from the imaging unit 101, characters for interactive operation, a menu screen, and the like to be displayed on an external display via an external output unit 115, which will be described later.

[0028] The encoding processing unit 108 can encode the image data and audio data temporarily recorded in the volatile memory 105. For example, the encoding processing unit 108 can generate moving image data by encoding and data-compressing the image data according to the JPEG standard or RAW image format. For example, the encoding processing unit 108 can generate moving image data by encoding and data-compressing the moving image data according to the MPEG2 standard or the H.264 / MPEG4-AVC standard. Also, for example, the encoding processing unit 108 can generate audio data by encoding and data-compressing the audio data according to the AC3AAC standard, the ATRAC standard, or the ADPCM method. Also, the encoding processing unit 108 may encode the audio data according to, for example, the linear PCM method so as not to compress the data.

[0029] The recording control unit 109 can record data on the recording medium 110 and read data from the recording medium 110. For example, the recording control unit 109 can record still image data, video data, and audio data generated by the encoding processing unit 108 on the recording medium 110 and read data from the recording medium 110. The recording medium 110 is, for example, an SD card, a CF card, an XQD memory card, an HDD (magnetic disk), an optical disk, or a semiconductor memory. The recording medium 110 may be configured to be detachable from the imaging device 100, or may be built into the imaging device 100. That is, the recording control unit 109 only needs to have at least a means for accessing the recording medium 110.

[0030] The control unit 111 controls each component of the imaging device 100 via a data bus 116 in accordance with an input signal and a program described later. The control unit 111 has a CPU, a ROM, and a RAM for executing various controls. Note that, instead of the control unit 111 controlling the entire imaging device 100, a plurality of hardware may share the control of the entire imaging device. The ROM of the control unit 111 stores a program for controlling each component. The RAM of the control unit 111 is a volatile memory used for arithmetic processing and the like.

[0031] The operation unit 112 is a user interface for receiving instructions from a user for the imaging device 100. The operation unit 112 has, for example, a power switch 72 for turning the power of the imaging device 100 on or off, a release switch 61 for issuing an instruction to shoot, a playback button for issuing an instruction to play back image data or video data, a mode change switch 60, and the like.

[0032] The operation unit 112 outputs a control signal to the control unit 111 in response to a user's operation. The operation unit 112 may also include a touch panel formed on the display unit 107. The release switch 61 has SW1 and SW2. When the release switch 61 is pressed halfway, SW1 is turned on. This accepts a preparation instruction for performing a preparation operation for imaging, such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing. When the release switch 61 is pressed fully, SW2 is turned on. This user operation accepts an imaging instruction for performing an imaging operation. The operation unit 112 also includes an operation member (e.g., a button) that can adjust the volume of audio data reproduced from a speaker 114, which will be described later.

[0033] The audio output unit 113 can output audio data to the speaker 114 and the external output unit 115. The audio data input to the audio output unit 113 includes audio data read from the recording medium 110 by the recording control unit 109, audio data output from the non-volatile memory 117, and audio data output from the encoding processing unit. The speaker 114 is an electro-acoustic transducer capable of reproducing audio data.

[0034] The external output unit 115 can output image data, video data, audio data, etc. to an external device. The external output unit 115 is composed of, for example, a video terminal, a microphone terminal, a headphone terminal, etc.

[0035] The data bus 116 is a data bus for transmitting various data such as audio data, video data, and image data, and various control signals to each block of the imaging device 100 .

[0036] The nonvolatile memory 117 is a nonvolatile memory that stores programs, etc., which are executed by the control unit 111 and will be described later. Furthermore, sound data is recorded in the nonvolatile memory 117. This sound data is, for example, sound data of electronic sounds such as a focusing sound output when a subject is focused on, an electronic shutter sound output when an instruction to shoot is given, and an operation sound output when the imaging device 100 is operated.

[0037] <Operation of the imaging device 100> The operation of the imaging device 100 of this embodiment will now be described.

[0038] The imaging device 100 of this embodiment supplies power to each component of the imaging device from a power supply (not shown) in response to a user turning on the power by operating the power switch 72. For example, the power supply is a battery such as a lithium ion battery or an alkaline manganese dry battery.

[0039] The control unit 111 determines whether the camera will operate in a shooting mode or a playback mode, for example, based on the state of the mode changeover switch 60 in response to the supply of power. In the moving image recording mode, the control unit 111 records the moving image data output from the imaging unit 101 and the audio data output from the audio input unit 104 as one piece of moving image data with audio. In the playback mode, the control unit 111 reads out the image data or moving image data recorded on the recording medium 110 by the recording control unit 109, and controls the display unit 107 to display the image data or moving image data.

[0040] First, the moving image recording mode will be described. In the moving image recording mode, the control unit 111 first transmits a control signal to each component of the imaging device 100 to transition the imaging device 100 to a shooting standby state. For example, the control unit 111 controls the imaging unit 101 and the audio input unit 104 to perform the following operations.

[0041] The imaging unit 101 converts an optical image of a subject formed on an imaging surface via the optical lens 300 into an electrical signal, and generates video data from the electrical signal generated by the imaging element. The imaging unit 101 then transmits the video data to a display control unit 106, which displays it on a display unit 107. The user can prepare for shooting while viewing the video data displayed on the display unit 107.

[0042] The audio input unit 104 performs A / D conversion on analog audio signals input from the multiple microphones, respectively, to generate multiple digital audio signals. The audio input unit 104 then generates audio data for multiple channels from the multiple digital audio signals. The audio input unit 104 transmits the generated audio data to the audio output unit 113, which reproduces the audio data from the speaker 114. While listening to the audio data reproduced from the speaker 114, the user can adjust the volume of the audio data recorded in the audio-accompanied video data using the operation unit 112.

[0043] Next, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal to start shooting to each component of the imaging device 100. For example, the control unit 111 controls the imaging unit 101, the audio input unit 104, the encoding processing unit 108, and the recording control unit 109 to perform the following operations.

[0044] The imaging unit 101 converts an optical image of a subject formed on an imaging surface via an optical lens 300 into an electrical signal, and generates video data from the electrical signal generated by the imaging element. The imaging unit 101 then transmits the video data to a display control unit 106, which displays the video data on a display unit 107. The imaging unit 101 also transmits the generated video data to a volatile memory 105.

[0045] The audio input unit 104 performs A / D conversion on the analog audio signals input from the multiple microphones to generate multiple digital audio signals. The audio input unit 104 then generates multi-channel audio data from the multiple digital audio signals. The audio input unit 104 then transmits the generated audio data to the volatile memory 105.

[0046] The encoding processing unit 108 reads out the moving image data and audio data temporarily recorded in the volatile memory 105 and encodes each of them. The control unit 111 generates a data stream from the moving image data and audio data encoded by the encoding processing unit 108, and outputs it to the recording control unit 109. The recording control unit 109 records the input data stream as moving image data with audio on the recording medium 110 in accordance with a file system such as UDF or FAT.

[0047] Each component of the image capturing apparatus 100 continues to perform the above operations during moving image capturing.

[0048] Then, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal to end shooting to each component of the imaging device 100. For example, the control unit 111 controls the imaging unit 101, the audio input unit 104, the encoding processing unit 108, and the recording control unit 109 to perform the following operations.

[0049] The image capturing unit 101 stops generating moving image data, and the audio input unit 104 stops generating audio data.

[0050] The encoding processing unit 108 reads and encodes the remaining video data and audio data recorded in the volatile memory 105. The control unit 111 generates a data stream from the video data and audio data encoded by the encoding processing unit 108, and outputs the data stream to the recording control unit 109.

[0051] The recording control unit 109 records the data stream as a file of moving image data with audio on the recording medium 110 in accordance with a file system such as UDF or FAT. Then, when the input of the data stream stops, the recording control unit 109 completes the moving image data with audio. Upon completion of the moving image data with audio, the recording operation of the imaging device 100 stops.

[0052] In response to the stop of the recording operation, the control unit 111 transmits a control signal to each component of the imaging device 100 to transition to a shooting standby state. In response to this, the control unit 111 controls the imaging device 100 to return to the shooting standby state.

[0053] Next, the playback mode will be described. In the playback mode, the control unit 111 transmits a control signal to each component of the image pickup device 100 to transition to a playback state. For example, the control unit 111 controls the encoding processing unit 108, the recording control unit 109, the display control unit 106, and the audio output unit 113 to perform the following operations.

[0054] The recording control unit 109 reads out the moving image data with audio recorded on the recording medium 110 and transmits the read moving image data with audio to the encoding processing unit 108 .

[0055] The encoding processing unit 108 decodes the audio-accompanied video data into image data and audio data. The encoding processing unit 108 transmits the decoded video data to the display control unit 106 and the decoded audio data to the audio output unit 113.

[0056] The display control unit 106 displays the decoded image data on the display unit 107. The audio output unit 113 reproduces the decoded audio data through the speaker 114.

[0057] As described above, the imaging device 100 of this embodiment is capable of recording and reproducing image data and audio data.

[0058] In this embodiment, the audio input unit 104 performs audio processing such as adjustment of the level of an audio signal input from a microphone. In this embodiment, the audio input unit 104 performs this audio processing in response to the start of moving image recording. Note that this audio processing may be performed after the power of the imaging device 100 is turned on. Also, this audio processing may be performed in response to the selection of a shooting mode. Also, this audio processing may be performed in response to the selection of a mode related to audio recording, such as a moving image recording mode or a voice memo function. Also, this audio processing may be performed in response to the start of recording of an audio signal.

[0059] <Configuration of voice input unit 104> FIG. 3 is a block diagram showing an example of a detailed configuration of the voice input unit 104 in this embodiment.

[0060] In this embodiment, the audio input unit 104 has three microphones: an L microphone 201a, an R microphone 201b, and a noise microphone 201c. The L microphone 201a and the R microphone 201b are each an example of a first microphone. In this embodiment, the image capture device 100 collects environmental sounds using the L microphone 201a and the R microphone 201b, and records the audio signals input from the L microphone 201a and the R microphone 201b in a stereo format. For example, environmental sounds are sounds generated outside the housing of the image capture device 100 and outside the housing of the optical lens 300, such as the user's voice, animal cries, the sound of rain, and music.

[0061] Moreover, the noise microphone 201c is an example of a second microphone. The noise microphone 201c is a microphone for collecting noise such as a driving sound from a predetermined noise source (noise source) generated in the housing of the imaging device 100 and the housing of the optical lens 300. The noise source is, for example, a motor such as an ultrasonic motor (USM) and a stepper motor (STM). The noise is, for example, a vibration sound generated by driving a motor such as a USM and an STM. For example, the motor is driven in an AF process for focusing on a subject. The imaging device 100 acquires noise such as a driving sound generated in the housing of the imaging device 100 and the housing of the optical lens 300 by the noise microphone 201c, and generates a noise parameter to be described later using the audio data of the acquired noise. In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are non-directional microphones. An example of the arrangement of the L microphone 201a, the R microphone 201b, and the noise microphone 201c in this embodiment will be described later with reference to FIG.

[0062] The L microphone 201a, R microphone 201b, and noise microphone 201c each generate an analog audio signal from the captured sound and input it to the A / D conversion unit 202. Here, the audio signal input from the L microphone 201a is referred to as Lch, the audio signal input from the R microphone 201b is referred to as Rch, and the audio signal input from the noise microphone 201c is referred to as Nch.

[0063] The A / D conversion unit 202 converts analog audio signals input from the L microphone 201a, the R microphone 201b, and the noise microphone 201c into digital audio signals. The A / D conversion unit 202 outputs the converted digital audio signals to the FFT unit 203. In this embodiment, the A / D conversion unit 202 converts the analog audio signals into digital audio signals by performing sampling processing with a sampling frequency of 48 kHz and a bit depth of 16 bits.

[0064] The FFT unit 203 performs fast Fourier transform processing on the time domain digital audio signal input from the A / D conversion unit 202 to convert it into a frequency domain digital audio signal. In this embodiment, the frequency domain digital audio signal has a frequency spectrum of 1024 points in a frequency band from 0 Hz to 48 kHz. Also, the frequency domain digital audio signal has a frequency spectrum of 513 points in a frequency band from 0 Hz to 24 kHz, which is the Nyquist frequency. In this embodiment, the imaging device 100 performs noise reduction processing using the 513-point frequency spectrum from 0 Hz to 24 kHz of the audio data output from the FFT unit 203.

[0065] Here, the frequency spectrum of the Lch subjected to the fast Fourier transform is represented by 513-point array data of Lch_Before[0] to Lch_Before

[0512] . When these array data are collectively referred to, they are written as Lch_Before. Also, the frequency spectrum of the Rch subjected to the fast Fourier transform is represented by 513-point array data of Rch_Before[0] to Rch_Before

[0512] . When these array data are collectively referred to, they are written as Rch_Before. Here, in this embodiment, Lch_Before[0] is the frequency spectrum of a 0 Hz voice, and Lch_Before

[0512] is the frequency spectrum of a 24 kHz voice. Note that Lch_Before and Rch_Before are each an example of the first frequency spectrum data.

[0066] The Nch frequency spectrum after the fast Fourier transform is represented by array data of 513 points, Nch_Before[0] to Nch_Before

[0512] . When these array data are collectively referred to, they are written as Nch_Before. Note that Nch_Before is an example of second frequency spectrum data.

[0067] The noise data generating unit 204 generates data for reducing noise contained in Lch_Before and Rch_Before based on Nch_Before. In this embodiment, the noise data generating unit 204 generates array data of NL[0] to NL

[0512] for reducing noise contained in Lch_Before[0] to Lch_Before

[0512] , respectively, using noise parameters. In addition, the noise data generating unit 204 generates array data of NR[0] to NR

[0512] for reducing noise contained in Rch_Before[0] to Rch_Before

[0512] , respectively. The frequency points in the array data of NL[0] to NL

[0512] are the same as the frequency points in the array data of Lch_Before[0] to Lch_Before

[0512] . Furthermore, the frequency points in the array data of NR[0] to NR

[0512] are the same as the frequency points in the array data of Rch_Before[0] to Rch_Before

[0512] .

[0068] The sequence data NL[0] to NL

[0512] are collectively referred to as NL. The sequence data NR[0] to NR

[0512] are collectively referred to as NR. NL and NR are each an example of the third frequency spectrum data.

[0069] The noise parameter recording unit 205 records noise parameters for the noise data generating unit 204 to generate NL and NR from Nch_Before. The noise parameter recording unit 205 records a plurality of types of noise parameters according to the type of noise for each lens. The noise parameters for generating NL from Nch_Before are collectively referred to as PLx. The noise parameters for generating NR from Nch_Before are collectively referred to as PRx.

[0070] PLx and PRx have the same number of sequences as NL and NR, respectively. For example, PL1 is sequence data from PL1[0] to PL1

[0512] . The frequency points of PL1 are the same as the frequency points of Lch_Before. For example, PR1 is sequence data from PR1[0] to PR1

[0512] . The frequency points of PR1 are the same as the frequency points of Rch_Before. The noise parameters will be described later with reference to FIG. 5.

[0071] The noise parameter selection unit 206 determines the noise parameters to be used in the noise data generation unit 204 from the noise parameters recorded in the noise parameter recording unit 205. The noise parameter selection unit 206 determines the noise parameters to be used in the noise data generation unit 204 based on data such as Lch_Before, Rch_Before, Nch_Before, and lens information received from the lens control unit 102. The operation of the noise parameter selection unit 206 will be described in detail later with reference to FIG. 8.

[0072] In this embodiment, the noise parameter recording unit 205 records all the coefficients for the 513 frequency spectrum points as noise parameters. However, it is sufficient that at least the coefficients for the frequency points required for reducing noise are recorded, not the coefficients for all the 513 frequency points. For example, the noise parameter recording unit 205 may record, as noise parameters, the coefficients for the frequency spectrum of 20 Hz to 20 kHz, which are considered to be typical audible frequencies, and may not record the coefficients for other frequency spectra. Also, for example, the coefficients for the frequency spectrum whose coefficient value is zero may not be recorded in the noise parameter recording unit 205 as noise parameters.

[0073] The subtraction processing unit 207 subtracts NL and NR from Lch_Before and Rch_Before, respectively. For example, the subtraction processing unit 207 has an L subtractor 207a that subtracts NL from Lch_Before, and an R subtractor 207b that subtracts NR from Rch_Before. The L subtractor 207a subtracts NL from Lch_Before, and outputs array data of 513 points from Lch_After[0] to Lch_After

[0512] . The R subtractor 207b subtracts NR from Rch_Before, and outputs array data of 513 points from Rch_After[0] to Rch_After

[0512] . In this embodiment, the subtraction processing unit 207 performs subtraction processing by a spectral subtraction method.

[0074] The iFFT unit 208 performs inverse fast Fourier transform (inverse Fourier transform) on the frequency domain digital audio signal input from the subtraction processing unit 207 to convert it into a time domain digital audio signal.

[0075] The audio processor 209 executes audio processing on the time domain digital audio signal, such as an equalizer, an auto level controller, and stereo enhancement processing, etc. The audio processor 209 outputs the processed audio data to the volatile memory 105.

[0076] The noise parameter update unit 216 updates the noise parameters recorded in the noise parameter recording unit 205. The noise parameter update unit 216 generates noise parameters based on data such as Lch_Before, Rch_Before, Nch_Before, and lens information received from the lens control unit 102. Then, the noise parameter update unit 216 uses the generated noise parameters to update the noise parameters recorded in the noise parameter recording unit 205. The operation of the noise parameter update unit 216 will be described later with reference to FIG. 11.

[0077] In this embodiment, the imaging device 100 has two microphones as the first microphone, but the imaging device 100 may have one microphone or three or more microphones as the first microphone. For example, when the imaging device 100 has one microphone as the first microphone in the audio input unit 104, the imaging device 100 records audio data acquired by the one microphone in a monaural format. Also, for example, when the imaging device 100 has three or more microphones as the first microphone in the audio input unit 104, the imaging device 100 records audio data acquired by the three or more microphones in a surround format.

[0078] In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are non-directional microphones, but these microphones may be directional microphones.

[0079] <Arrangement of microphones in the voice input unit 104> Here, a description will be given of an example of the arrangement of the microphones of the voice input unit 104 of this embodiment. Fig. 4 shows an example of the arrangement of the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

[0080] 4 is an example of a cross-sectional view of a portion of the imaging device 100 to which the L microphone 201a, the R microphone 201b, and the noise microphone 201c are attached. This portion of the imaging device 100 is composed of an exterior part 302, a microphone bushing 303, and a fixed part 304.

[0081] The exterior part 302 has a hole (hereinafter referred to as a microphone hole) for inputting environmental sound to the microphone. In this embodiment, the microphone hole is formed above the L microphone 201a and the R microphone 201b. On the other hand, the noise microphone 201c is provided to acquire driving sound generated within the housing of the image capture device 100 and within the housing of the optical lens 300, and does not need to acquire environmental sound. Therefore, in this embodiment, no microphone hole is formed in the exterior part 302 above the noise microphone 201c.

[0082] Drive sounds generated within the housings of the image capture device 100 and the optical lens 300 are picked up by the L microphone 201a and the R microphone 201b through the microphone holes. When drive sounds or the like are generated within the housings of the image capture device 100 and the optical lens 300 while the ambient sound is low, the sound picked up by each microphone is mainly this drive sound. Therefore, the sound level from the noise microphone 201c is higher than the sound levels from the L microphone 201a and the R microphone 201b. That is, in this case, the relationship between the levels of the sound signals output from each microphone is as follows: Lch ≒ Rch <Nch Furthermore, when the environmental sound becomes louder, the sound level of the environmental sound from the L microphone 201a and the R microphone 201b becomes louder than the sound level of the drive sound from the microphone 201c generated by the image capture device 100 or the optical lens 300. Therefore, in this case, the relationship between the levels of the audio signals output from each microphone is as follows: Lch ≒ Rch > Nch In this embodiment, the shape of the microphone hole formed in exterior part 302 is elliptical, but it may be other shapes such as circular or rectangular. Also, the shape of the microphone hole on microphone 201a and the shape of the microphone hole on microphone 201b may be different from each other.

[0083] In this embodiment, the noise microphone 201c is disposed close to the L microphone 201a and the R microphone 201b. Furthermore, in this embodiment, the noise microphone 201c is disposed between the L microphone 201a and the R microphone 201b. As a result, the audio signal generated by the noise microphone 201c from drive sounds and the like generated within the housing of the image capture device 100 and the housing of the optical lens 300 becomes similar to the audio signal generated by the L microphone 201a and the R microphone 201b from this drive sound and the like.

[0084] The microphone bushing 303 is a member for fixing the L microphone 201a, the R microphone 201b, and the noise microphone 201c. The fixing portion 304 is a member for fixing the microphone bushing 303 to the exterior portion 302.

[0085] In this embodiment, exterior part 302 and fixed part 304 are made of a molded material such as PC material. Also, exterior part 302 and fixed part 304 may be made of a metal material such as aluminum or stainless steel. Also, in this embodiment, microphone bushing 303 is made of a rubber material such as ethylene propylene diene rubber.

[0086] <Noise parameters> FIG. 5 is an example of noise parameters recorded in the noise parameter recording unit 205. The noise parameters shown in FIG. 5 are noise parameters for noise generated from a certain lens. The noise parameters are parameters for correcting an audio signal generated by the noise microphone 201c acquiring drive sounds generated in the housing of the image capture device 100 and the housing of the optical lens 300. As shown in FIG. 5, in this embodiment, PLx and PRx are recorded in the noise parameter recording unit 205. In this embodiment, the source of the drive sounds will be described as being in the housing of the optical lens 300. The drive sounds generated in the housing of the optical lens 300 are transmitted to the housing of the image capture device 100 via the lens mount 301 and are acquired by the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

[0087] The frequency of the driving sound varies depending on the type of the driving sound. Therefore, in this embodiment, the imaging device 100 records a plurality of noise parameters corresponding to the type of the driving sound (noise). Then, noise data is generated using one of the plurality of noise parameters. In this embodiment, the imaging device 100 records a noise parameter for white noise as a constant noise. The imaging device 100 also records a noise parameter for short-term noise generated by, for example, meshing of gears in the optical lens 300. The imaging device 100 also records a noise parameter for, for example, sliding noise in the housing of the lens 300 as a long-term noise. In addition, the imaging device 100 may record noise parameters for each type of the optical lens 300, and for each temperature in the housing of the imaging device 100 and tilt of the imaging device 100 detected by the information acquisition unit 103.

[0088] <How to generate noise data> 6 and 7, the noise data generation process in the noise data generator 204 will be described. Here, the noise data generation process for the Lch data will be described, but the method of generating noise data for the Rch data is similar.

[0089] First, a process of generating noise parameters in a situation where it is considered that there is no environmental sound will be described. Fig. 6(a) is an example of a frequency spectrum of Lch_Before when a driving sound occurs within the housing of the optical lens 300 in a situation where it is considered that there is no environmental sound. Fig. 6(b) is an example of a frequency spectrum of Nch_Before when a driving sound occurs within the housing of the optical lens 300 in a situation where it is considered that there is no environmental sound. The horizontal axis indicates the frequency from the 0th point to the 512th point, and the vertical axis indicates the amplitude of the frequency spectrum.

[0090] Since it is assumed that there is no environmental sound, the amplitude of the frequency spectrum in the same frequency band is large in Lch_Before and Nch_Before. In addition, since the drive sound is generated inside the housing of the optical lens 300, the amplitude of each frequency spectrum for the same drive sound tends to be larger in Nch_Before than in Lch_Before.

[0091] FIG. 6(c) is an example of PLx in this embodiment. In this embodiment, PLx is a coefficient of each frequency spectrum calculated by dividing the amplitude of each frequency spectrum of Lch_Before by the amplitude of each frequency spectrum of Nch_Before in a situation where the environmental sound is small. The result of this division is described as Lch_Before / Nch_Before. In other words, PLx is the ratio of the amplitudes of Lch_Before and Nch_Before. The noise parameter recording unit 205 records the value of Lch_Before / Nch_Before as the noise parameter PLx. As described above, the amplitude of the frequency spectrum for the same driving sound tends to be larger in Nch_Before than in Lch_Before, so the value of each coefficient of the noise parameter PLx tends to be smaller than 1. However, when the value of Nch_Before[n] is smaller than a predetermined threshold value, the noise parameter recording unit 205 records the noise parameter PLx as PLx[n]=0.

[0092] Next, a process of applying the generated noise parameters to Nch_Before will be described. Fig. 7(a) is an example of a frequency spectrum of Lch_Before when a drive sound occurs within the housing of the optical lens 300 in a situation where an environmental sound is present. Fig. 7(b) is an example of a frequency spectrum of Nch_Before when a drive sound occurs within the housing of the optical lens 300 in a situation where an environmental sound is present. The horizontal axis indicates the frequency from the 0th point to the 512th point, and the vertical axis indicates the amplitude of the frequency spectrum.

[0093] 7(c) is an example of NL when a driving sound occurs inside the housing of the optical lens 300 in a situation where an environmental sound exists. The noise data generating unit 204 multiplies each frequency spectrum of Nch_Before by each coefficient of PLx to generate NL. NL is the frequency spectrum generated in this way.

[0094] 7D is an example of Lch_After when a driving sound occurs inside the housing of the optical lens 300 in a situation where an environmental sound exists. The subtraction processing unit 207 subtracts NL from Lch_Before to generate Lch_After. Lch_After is a frequency spectrum generated in this way.

[0095] This allows the imaging device 100 to reduce noise caused by driving sounds within the housing of the optical lens 300, and record environmental sounds with less noise.

[0096] <Explanation of the noise parameter selection unit 206> FIG. 8 is a block diagram showing an example of a detailed configuration of the noise parameter selection unit 206. As shown in FIG.

[0097] The noise parameter selection unit 206 receives Lch_Before, Rch_Before, Nch_Before, lens information, and a lens control signal.

[0098] The Nch noise detection unit 2061 detects from Nch_Before noise caused by drive sounds generated within the housing of the optical lens 300. In this embodiment, the Nch noise detection unit 2061 detects noise by using data of 513 points of Nch_Before.

[0099] The environmental sound detection unit 2062 detects the level of the environmental sound from Lch_Before and Rch_Before.

[0100] The noise determination unit 2063 determines the noise parameters to be used by the noise data generation unit 204 based on the lens information, the lens control signal, the data input from the Nch noise detection unit 2061, and the data input from the environmental sound detection unit. The noise determination unit 2063 outputs data indicating the type of the determined noise parameters to the noise data generation unit 204 and the noise parameter update unit 216.

[0101] The Nch differentiation unit 2064 performs differentiation processing on Nch_Before. The Nch differentiation unit 2064 outputs data indicating the result of differentiation processing on Nch_Before to the short-term noise detection unit 2065. The short-term noise detection unit 2065 detects whether or not short-term noise is occurring based on the data input from the Nch differentiation unit 2064. The short-term noise detection unit 2065 outputs data indicating whether or not short-term noise is occurring to the noise determination unit 2063. Note that the Nch differentiation unit 2064 and the short-term noise detection unit 2065 are included in the Nch noise detection unit 2061.

[0102] The Nch integrator 2066 executes integration processing on Nch_Before. The Nch integrator 2066 outputs data indicating the result of differentiation processing of Nch_Before to the long-term noise detector 2067. The long-term noise detector 2067 detects whether or not long-term noise is occurring based on the data input from the Nch integrator 2066. The long-term noise detector 2067 outputs data indicating whether or not long-term noise is occurring to the noise determination unit 2063. Note that the Nch integrator 2066 and the long-term noise detector 2067 are included in the Nch noise detector 2061.

[0103] The environmental sound extraction unit 2068 extracts environmental sounds. In this embodiment, the environmental sound extraction unit 2068 extracts data of frequencies that are less affected by noise based on a noise parameter. For example, the environmental sound extraction unit 2068 extracts data of frequencies where the noise parameter is equal to or less than a predetermined value. Then, the environmental sound extraction unit 2068 outputs data indicating the volume of the environmental sound based on the extracted frequency data. Note that the environmental sound extraction unit 2068 is included in the environmental sound detection unit 2062.

[0104] The environmental sound determination unit 2069 determines the volume of the environmental sound. The environmental sound determination unit 2069 inputs data indicating the determined volume of the environmental sound to the Nch noise detection unit 2061 and the noise determination unit 2063. The Nch noise detection unit 2061 changes a first threshold value and a second threshold value, which will be described later, based on the data indicating the volume of the environmental sound input from the environmental sound determination unit 2069. Note that the environmental sound determination unit 2069 is included in the environmental sound detection unit 2062.

[0105] <Timing chart for noise reduction processing> The noise reduction process in this embodiment will be described with reference to FIG.

[0106] 9(a) to (i) are examples of timing charts of audio processing in the noise data generator 204, the noise parameter selector 206, and the subtraction processor 207. In this embodiment, for the sake of simplicity, audio processing for the Lch will be described, but audio processing for the Rch is similar. The horizontal axis of the graphs in Figs. 9(a) to (i) is the time axis.

[0107] Fig. 9(a) shows an example of a lens control signal. The lens control signal is a signal that the lens control unit 102 uses to instruct the optical lens 300 to drive. In this embodiment, the level of the lens control signal is expressed as two values, High and Low. When the level of the lens control signal is High, the lens control unit 102 is instructing the optical lens 300 to drive. When the level of the lens control signal is Low, the lens control unit 102 is not instructing the optical lens 300 to drive.

[0108] FIG. 9(b) is a graph showing an example of the value of Lch_Before[n]. The vertical axis indicates the value of Lch_Before[n]. In this embodiment, Lch_Before[n] is a signal at the nth frequency point, among the Lch_Before output from the FFT unit 203, at which a signal indicating the driving sound of the optical lens 300 appears characteristically. Note that in this embodiment, the signal at the nth frequency point will be described, but audio processing is similarly performed for other frequencies. Furthermore, the signals indicated by signal X and signal Y are signals containing noise. In this embodiment, signal X indicates a signal containing short-term noise. Signal Y indicates a noise signal containing long-term noise.

[0109] 9(c) is a graph showing an example of the volume of the environmental sound extracted by the environmental sound extraction unit 2068. The vertical axis indicates the level of the audio signal generated from the acquired environmental sound. Threshold value Th1 and threshold value Th2 are two threshold values ​​used in the environmental sound determination unit 2069.

[0110] Fig. 9(d) is a graph showing an example of the value of Nch_Before[n]. Nch_Before[n] is a signal at the nth frequency point, among Nch_Before output from the FFT unit 203, at which a signal indicating the driving sound of the optical lens 300 appears characteristically. The vertical axis indicates the value of Nch_Before[n]. In Nch_Before[n], the noise signals indicated by the signals X and Y in Fig. 9(b) appear more characteristically than in Lch_Before.

[0111] FIG. 9(e) is a graph showing an example of the value of Ndiff[n]. Ndiff[n] indicates the value of the signal at the n-th frequency point among Ndiff output from the Nch differentiation unit 2064. The vertical axis indicates the value of Ndiff[n]. If the amount of change in the value of Nch_Before[n] per predetermined time is large, the value of Ndiff[n] becomes large. The short-term noise detection unit 2065 has a threshold value Th_Ndiff[n], which is a first threshold value, in order to detect short-term noise. The threshold value Th_Ndiff[n] changes between levels 1 to 3 based on the data indicating the volume of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold value Th_Ndiff[n] is set to level 2. The level of the threshold value Th_Ndiff[n] is represented by a horizontal dashed line.

[0112] FIG. 9(f) is a graph showing an example of the value of Nint[n]. In this embodiment, Nint[n] indicates the value of the signal at the n-th frequency point among Nint output from the Nch integration unit 2066. The vertical axis indicates the value of Nint[n]. If Nch_Before[n] is continuously large, the value of Nint[n] increases. The long-term noise detection unit 2067 has a second threshold, a threshold Th_Nint[n], in order to detect long-term noise. The threshold Th_Nint[n] changes between levels 1 to 3 based on the data indicating the volume of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold Th_Nint[n] is set to level 2. The level of the threshold Th_Nint[n] is represented by a horizontal dashed line.

[0113] 9(g) shows an example of noise parameters selected by the noise parameter selection unit 206. In this embodiment, the solid part indicates that only the noise parameter of PL1 is selected. The diagonal line part indicates that the noise parameters of PL1 and PL2 are selected. The checkered part indicates that the noise parameters of PL1 and PL3 are selected.

[0114] 9(h) is a graph showing an example of the value of NL[n]. In this embodiment, NL[n] indicates the signal value of the n-th frequency point among the NL generated by the noise data generating unit 204. The vertical axis indicates the value of NL[n].

[0115] 9(i) is a graph showing an example of the value of Lch_After[n]. In this embodiment, Lch_After[n] indicates the value of the signal at the n-th frequency point among the Lch_After output from the subtraction processing unit 207. The vertical axis indicates the value of Lch_After[n].

[0116] Next, the timing of each operation will be described using times t701 to t709.

[0117] At time t701, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9(a)). At time t701, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the short-term noise detection unit 2065 lowers the threshold Th_Ndiff[n] to level 1 (FIG. 9(e)). Also at time t701, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold Th_Nint[n] to level 1 (FIG. 9(f)).

[0118] At time t702, the optical lens 300 is driven, generating short-term drive sound such as the sound of gears meshing. The noise microphone 201c picks up the short-term drive sound, causing the value of Ndiff[n] to exceed the threshold value Th_Ndiff[n] (FIG. 9(e)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL2 (FIG. 9(g)). The noise data generation unit 204 generates NL[n] based on Nch_Before[n] and the noise parameters PL1 and PL2 (FIG. 9(h)). The subtraction processing unit 207 subtracts NL[n] from Lch_Before[n] and outputs Lch_After[n] (FIG. 9(i)). In this case, Lch_After[n] becomes an audio signal in which constant noise and short-term noise have been reduced.

[0119] At time t703, the optical lens 300 starts to drive continuously, and a long-term driving sound such as a sliding sound is generated in the housing of the optical lens 300. The noise microphone 201c picks up the long-term driving sound, and the value of Nint[n] exceeds the threshold value Th_Nint[n] (FIG. 9(f)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL3 (FIG. 9(g)). The noise data generation unit 204 generates NL[n] based on Nch_Before[n] and the noise parameters PL1 and PL3 (FIG. 9(h)). The subtraction processing unit 207 subtracts NL[n] from Lch_Before[n] and outputs Lch_After[n] (FIG. 9(i)). In this case, Lch_After[n] becomes an audio signal in which constant noise and long-term noise have been reduced.

[0120] At time t704, the optical lens 300 stops continuous driving. Since the noise microphone 201c no longer picks up the long-term driving sound, the value of Nint[n] becomes equal to or less than the threshold value Th_Nint[n] (FIG. 9(f)). In response to this, the noise parameter selection unit 206 selects the noise parameter PL1 (FIG. 9(g)). The noise data generation unit 204 generates NL[n] based on Nch_Before[n] and the noise parameter PL1 (FIG. 9(h)). The subtraction processing unit 207 subtracts NL[n] from Lch_Before[n] and outputs Lch_After[n] (FIG. 9(i)). In this case, Lch_After[n] becomes an audio signal with constant noise reduced.

[0121] At time t705, the lens control unit 102 outputs a Low signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9(a)). In this case, since the possibility of drive sound occurring within the housing of the optical lens 300 is low, the short-term noise detection unit 2065 increases the threshold Th_Ndiff[n] to level 2 (FIG. 9(e)). Also, in this case, since the possibility of drive sound occurring within the housing of the optical lens 300 is low, the long-term noise detection unit 2067 increases the threshold Th_Nint[n] to level 2 (FIG. 9(f)).

[0122] At time t706, the volume of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold Th1. When the environmental sound is loud, the user is less likely to perceive the noise contained in the audio signal, so the short-term noise detection unit 2065 raises the threshold Th_Ndiff[n] to level 3 (FIG. 9(e)). Also, when the environmental sound is loud, the user is less likely to perceive the noise contained in the audio signal, so the long-term noise detection unit 2067 raises the threshold Th_Nint[n] to level 3 (FIG. 9(f)).

[0123] At time t707, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9(a)). In this case, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the short-term noise detection unit 2065 lowers the threshold Th_Ndiff[n] to level 2 (FIG. 9(e)). Also, in this case, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold Th_Nint[n] to level 2 (FIG. 9(f)).

[0124] At time t708, the volume of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold value Th2. Here, the noise parameter selection unit 206 selects only the noise parameter PL1 regardless of the data input from the Nch noise detection unit 2061. In this way, when the environmental sound is very loud, the user is less likely to perceive the noise contained in the audio signal, so the imaging device 100 reduces only the constant noise, thereby recording environmental sound that is more natural than when processing is performed to further reduce short-term noise and long-term noise.

[0125] As described above, the imaging device 100 can record environmental sounds with reduced noise by performing noise reduction processing using the noise microphone 201c, which is the second microphone.

[0126] Furthermore, the image capturing device 100 detects the occurrence of noise using the output signal of the noise microphone 201c, and sets the noise parameters in accordance with the timing of detecting the occurrence of the noise. Therefore, the image capturing device 100 can appropriately set the noise parameters in synchronization with the occurrence of noise, and appropriately reduce the noise.

[0127] Furthermore, when the volume of the environmental sound is equal to or less than the threshold Th2, the imaging device 100 executes noise reduction processing according to the noise detected by the Nch noise detection unit 206, and when the environmental sound is greater than the threshold Th2, reduces only constant noise. This enables the imaging device 100 to record environmental sound with reduced noise according to the volume of the environmental sound so as to reduce discomfort to the user.

[0128] In this embodiment, the imaging device 100 reduces the driving sound generated within the housing of the optical lens 300, but the driving sound generated within the imaging device 100 may be reduced. The driving sound generated within the imaging device 100 is, for example, noise from the board and radio wave noise. The noise from the board is, for example, a sound generated by the creaking of the board when a voltage is applied to a capacitor on the board.

[0129] The thresholds Th1 and Th2 of the environmental sound determination unit 2069, the threshold Th_Ndiff[n] of the short-term noise detection unit 2065, and the threshold Th_Nint[n] of the long-term noise detection unit 2067 are determined based on the generated driving sound and environmental sound. Therefore, the imaging device 100 may change these thresholds depending on the type of the optical lens 300, the inclination of the imaging device 100, and the like.

[0130] <Noise parameters for each lens> First, a case where the user replaces the optical lens 300 and the same noise parameters are applied to each lens will be described with reference to Fig. 7 and Fig. 10. It is assumed that different lenses are attached to the imaging device 100 in Fig. 7 and Fig. 10. It is assumed that the same environmental sound is occurring in Fig. 7 and Fig. 10.

[0131] 7(a) and 10(a) show the amplitude of each frequency spectrum of Lch_Before. FIG. 7(b) and 10(b) show the amplitude of each frequency spectrum of Nch_Before. Comparing the frequency spectra, it can be seen that they are at least partially different from each other. This is because the noise picked up by the L microphone 201a, the R microphone 201b, and the noise microphone 201c differs depending on the shape and structure of the lens, and the number and positions of noise sources such as drivers included in the lens.

[0132] 7(c) and 10(c) show the amplitude of each frequency spectrum of NL, where the same noise parameters are used to generate NL regardless of the lens attached to the imaging device 100.

[0133] In FIG. 7(d) and FIG. 10(d), the amplitude of each frequency spectrum of Lch_After is shown. Since Lch_After is an audio signal recorded as environmental sound, ideally the frequency spectrum shown in FIG. 7(d) and FIG. 10(d) will be the same. However, when comparing the frequency spectra of FIG. 7(d) and FIG. 10(d), it can be seen that they are at least partially different from each other. This is because the frequency spectrum of NL subtracted from Lch_Before is different from the frequency spectrum of noise generated in each lens. The reason why the frequency spectrum of NL is different from the frequency spectrum of noise is because the noise parameters are not parameters made to match the noise generated from the lens, but are general-purpose parameters for a certain type of noise. In other words, if the same noise parameters are applied to noise generated from various lenses, the audio data recorded on the recording medium 110 as environmental sound will be different for each lens attached to the imaging device 100.

[0134] Therefore, in this embodiment, the image capturing apparatus 100 updates the noise parameters in response to lens replacement, thereby effectively reducing noise and more accurately recording environmental sounds.

[0135] <Noise parameter update> Here, a process in which the noise parameter update unit 216 updates the noise parameters will be described. In this embodiment, a process in which the imaging device 100 updates the noise parameters of PLx will be described. Note that the process in which the noise parameters of PRx are updated is similar to the process in which the noise parameters of PLx are updated.

[0136] FIG. 11 is an example of a block diagram of the noise parameter update unit 216. As shown in FIG.

[0137] The noise parameter calculation section 2161 generates PLx from the input Lch_Before and Nch_Before.

[0138] The comparator 2162 compares the PLx generated in the noise parameter calculation unit 2161 with the PLx recorded in the noise parameter recording unit 205. Here, the comparator 2162 reads the PLx recorded in the noise parameter recording unit 205 based on data indicating the type of noise parameter input from the noise parameter selection unit 206. For example, when the type of noise parameter is long-term noise, the comparator 2162 reads the noise parameter for the long-term noise from the noise parameter recording unit 205. The comparator 2162 overwrites and records the coefficients that are smaller in value than the coefficients of the frequency spectrum of PLx recorded in the noise parameter recording unit 205, among the frequency spectrum coefficients of PLx generated by the noise parameter calculation unit 2161. The reason for such updating is as follows.

[0139] For example, let S be the amplitude of the frequency spectrum of the environmental sound picked up by the L microphone 201a at a certain frequency, and Nfl be the amplitude of the frequency spectrum of the noise sound from the lens. Also, let Nfn be the amplitude of the frequency spectrum of the noise sound from the lens picked up by the noise microphone 201c at that frequency. Note that in this embodiment, the amplitude of the environmental sound picked up by the noise microphone 201c is sufficiently smaller than Nfn. In this case, PLx can be expressed by the following formula. PLx = (S + Nfl) / Nfn In order to effectively reduce noise, the noise parameter PLx (≈Nfl / Nfn) is preferably set when S is close to 0. That is, the desirable value of the noise parameter PLx is a small value. Also, when S is close to 0, the environmental sound is small. Therefore, in this embodiment, the comparator 2162 records, as a new noise parameter, the coefficient of the frequency spectrum that is smaller than the value of PLx recorded in the noise parameter recording unit 205, among the values ​​of PLx generated in the noise parameter calculation unit 2161. In this case, the amplitude of the environmental sound when PLx is generated in the noise parameter calculation unit 2161 is smaller than the amplitude of the environmental sound when PLx recorded in the noise parameter recording unit 205 is generated.

[0140] The comparator 2162 compares PLx for each frequency spectrum recorded in the noise parameters. The comparator 2162 does not update the coefficients of frequency spectra that are equal to or greater than the PLx value recorded in the noise parameter recording unit 205, among the PLx values ​​generated in the noise parameter calculation unit 2161.

[0141] The counter 2163 counts the number of times the noise parameters are compared in the comparator 2162. In this embodiment, the initial value of the number of times the noise parameters are compared is 0. When the lens is replaced, the counter 2163 resets the number of times the noise parameters are compared. In this embodiment, the counter 2163 resets the number of times the noise parameters are compared in response to receiving lens information from the lens control unit 102.

[0142] Now, the noise parameter update process by the noise parameter update unit 216 will be described with reference to the timing chart of Fig. 12. In this timing chart, the timing when the user turns on the power of the imaging device 100 is set to time t0. In this embodiment, the noise parameter PLx for one frequency spectrum will be described. In this embodiment, the initial value of this noise parameter PLx is set to 1.4.

[0143] In this embodiment, after time t0, the imaging device 100 starts a live view operation and performs AF processing on the subject. If noise caused by the driving of the motor of the optical lens 300 or the like is detected during this AF processing, a noise parameter update process is performed. During the noise parameter update process, the noise parameter selection unit 206 performs a process of determining the type of noise.

[0144] At time t1, the noise parameter calculation unit 2161 calculates the noise parameter PLx in response to the start of driving the lens. For example, the lens is driven in response to the image capture device 100 receiving an operation for focusing on a subject. Here, for example, when the lens is driven from time t1 to time t2, the lens control signal becomes High during that time. The noise parameter calculation unit 2161 calculates PLx based on data indicating the type of noise parameter from the noise parameter selection unit 206 while the lens control signal is High. Also, the noise parameter calculation unit 2161 ends the calculation of PLx in response to the lens control signal becoming Low at time t2. Here, the value of PLx generated by the noise parameter calculation unit 2161 is set to 1.3. Note that, for example, if multiple types of noise occur in the optical lens 300 between time t1 and time t2, the noise parameter calculation unit 2161 generates PLx for each noise. The noise parameter calculation unit 2161 outputs data indicating the type of noise parameter and the generated PLx to the comparator 2162.

[0145] At time t2, the comparator 2162 compares PLx generated by the noise parameter calculation unit 2161 with PLx recorded in the noise parameter recording unit 205, and determines the smaller value as the value of the noise parameter PLx. At time t2, PLx (=1.3) newly generated by the noise parameter calculation unit 2161 is smaller than PLx recorded in the noise parameter recording unit 205. Therefore, the comparator 2162 updates PLx by recording 1.3 in the noise parameter recording unit 205 as the value of the noise parameter PLx.

[0146] The comparator 2162 determines the type of noise parameter to be updated based on the data indicating the type of noise parameter and PLx output from the noise parameter calculation section 2161. For example, when PLx for short-term noise is output from the noise parameter calculation section 2161, the comparator 2162 compares PLx output from the noise parameter calculation section 2161 with PL2 recorded in the noise parameter recording section 205.

[0147] When the noise parameter calculation unit 2161 outputs multiple PLx for the same type of noise, the comparator 2162 uses the noise parameters generated by combining the coefficients with the lowest values ​​for each frequency spectrum among the multiple PLx. This is because it is considered that the amplitude of the environmental sound when generated is smaller when the coefficient has a lower value. In this case, the comparator 2162 compares the noise parameters generated by combining them with the PLx recorded in the noise parameter recording unit 205.

[0148] Similarly, when the lens is driven from time t3 to time t4, the PLx calculated by the noise parameter calculation unit 2161 at time t4 is 1.1. At time t4, the comparator 2162 compares the PLx calculated by the noise parameter calculation unit 2161 with the PLx recorded in the noise parameter recording unit 205, and determines the smaller value as the noise parameter PLx. Since the PLx (=1.1) calculated by the noise parameter calculation unit 2161 is smaller at time t4, the comparator 2162 updates PLx by recording 1.1 in the noise parameter recording unit 205 as the value of the noise parameter PLx.

[0149] On the other hand, when the lens is driven from time t5 to time t6, the PLx calculated by the noise parameter calculation unit 2161 at time t6 is 1.6. At time t6, the comparator 2162 compares the PLx calculated by the noise parameter calculation unit 2161 with the PLx recorded in the noise parameter recording unit 205, and determines the smaller value as the noise parameter PLx. At time t6, the PLx (=1.1) recorded in the noise parameter recording unit 205 is smaller, so the comparator 2162 does not update PLx.

[0150] Thereafter, in response to the optical lens 300 being driven in a similar manner, the noise parameter update unit 216 executes the process of updating the noise parameter PLx.

[0151] Additionally, when the user performs an operation that causes the imaging device 100 to perform an operation involving noise before starting video shooting, the imaging device 100 also performs the noise parameter update process. For example, when the user half-presses the release switch 61, the imaging device 100 drives the motor of the optical lens 300 to focus on the subject. Using the noise generated at this time, the imaging device 100 performs the noise parameter update process.

[0152] Note that even when the noise parameter update unit 216 updates the noise parameters, the noise parameter recording unit 205 holds the initial values ​​of the noise parameters. This is because the noise parameter update unit 216 changes the noise parameters to the initial values ​​when the optical lens 300 attached to the imaging device 100 is replaced. As described above, when the optical lens 300 attached to the imaging device 100 is replaced, the noise acquired by the L microphone 201a, the R microphone 201b, and the noise microphone 201c changes. In this embodiment, in order to accommodate the change, the noise parameter update unit 216 updates the noise parameters from general-purpose initial values. By updating the noise parameters from general-purpose initial values ​​every time the lens is replaced, the imaging device 100 can perform noise reduction processing using noise parameters adapted to the attached lens.

[0153] Next, the noise parameter update process of the imaging device 100 will be described with reference to the flowchart of Fig. 13 and the timing chart of Fig. 14. The process of the imaging device 100 is triggered, for example, by the user operating the power switch 72 to turn on the power. Also, the process of the imaging device 100 is triggered, for example, by the user operating the mode dial to set the imaging device 100 to a shooting mode.

[0154] Here, in this embodiment, after the imaging device 100 is turned on, it displays a live view of an image captured by the imaging unit 101.

[0155] In step S1300, the lens control unit 102 receives lens information from the optical lens 300. The lens information includes, for example, the type of lens, the model number of the lens, and the type of noise source. The lens control unit 102 outputs the received lens information to the control unit 111 and the counter 2163. The control unit 111 records the received lens information in the non-volatile memory 117.

[0156] In step S1301, the lens control unit 102 determines whether the lens has been replaced. For example, the lens control unit 102 compares the lens information recorded in the non-volatile memory 117 before the process of step S1301 with the lens information received in step S1300. If it is determined that the two pieces of lens information match, the lens control unit 102 determines that the lens has not been replaced. If it is determined that the two pieces of lens information do not match, the lens control unit 102 determines that the lens has been replaced. Also, for example, if the lens information has not been recorded in the non-volatile memory 117 before the process of step S1301, the lens control unit 102 determines that the lens has been replaced. If the lens control unit 102 determines that the lens has been replaced, the process of step S1302 is executed. If the lens control unit 102 determines that the lens has not been replaced, the process of step S1305 is executed.

[0157] In step S1302, the noise parameter update unit 216 changes the values ​​of the noise parameters recorded in the noise parameter recording unit 205 to their initial values.

[0158] In step S1303, the counter 2163 resets the number of times the noise parameters have been compared. For example, the counter 2163 sets the number of times the noise parameters have been compared to zero.

[0159] In step S1304, the control unit 111 turns on the recording path to the noise parameter update unit 216. For example, the control unit 111 controls the voice input unit 104 to start voice processing. Note that in this step, the noise parameter selection unit 206 also starts processing.

[0160] In step S1305, the control unit 111 determines whether the number of times the noise parameters have been compared is equal to or less than a predetermined number. For example, the control unit 111 determines whether the number of times the noise parameters have been compared is equal to or less than four. If it is determined that the number of times the noise parameters have been compared is equal to or less than the predetermined number, the process of step S1306 is executed. If it is determined that the number of times the noise parameters have been compared is greater than the predetermined number, the process of step S1312 is executed.

[0161] In step S1306, the control unit 111 determines whether the optical lens 300 has started to drive. For example, the control unit 111 determines whether the level of the lens control signal output from the lens control unit 102 is High or Low. If the level of the lens control signal is High, the control unit 111 determines that the optical lens has started to drive. If the level of the lens control signal is Low, the control unit 111 determines that the optical lens is not driven. If it is determined that the optical lens 300 has started to drive, the process of step S1307 is executed. If it is determined that the optical lens 300 is not driven, the process of step S1310 is executed.

[0162] In step S1307, the noise parameter update unit 216 generates the noise parameters PLx. Note that in this step, every time noise occurs, the noise parameter update unit 216 generates noise parameters for that noise.

[0163] In step S1308, the control unit 111 determines whether the optical lens 300 has finished driving. For example, the control unit 111 determines whether the level of the lens control signal output from the lens control unit 102 is High or Low. If the level of the lens control signal is High, the control unit 111 determines that the optical lens is driving. If the level of the lens control signal is Low, the control unit 111 determines that the optical lens has finished driving. If it is determined that the optical lens 300 is driving, the process of step S1307 is executed. If it is determined that the optical lens 300 has finished driving, the process of step S1309 is executed.

[0164] In step S1309, the noise parameter update unit 216 increments the number of times the noise parameters have been compared by one.

[0165] In step S1310, the noise parameter update unit 216 compares the noise parameter PLx generated in step S1307 with the PLx recorded in the noise parameter recording unit 205. If it is determined in step S1307 that the noise parameter PLx is smaller, the process proceeds to step S1311. If it is determined that the PLx recorded in the noise parameter recording unit 205 is smaller, the process proceeds to step S1313.

[0166] In step S1311, the noise parameter update unit 216 updates PLx. For example, the noise parameter update unit 216 overwrites and records the noise parameters PLx generated in step S1307 onto the PLx recorded in the noise parameter recording unit 205. This allows the imaging device 100 to use desirable noise parameters in recording a moving image, which will be described later. Furthermore, the imaging device 100 can effectively reduce noise by using desirable noise parameters. Note that the noise parameter recording unit 205 holds the initial values ​​of the noise parameters even when the processing of this step is executed.

[0167] So far, we have described the case where the number of noise parameter comparisons is equal to or less than the predetermined number. In the timing chart of Fig. 14, this corresponds to the period from time t1 to t8. Next, we will describe the process when it is determined in the process of step S1305 that the number of noise parameter comparisons is greater than the predetermined number. In the timing chart of Fig. 14, this corresponds to the period after time t8.

[0168] In step S1312, the control unit 111 turns off the recording path to the noise parameter update unit 216. For example, the control unit 111 controls the voice input unit 104 to stop voice processing. Note that in this step, the noise parameter selection unit 206 also stops processing.

[0169] In step S1313, control unit 111 determines whether or not to start moving image recording. For example, control unit 111 determines to start moving image recording in response to pressing of release switch 61. Conversely, control unit 111 determines not to start moving image recording when release switch 61 is not pressed. If control unit 111 determines to start moving image recording, the process of step S1314 is executed. If control unit 111 determines not to start moving image recording, the process of step S1300 is executed.

[0170] In step S1314, the control unit 111 turns on the recording path to the audio input unit 104. For example, the control unit 111 controls the audio input unit 104 to start audio processing.

[0171] In step S1315, control unit 111 records moving images. For example, as described above, control unit 111 records moving image data with audio on recording medium 110. In the audio processing during this moving image recording, the noise parameters recorded in noise parameter recording unit 205 are used.

[0172] In step S1316, control unit 111 determines whether to end moving image recording. For example, control unit 111 determines to end moving image recording in response to pressing of release switch 61. Conversely, control unit 111 determines not to end moving image recording when release switch 61 is not pressed. If control unit 111 determines to end moving image recording, processing of step S1317 is executed. If control unit 111 determines not to end moving image recording, processing of step S1315 is executed.

[0173] In step S1317, the control unit 111 turns off the recording path to the noise parameter update unit 216. For example, the control unit 111 controls the audio input unit 104 to stop audio processing.

[0174] The noise parameter update process of the imaging device 100 has been described above. In this way, new noise parameters are generated in accordance with the driving of the lens before the start of the moving image recording process. Then, if the newly generated noise parameters can reduce noise more appropriately than the already generated noise parameters, the recorded noise parameters are updated using the newly generated noise parameters. By using the updated noise parameters to reduce noise contained in the audio during the recording of the moving image, the imaging device 100 can record moving image data with audio of good sound quality.

[0175] Note that the noise parameter recording unit 205 holds the updated noise parameters even after the power supply of the imaging device 100 is turned off. As a result, the imaging device 100 can record audio-accompanying video data using the updated noise parameters the next time the imaging device 100 is turned on, provided that the lens has not been replaced.

[0176] In this embodiment, the noise parameter update unit 216 does not execute the process of updating the noise parameters while recording a moving image. The imaging device 100 can record audio while maintaining the sound quality by keeping the noise parameters constant while recording a moving image.

[0177] In this embodiment, the noise parameter update unit 216 increases the number of times the noise parameters are compared in a step before the noise parameters are compared, but the noise parameter update unit 216 may increase the number of times the noise parameters are compared after the noise parameters are compared.

[0178] The noise parameter update unit 216 may determine whether to update the noise parameters based on the inclination of the imaging device 100 acquired by the information acquisition unit 103. For example, if the noise parameters are assumed to be applied when the imaging device 100 is horizontal, the noise parameter update unit 216 may not be able to correctly calculate the noise parameters when the imaging device 100 is facing upward. Therefore, in this case, when the noise parameter update unit 216 determines that the imaging device 100 is horizontal, it executes an update process to update the noise parameters. Conversely, when the noise parameter update unit 216 determines that the imaging device 100 is not horizontal, it does not execute an update process to update the noise parameters. For example, the range of inclination in which the imaging device 100 can be considered to be horizontal is a range in which the inclination of the imaging device 100 is within ±30 degrees with respect to the horizontal direction.

[0179] In addition, when the optical lens 300 is a zoom lens, the noise parameter update unit 216 may determine whether or not to update the noise parameters based on the zoom magnification of the optical lens 300 acquired by the lens control unit 102. For example, when it is assumed that the noise parameters are applied when the optical lens 300 is at the wide end, the noise parameter update unit 216 may not be able to correctly calculate the noise parameters when the optical lens 300 is at the telephoto end. Therefore, in this case, when the noise parameter update unit 216 determines that the optical lens 300 is at the wide end, it executes an update process to update the noise parameters. Conversely, when the noise parameter update unit 216 determines that the optical lens 300 is not at the wide end, it does not execute an update process to update the noise parameters. In addition, for example, the zoom magnification range in which the optical lens 300 can be considered to be at the wide end is a range of 1 to 2 times the zoom magnification of the optical lens 300.

[0180] The noise parameter recording unit 205 may record the noise parameters for each lens. This is to reduce noise in response to the fact that noise picked up by the microphone of the audio input unit 104 differs for each lens. In this case, the imaging device 100 executes an update process for updating the noise parameters for each lens.

[0181] [Second Example] In the second embodiment, a noise parameter update process different from that in the first embodiment will be described. Here, the configuration of the imaging device 100 is similar to that in the first embodiment. Note that, in the present embodiment, the noise parameter update process of PLx will be described, but the noise parameter update process of PRx is similar to the process of updating the noise parameters of PLx.

[0182] Now, the noise parameter update process of the image capture device 100 will be described with reference to the flowchart of Fig. 15 and the timing chart of Fig. 16. The process of the image capture device 100 is started when, for example, the user operates the power switch 72 to turn on the power.

[0183] The processes in steps S1500 to S1510 are similar to the processes in steps S1300 to S1310 in FIG. 13, respectively, and therefore will not be described.

[0184] In step S1511, the noise parameter update unit 216 overwrites and records the noise parameters PLx generated in step S1307 onto the PLx recorded in the noise parameter recording unit 205. After the process of step S1510, the process of step S1511 is executed.

[0185] In step S1512, the control unit 111 turns off the recording path to the noise parameter update unit 216. For example, the control unit 111 controls the audio input unit 104 to stop audio processing.

[0186] The processes in steps S1513 to S1515 are similar to the processes in steps S1313 to S1315 in FIG. 13, respectively, and therefore will not be described.

[0187] In step S1516, the control unit 111 determines whether the optical lens 300 has been driven. For example, the control unit 111 determines whether the level of the lens control signal output from the lens control unit 102 is High or Low. If the level of the lens control signal is High, the control unit 111 determines that the optical lens has been driven. If the level of the lens control signal is Low, the control unit 111 determines that the optical lens has not been driven. If it is determined that the optical lens 300 has been driven, the process of step S1517 is executed. If it is determined that the optical lens 300 has not been driven, the process of step S1519 is executed.

[0188] In step S1517, the noise parameter update unit 216 generates the noise parameter PLx.

[0189] In step S1518, the noise parameter update unit 216 increments the number of times the noise parameters have been compared by one.

[0190] In step S1519, the noise parameter update unit 216 compares the noise parameter PLx generated in step S1517 with the PLx recorded in the noise parameter recording unit 205. If it is determined in step S1517 that the noise parameter PLx is smaller, the process proceeds to step S1520. If it is determined that the PLx recorded in the noise parameter recording unit 205 is smaller, the process proceeds to step S1521.

[0191] In step S1520, the noise parameter update unit 216 updates PLx. For example, the noise parameter update unit 216 overwrites and records the noise parameters PLx generated in step S1517 onto the PLx recorded in the noise parameter recording unit 205. Note that the noise parameter recording unit 205 holds the initial values ​​of the noise parameters even when the processing of this step is executed.

[0192] In step S1521, control unit 111 determines whether to end moving image recording. For example, control unit 111 determines to end moving image recording in response to pressing of release switch 61. Conversely, control unit 111 determines not to end moving image recording when release switch 61 is not pressed. If control unit 111 determines to end moving image recording, processing of step S1522 is executed. If control unit 111 determines not to end moving image recording, processing of step S1515 is executed.

[0193] In step S1521, the control unit 111 turns off the recording path to the noise parameter update unit 216. For example, the control unit 111 controls the audio input unit 104 to stop audio processing.

[0194] Thus, in this embodiment, the image capture device 100 turns off the sound recording path in response to updating the noise parameters before recording a moving image. This allows the image capture device 100 to turn off the sound recording path as early as by driving the lens once, thereby reducing power consumption in the audio input unit 104. Furthermore, in this embodiment, the image capture device 100 updates the noise parameters even while recording a moving image. This makes it possible to record audio with efficient noise reduction even while recording a moving image.

[0195] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0196] The present invention is not limited to the above-described embodiment, and the components can be modified and embodied in the implementation stage without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining the multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

Claims

1. a first microphone for acquiring environmental sounds; a second microphone for acquiring sound from the noise source; a first transforming means for Fourier transforming the audio signal from the first microphone to generate a first audio signal in the frequency domain; a second transforming means for Fourier transforming the audio signal from the second microphone to generate a second audio signal in the frequency domain; a generating means for calculating the second audio signal and a first parameter related to the noise of the noise source to generate noise data; a subtraction means for subtracting the noise data from the first audio signal; a third transform means for performing an inverse Fourier transform on the audio signal from the subtraction means; an update means for generating new parameters related to the noise of the noise source using the first voice signal and the second voice signal, and updating the first parameters using the newly generated parameters; having when the first parameter is updated by the update means, the generation means calculates the first parameter updated by the update means and the second sound signal to generate the noise data; The voice processing apparatus according to claim 1, wherein the updating means does not update the first parameter when the value of the newly generated parameter is not smaller than the value of the first parameter.

2. the first parameter having values ​​for a plurality of frequency points of a frequency spectrum of the second audio signal; 2. The audio processing device according to claim 1, wherein the update means updates the first parameters for each frequency point based on the newly generated parameters.

3. A driving means as the noise source is provided, 3. The audio processing device according to claim 1, wherein the update means generates parameters related to noise from the noise source by using the first audio signal and the second audio signal while the drive means is driving.

4. The imaging means further includes:

4. The audio processing device according to claim 3, wherein the driving means is driven when the imaging means captures an image.

5. 5. The audio processing device according to claim 4, wherein the update means executes the update after the audio processing device is powered on and before recording of the image captured by the imaging means is started.

6. 6. The audio processing device according to claim 3, wherein the update means changes the first parameter updated by the update means to an initial value when the drive means is replaced.

7. Further comprising a recording means, 7. The audio processing device according to claim 1, wherein the first parameter updated by the update means is retained by the recording means even when the power supply of the audio processing device is turned off.

8. 8. The audio processing device according to claim 1, wherein the first parameter is a ratio of amplitudes of the first audio signal and the second audio signal.

9. a first microphone for acquiring environmental sounds; and a second microphone for acquiring a sound from a noise source, performing a Fourier transform on the audio signal from the first microphone to generate a first audio signal in the frequency domain; performing a Fourier transform on the audio signal from the second microphone to generate a second audio signal in the frequency domain; a generating step of calculating the second audio signal and a first parameter related to the noise of the noise source to generate noise data; a subtraction step of subtracting the noise data from the first audio signal; a step of performing an inverse Fourier transform on the audio signal output by the subtraction step; and, an updating step of generating new parameters related to the noise of the noise source using the first voice signal and the second voice signal, and updating the first parameters using the newly generated parameters; In the generating step, when the first parameter is updated in the updating step, the first parameter updated in the updating step and the second sound signal are calculated to generate the noise data; A control method according to claim 1, wherein said updating step does not update said first parameter if the value of said newly generated parameter is not smaller than the value of said first parameter.

10. A computer-readable program for causing a computer to function as each of the means of the voice processing device according to any one of claims 1 to 8.

Images