Audio processing device, control method, and program

Abstract

To effectively reduce noise.SOLUTION: A sound processor comprises: mounting means on which a lens having a noise source is mounted; a first microphone for acquiring environmental sounds; a second microphone for acquiring sounds from the noise source; first transform means for performing Fourier transform on sound signals from the first microphone to generate first sound signals; second transform means for performing Fourier transform on sound signals from the second microphone to generate second sound signals; generation means for generating noise data using the second sound signals and a parameter related to the noise of the noise source; subtraction means for subtracting the noise data from the first sound signals; and third transform means for performing inverse Fourier transform on the sound signals from the subtraction means. The generation means uses the parameter in accordance with a kind of the lens mounted on the mounting means as a parameter.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to an audio processing device capable of reducing noise contained in audio data. [Background technology]

[0002] A digital camera, which is an example of an audio processing device, can record surrounding audio when recording video data. Digital cameras also have an autofocus function that drives an optical lens to focus on a subject while recording video data. Digital cameras also have a function to drive an optical lens to zoom while recording a video.

[0003] In this way, when an optical lens is driven while a video is being recorded, the driving sound of the optical lens may be included as noise in the audio recorded with the video. Conventionally, therefore, when a digital camera picks up a sliding sound generated when the optical lens is driven as noise, the noise can be reduced and the surrounding audio can be recorded. Patent Document 1 discloses a digital camera that reduces noise by a spectral subtraction method. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-205527 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, since the digital camera creates a noise pattern from noise collected by a microphone that records surrounding sounds, there is a possibility that an accurate noise pattern cannot be obtained from the sliding sound generated inside the housing of the optical lens. In this case, there is a risk that the digital camera cannot effectively reduce the noise contained in the collected sounds.

[0006] Therefore, an object of the present invention is to effectively reduce noise. [Means for solving the problem]

[0007] The audio processing device has a mounting means for mounting a lens having a noise source, a first microphone for acquiring environmental sound, a second microphone for acquiring sound from the noise source, a first conversion means for Fourier transforming an audio signal from the first microphone to generate a first audio signal in a frequency domain, a second conversion means for Fourier transforming an audio signal from the second microphone to generate a second audio signal in the frequency domain, a generation means for calculating the second audio signal and a parameter related to the noise of the noise source corresponding to the type of the lens mounted on the mounting means to generate noise data, a subtraction means for subtracting the noise data from the first audio signal, and a third conversion means for performing an inverse Fourier transform on the audio signal output from the subtraction means, wherein the parameters have coefficients for each of a plurality of frequency points that constitute a frequency spectrum, and the generation means generates the noise data by multiplying a frequency spectrum of the plurality of frequency points of the second audio signal by the coefficient of the parameter, and the subtraction means subtracts a frequency spectrum of a plurality of frequency points of the noise data from the frequency spectrum of the plurality of frequency points of the first audio signal. Effect of the Invention

[0008] The audio processing device of the present invention can effectively reduce noise. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a perspective view of an imaging device according to a first embodiment. [Diagram 2] 1 is a block diagram showing a configuration of an imaging apparatus according to a first embodiment. [Diagram 3] 2 is a block diagram showing a configuration of a voice input unit of the imaging device according to the first embodiment. FIG. [Figure 4] 3 is a diagram showing an arrangement of microphones in a voice input unit of the imaging device according to the first embodiment. FIG. [Diagram 5] FIG. 4 is a diagram showing noise parameters in the first embodiment. [Figure 6] 13A and 13B are diagrams illustrating a frequency spectrum of an audio and a frequency spectrum of a noise parameter in a case where a drive sound occurs in a situation where it is considered that there is no environmental sound in the first embodiment. [Figure 7] 11 is a diagram showing a frequency spectrum of audio when a drive sound occurs in a situation where an environmental sound is present in the first embodiment; FIG. [Figure 8] FIG. 4 is a block diagram showing a configuration of a noise parameter selection unit in the first embodiment. [Figure 9] 4 is a timing chart relating to the audio noise reduction process in the first embodiment. [Figure 10] FIG. 11 is a diagram showing noise parameters for each lens in the first embodiment. [Figure 11] 11 is a diagram showing a frequency spectrum of audio when a drive sound occurs in a situation where an environmental sound is present in the first embodiment; FIG. [Figure 12] 4 is a flowchart showing a process of the imaging device 100 in the first embodiment. [Figure 13] FIG. 11 is a diagram showing attitude information of an imaging device in the second embodiment. [Figure 14] 13A and 13B are diagrams illustrating changes in frequency spectrum due to changes in the attitude of the imaging device in the second embodiment. [Figure 15] 13 shows correction noise parameters for each piece of posture information of the image capturing device in the second embodiment. [Figure 16] 11 is a comparison of the length of the lens barrel at different zoom magnifications of the imaging device in the second embodiment. [Figure 17] 13A and 13B are diagrams illustrating changes in frequency spectrum due to differences in the length of the lens barrel of the imaging device in the second embodiment. [Figure 18]13 shows correction noise parameters for each zoom magnification of the image pickup apparatus in the second embodiment. [Figure 19] 13 is a flowchart showing a process of the imaging device 100 in the second embodiment. [Figure 20] FIG. 13 is a block diagram showing a configuration of a voice input unit of an imaging device according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] [First Example] <External view of imaging device 100> 1(a) and (b) show an example of an external view of an image capture device 100 as an example of an audio processing device to which the present invention can be applied. Fig. 1(a) is an example of a front perspective view of the image capture device 100. Fig. 1(b) is an example of a rear perspective view of the image capture device 100. In Fig. 1, an optical lens (not shown) is attached to a lens mount 301.

[0012] The display unit 107 displays image data, text information, and the like. The display unit 107 is provided on the rear surface of the imaging device 100. The outside-finder display unit 43 is a display unit provided on the top surface of the imaging device 100. The outside-finder display unit 43 displays the settings of the imaging device 100, such as the shutter speed and aperture value. The eyepiece viewfinder 16 is a peer-type viewfinder. The user can check the focus and composition of the optical image of the subject by observing the focusing screen in the eyepiece viewfinder 16.

[0013] The release switch 61 is an operation member that allows the user to issue a shooting instruction. The mode changeover switch 60 is an operation member that allows the user to switch between various modes. The main electronic dial 71 is a rotating operation member. The user can change the settings of the imaging device 100, such as the shutter speed and aperture value, by turning this main electronic dial 71. The release switch 61, the mode changeover switch 60, and the main electronic dial 71 are included in the operation unit 112.

[0014] The power switch 72 is an operating member that switches the power of the imaging device 100 on and off. The sub electronic dial 73 is a rotary operating member. The user can use the sub electronic dial 73 to move the selection frame displayed on the display unit 107 and to advance images in playback mode. The cross key 74 is a cross key (four-way key) whose up, down, left, and right portions can be pressed. The imaging device 100 executes processing according to the portion (direction) of the cross key 74 that is pressed. The power switch 72, the sub electronic dial 73, and the cross key 74 are included in the operation unit 112.

[0015] The SET button 75 is a push button. The SET button 75 is mainly used by the user to confirm a selection item displayed on the display unit 107. The LV button 76 is a button used to switch live view (hereinafter, LV) on and off. In the video recording mode, the LV button 76 is used to instruct the start and stop of video shooting (recording). The enlargement button 77 is a push button for turning the enlargement mode on and off in the live view display in the shooting mode, and for changing the magnification ratio during the enlargement mode. The SET button 75, the LV button 76, and the enlargement button 77 are included in the operation unit 112.

[0016] In the playback mode, the enlargement button 77 functions as a button for increasing the magnification rate of image data displayed on the display unit 107. The reduction button 78 is a button for decreasing the magnification rate of image data enlarged and displayed on the display unit 107. The playback button 79 is an operation button for switching between the shooting mode and the playback mode. When the playback button 79 is pressed while the imaging device 100 is in the shooting mode, the imaging device 100 transitions to the playback mode, and the image data recorded on the recording medium 110 is displayed on the display unit 107. The reduction button 78 and the playback button 79 are included in the operation unit 112.

[0017] The quick return mirror 12 (hereinafter, mirror 12) is a mirror for switching the light beam incident from an optical lens attached to the imaging device 100 so that it is incident on either the eyepiece finder 16 side or the imaging unit 101 side. The mirror 12 is raised and lowered by the control unit 111 controlling an actuator (not shown) during exposure, live view shooting, and video shooting. The mirror 12 is normally arranged to allow the light beam to be incident on the eyepiece finder 16. When shooting or live view display is performed, the mirror 12 flips up (mirror up) so that the light beam is incident on the imaging unit 101. The center of the mirror 12 is a half mirror. A part of the light beam that passes through the center of the mirror 12 is incident on a focus detection unit (not shown) for focus detection.

[0018] The communication terminal 10 is a communication terminal for communication between the imaging device 100 and the optical lens 300 attached to the imaging device 100. The terminal cover 40 is a cover for protecting a connector (not shown) such as a connection cable that connects an external device to the imaging device 100. The lid 41 is a lid for a slot that stores the recording medium 110. The lens mount 301 is an attachment portion to which the optical lens 300 (not shown) can be attached.

[0019] The L microphone 201a and the R microphone 201b are microphones for picking up the user's voice, etc. When viewed from the rear of the imaging device 100, the L microphone 201a is disposed on the left side and the R microphone 201b is disposed on the right side.

[0020] <Configuration of imaging device 100> FIG. 2 is a block diagram showing an example of the configuration of the imaging device 100 in this embodiment.

[0021] The optical lens 300 is a lens unit that is detachable from the imaging device 100. For example, the optical lens 300 is a zoom lens or a varifocal lens. The optical lens 300 has an optical lens, a motor for driving the optical lens, and a communication unit that communicates with a lens control unit 102 of the imaging device 100, which will be described later. The optical lens 300 can focus and zoom on a subject and correct camera shake by moving the optical lens using the motor based on a control signal received by the communication unit.

[0022] The imaging unit 101 has an imaging element for converting an optical image of a subject formed on an imaging surface via an optical lens 300 into an electrical signal, and an image processing unit for generating and outputting image data or video data from the electrical signal generated by the imaging element. The imaging element is, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). In this embodiment, a series of processes for generating image data including still image data and video data in the imaging unit 101 and outputting the image data from the imaging unit 101 is called "shooting". In the imaging device 100, the image data is recorded on a recording medium 110 (described later) in accordance with the DCF (Design rule for Camera File system) standard.

[0023] The lens control unit 102 transmits a control signal to the optical lens 300 via the communication terminal 10 based on data output from the imaging unit 101 and a control signal output from a control unit 111 (described later) to control the optical lens 300. The lens control unit 102 also receives lens information from the optical lens 300 attached to the imaging device 100. The lens information includes, for example, the type of lens, the lens model number, and the type of noise source.

[0024] The information acquisition unit 103 detects the inclination of the imaging device 100 and the temperature inside the housing of the imaging device 100. For example, the information acquisition unit 103 detects the inclination of the imaging device 100 by an acceleration sensor or a gyro sensor. Also, for example, the information acquisition unit 103 detects the temperature inside the housing of the imaging device 100 by a temperature sensor.

[0025] The audio input unit 104 generates audio data from audio acquired by a microphone. The audio input unit 104 acquires audio around the imaging device 100 by the microphone, performs analog-to-digital conversion (A / D conversion) on the acquired audio, and various audio processes to generate audio data. In this embodiment, the audio input unit 104 has a microphone. A detailed configuration example of the audio input unit 104 will be described later.

[0026] The volatile memory 105 temporarily records image data generated by the imaging unit 101 and audio data generated by the audio input unit 104. The volatile memory 105 is also used as a temporary recording area for image data displayed on the display unit 107, a working area for the control unit 111, etc.

[0027] The display control unit 106 controls the display unit 107 to display image data output from the imaging unit 101, characters for interactive operation, a menu screen, and the like. Furthermore, the display control unit 106 can control the display unit 107 to sequentially display digital data output from the imaging unit 101 during still image shooting and video shooting, thereby allowing the display unit 107 to function as an electronic viewfinder. For example, the display unit 107 is a liquid crystal display or an organic EL display. Furthermore, the display control unit 106 can also control the image data and video data output from the imaging unit 101, characters for interactive operation, a menu screen, and the like to be displayed on an external display via an external output unit 115, which will be described later.

[0028] The encoding processing unit 108 can encode the image data and audio data temporarily recorded in the volatile memory 105. For example, the encoding processing unit 108 can generate moving image data by encoding and data-compressing the image data according to the JPEG standard or RAW image format. For example, the encoding processing unit 108 can generate moving image data by encoding and data-compressing the moving image data according to the MPEG2 standard or the H.264 / MPEG4-AVC standard. Also, for example, the encoding processing unit 108 can generate audio data by encoding and data-compressing the audio data according to the AC3AAC standard, the ATRAC standard, or the ADPCM method. Also, the encoding processing unit 108 may encode the audio data according to, for example, the linear PCM method so as not to compress the data.

[0029] The recording control unit 109 can record data on the recording medium 110 and read data from the recording medium 110. For example, the recording control unit 109 can record still image data, video data, and audio data generated by the encoding processing unit 108 on the recording medium 110 and read data from the recording medium 110. The recording medium 110 is, for example, an SD card, a CF card, an XQD memory card, an HDD (magnetic disk), an optical disk, or a semiconductor memory. The recording medium 110 may be configured to be detachable from the imaging device 100, or may be built into the imaging device 100. That is, the recording control unit 109 only needs to have at least a means for accessing the recording medium 110.

[0030] The control unit 111 controls each component of the imaging device 100 via a data bus 116 in accordance with an input signal and a program described later. The control unit 111 has a CPU, a ROM, and a RAM for executing various controls. Note that, instead of the control unit 111 controlling the entire imaging device 100, a plurality of hardware may share the control of the entire imaging device. The ROM of the control unit 111 stores a program for controlling each component. The RAM of the control unit 111 is a volatile memory used for arithmetic processing and the like.

[0031] The operation unit 112 is a user interface for receiving instructions from a user for the imaging device 100. The operation unit 112 has, for example, a power switch 72 for turning the power of the imaging device 100 on or off, a release switch 61 for issuing an instruction to shoot, a playback button for issuing an instruction to play back image data or video data, a mode change switch 60, and the like.

[0032] The operation unit 112 outputs a control signal to the control unit 111 in response to a user's operation. The operation unit 112 may also include a touch panel formed on the display unit 107. The release switch 61 has SW1 and SW2. When the release switch 61 is pressed halfway, SW1 is turned on. This accepts a preparation instruction for performing a preparation operation for imaging, such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing. When the release switch 61 is pressed fully, SW2 is turned on. This user operation accepts an imaging instruction for performing an imaging operation. The operation unit 112 also includes an operation member (e.g., a button) that can adjust the volume of audio data reproduced from a speaker 114, which will be described later.

[0033] The audio output unit 113 can output audio data to the speaker 114 and the external output unit 115. The audio data input to the audio output unit 113 includes audio data read from the recording medium 110 by the recording control unit 109, audio data output from the non-volatile memory 117, and audio data output from the encoding processing unit. The speaker 114 is an electro-acoustic transducer capable of reproducing audio data.

[0034] The external output unit 115 can output image data, video data, audio data, etc. to an external device. The external output unit 115 is composed of, for example, a video terminal, a microphone terminal, a headphone terminal, etc.

[0035] The data bus 116 is a data bus for transmitting various data such as audio data, video data, and image data, and various control signals to each block of the imaging device 100 .

[0036] The nonvolatile memory 117 is a nonvolatile memory that stores programs, etc., which are executed by the control unit 111 and will be described later. Furthermore, sound data is recorded in the nonvolatile memory 117. This sound data is, for example, sound data of electronic sounds such as a focusing sound output when a subject is focused on, an electronic shutter sound output when an instruction to shoot is given, and an operation sound output when the imaging device 100 is operated.

[0037] <Operation of the imaging device 100> The operation of the imaging device 100 of this embodiment will now be described.

[0038] The imaging device 100 of this embodiment supplies power to each component of the imaging device from a power supply (not shown) in response to a user turning on the power by operating the power switch 72. For example, the power supply is a battery such as a lithium ion battery or an alkaline manganese dry battery.

[0039] The control unit 111 determines whether the camera will operate in a shooting mode or a playback mode, for example, based on the state of the mode changeover switch 60 in response to the supply of power. In the moving image recording mode, the control unit 111 records the moving image data output from the imaging unit 101 and the audio data output from the audio input unit 104 as one piece of moving image data with audio. In the playback mode, the control unit 111 reads out the image data or moving image data recorded on the recording medium 110 by the recording control unit 109, and controls the display unit 107 to display the image data or moving image data.

[0040] First, the moving image recording mode will be described. In the moving image recording mode, the control unit 111 first transmits a control signal to each component of the imaging device 100 to transition the imaging device 100 to a shooting standby state. For example, the control unit 111 controls the imaging unit 101 and the audio input unit 104 to perform the following operations.

[0041] The imaging unit 101 converts an optical image of a subject formed on an imaging surface via the optical lens 300 into an electrical signal, and generates video data from the electrical signal generated by the imaging element. The imaging unit 101 then transmits the video data to a display control unit 106, which displays it on a display unit 107. The user can prepare for shooting while viewing the video data displayed on the display unit 107.

[0042] The audio input unit 104 performs A / D conversion on analog audio signals input from the multiple microphones, respectively, to generate multiple digital audio signals. The audio input unit 104 then generates audio data for multiple channels from the multiple digital audio signals. The audio input unit 104 transmits the generated audio data to the audio output unit 113, which reproduces the audio data from the speaker 114. While listening to the audio data reproduced from the speaker 114, the user can adjust the volume of the audio data recorded in the audio-accompanied video data using the operation unit 112.

[0043] Next, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal to start shooting to each component of the imaging device 100. For example, the control unit 111 controls the imaging unit 101, the audio input unit 104, the encoding processing unit 108, and the recording control unit 109 to perform the following operations.

[0044] The imaging unit 101 converts an optical image of a subject formed on an imaging surface via an optical lens 300 into an electrical signal, and generates video data from the electrical signal generated by the imaging element. The imaging unit 101 then transmits the video data to a display control unit 106, which displays the video data on a display unit 107. The imaging unit 101 also transmits the generated video data to a volatile memory 105.

[0045] The audio input unit 104 performs A / D conversion on the analog audio signals input from the multiple microphones to generate multiple digital audio signals. The audio input unit 104 then generates multi-channel audio data from the multiple digital audio signals. The audio input unit 104 then transmits the generated audio data to the volatile memory 105.

[0046] The encoding processing unit 108 reads out the moving image data and audio data temporarily recorded in the volatile memory 105 and encodes each of them. The control unit 111 generates a data stream from the moving image data and audio data encoded by the encoding processing unit 108, and outputs it to the recording control unit 109. The recording control unit 109 records the input data stream as moving image data with audio on the recording medium 110 in accordance with a file system such as UDF or FAT.

[0047] Each component of the image capturing apparatus 100 continues to perform the above operations during moving image capturing.

[0048] Then, in response to the user pressing the LV button 76, the control unit 111 transmits an instruction signal to end shooting to each component of the imaging device 100. For example, the control unit 111 controls the imaging unit 101, the audio input unit 104, the encoding processing unit 108, and the recording control unit 109 to perform the following operations.

[0049] The image capturing unit 101 stops generating moving image data, and the audio input unit 104 stops generating audio data.

[0050] The encoding processing unit 108 reads and encodes the remaining video data and audio data recorded in the volatile memory 105. The control unit 111 generates a data stream from the video data and audio data encoded by the encoding processing unit 108, and outputs the data stream to the recording control unit 109.

[0051] The recording control unit 109 records the data stream as a file of moving image data with audio on the recording medium 110 in accordance with a file system such as UDF or FAT. Then, when the input of the data stream stops, the recording control unit 109 completes the moving image data with audio. Upon completion of the moving image data with audio, the recording operation of the imaging device 100 stops.

[0052] In response to the stop of the recording operation, the control unit 111 transmits a control signal to each component of the imaging device 100 to transition to a shooting standby state. In response to this, the control unit 111 controls the imaging device 100 to return to the shooting standby state.

[0053] Next, the playback mode will be described. In the playback mode, the control unit 111 transmits a control signal to each component of the image pickup device 100 to transition to a playback state. For example, the control unit 111 controls the encoding processing unit 108, the recording control unit 109, the display control unit 106, and the audio output unit 113 to perform the following operations.

[0054] The recording control unit 109 reads out the moving image data with audio recorded on the recording medium 110 and transmits the read moving image data with audio to the encoding processing unit 108 .

[0055] The encoding processing unit 108 decodes the audio-accompanied video data into image data and audio data. The encoding processing unit 108 transmits the decoded video data to the display control unit 106 and the decoded audio data to the audio output unit 113.

[0056] The display control unit 106 displays the decoded image data on the display unit 107. The audio output unit 113 reproduces the decoded audio data through the speaker 114.

[0057] As described above, the imaging device 100 of this embodiment is capable of recording and reproducing image data and audio data.

[0058] In this embodiment, the audio input unit 104 performs audio processing such as adjustment of the level of an audio signal input from a microphone. In this embodiment, the audio input unit 104 performs this audio processing in response to the start of moving image recording. Note that this audio processing may be performed after the power of the imaging device 100 is turned on. Also, this audio processing may be performed in response to the selection of a shooting mode. Also, this audio processing may be performed in response to the selection of a mode related to audio recording, such as a moving image recording mode or a voice memo function. Also, this audio processing may be performed in response to the start of recording of an audio signal.

[0059] <Configuration of voice input unit 104> FIG. 3 is a block diagram showing an example of a detailed configuration of the voice input unit 104 in this embodiment.

[0060] In this embodiment, the audio input unit 104 has three microphones: an L microphone 201a, an R microphone 201b, and a noise microphone 201c. The L microphone 201a and the R microphone 201b are each an example of a first microphone. In this embodiment, the image capture device 100 collects environmental sounds using the L microphone 201a and the R microphone 201b, and records the audio signals input from the L microphone 201a and the R microphone 201b in a stereo format. For example, environmental sounds are sounds generated outside the housing of the image capture device 100 and outside the housing of the optical lens 300, such as the user's voice, animal cries, the sound of rain, and music.

[0061] Moreover, the noise microphone 201c is an example of a second microphone. The noise microphone 201c is a microphone for collecting noise such as a driving sound from a predetermined noise source (noise source) generated in the housing of the imaging device 100 and the housing of the optical lens 300. The noise source is, for example, a motor such as an ultrasonic motor (USM) and a stepper motor (STM). The noise is, for example, a vibration sound generated by driving a motor such as a USM and an STM. For example, the motor is driven in an AF process for focusing on a subject. The imaging device 100 acquires noise such as a driving sound generated in the housing of the imaging device 100 and the housing of the optical lens 300 by the noise microphone 201c, and generates a noise parameter to be described later using the audio data of the acquired noise. In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are non-directional microphones. An example of the arrangement of the L microphone 201a, the R microphone 201b, and the noise microphone 201c in this embodiment will be described later with reference to FIG.

[0062] The L microphone 201a, R microphone 201b, and noise microphone 201c generate analog audio signals from the respective picked up sounds and input them to the A / D conversion unit 202. Here, the audio signal input from the L microphone 201a is referred to as Lch, the audio signal input from the R microphone 201b as Rch, and the audio signal input from the noise microphone 201c as Nch.

[0063] The A / D conversion unit 202 converts analog audio signals input from the L microphone 201a, the R microphone 201b, and the noise microphone 201c into digital audio signals. The A / D conversion unit 202 outputs the converted digital audio signals to the FFT unit 203. In this embodiment, the A / D conversion unit 202 converts the analog audio signals into digital audio signals by performing sampling processing with a sampling frequency of 48 kHz and a bit depth of 16 bits.

[0064] The FFT unit 203 performs fast Fourier transform processing on the time domain digital audio signal input from the A / D conversion unit 202 to convert it into a frequency domain digital audio signal. In this embodiment, the frequency domain digital audio signal has a frequency spectrum of 1024 points in a frequency band from 0 Hz to 48 kHz. Also, the frequency domain digital audio signal has a frequency spectrum of 513 points in a frequency band from 0 Hz to 24 kHz, which is the Nyquist frequency. In this embodiment, the imaging device 100 performs noise reduction processing using the 513-point frequency spectrum from 0 Hz to 24 kHz of the audio data output from the FFT unit 203.

[0065] Here, the frequency spectrum of the Lch subjected to the fast Fourier transform is represented by 513-point array data of Lch_Before[0] to Lch_Before

[0512] . When these array data are collectively referred to, they are written as Lch_Before. Also, the frequency spectrum of the Rch subjected to the fast Fourier transform is represented by 513-point array data of Rch_Before[0] to Rch_Before

[0512] . When these array data are collectively referred to, they are written as Rch_Before. Here, in this embodiment, Lch_Before[0] is the frequency spectrum of a 0 Hz voice, and Lch_Before

[0512] is the frequency spectrum of a 24 kHz voice. Note that Lch_Before and Rch_Before are each an example of the first frequency spectrum data.

[0066] The Nch frequency spectrum after the fast Fourier transform is represented by array data of 513 points, Nch_Before[0] to Nch_Before

[0512] . When these array data are collectively referred to, they are written as Nch_Before. Note that Nch_Before is an example of second frequency spectrum data.

[0067] The noise data generating unit 204 generates data for reducing noise contained in Lch_Before and Rch_Before based on Nch_Before. In this embodiment, the noise data generating unit 204 generates array data of NL[0] to NL

[0512] for reducing noise contained in Lch_Before[0] to Lch_Before

[0512] , respectively, using noise parameters. In addition, the noise data generating unit 204 generates array data of NR[0] to NR

[0512] for reducing noise contained in Rch_Before[0] to Rch_Before

[0512] , respectively. The frequency points in the array data of NL[0] to NL

[0512] are the same as the frequency points in the array data of Lch_Before[0] to Lch_Before

[0512] . Furthermore, the frequency points in the array data of NR[0] to NR

[0512] are the same as the frequency points in the array data of Rch_Before[0] to Rch_Before

[0512] .

[0068] The sequence data NL[0] to NL

[0512] are collectively referred to as NL. The sequence data NR[0] to NR

[0512] are collectively referred to as NR. NL and NR are each an example of the third frequency spectrum data.

[0069] The noise parameter recording unit 205 records noise parameters used by the noise data generating unit 204 to generate NL and NR from Nch_Before.

[0070] In this embodiment, a plurality of types of lenses can be attached to the imaging device 100. The plurality of types of lenses are different from each other in focal length, lens configuration, motor type, etc. Therefore, the characteristics of noise generated by each lens are different. The noise parameter recording unit 205 records a plurality of noise parameters corresponding to a plurality of types of lenses. For example, the noise parameter recording unit 205 records a first noise parameter corresponding to a first type of lens and a second noise parameter corresponding to a second type of lens. When noise parameters for generating NL from Nch_Before are collectively referred to, they are written as PLx. When noise parameters for generating NR from Nch_Before are collectively referred to, they are written as PRx.

[0071] PLx and PRx have the same number of sequences as NL and NR, respectively. For example, PL1 is sequence data from PL1[0] to PL1

[0512] . The frequency points of PL1 are the same as the frequency points of Lch_Before. For example, PR1 is sequence data from PR1[0] to PR1

[0512] . The frequency points of PR1 are the same as the frequency points of Rch_Before. The noise parameters will be described later with reference to FIG. 5.

[0072] The noise parameter selection unit 206 determines the noise parameters to be used in the noise data generation unit 204 from the noise parameters recorded in the noise parameter recording unit 205. The noise parameter selection unit 206 determines the noise parameters to be used in the noise data generation unit 204 based on data such as Lch_Before, Rch_Before, Nch_Before, and lens information received from the lens control unit 102. The operation of the noise parameter selection unit 206 will be described in detail later with reference to FIG. 8.

[0073] In this embodiment, the noise parameter recording unit 205 records all the coefficients for the 513 frequency spectrum points as noise parameters. However, it is sufficient that at least the coefficients for the frequency points required for reducing noise are recorded, not the coefficients for all the 513 frequency points. For example, the noise parameter recording unit 205 may record, as noise parameters, the coefficients for the frequency spectrum of 20 Hz to 20 kHz, which are considered to be typical audible frequencies, and may not record the coefficients for other frequency spectra. Also, for example, the coefficients for the frequency spectrum whose coefficient value is zero may not be recorded in the noise parameter recording unit 205 as noise parameters.

[0074] The subtraction processing unit 207 subtracts NL and NR from Lch_Before and Rch_Before, respectively. For example, the subtraction processing unit 207 has an L subtractor 207a that subtracts NL from Lch_Before, and an R subtractor 207b that subtracts NR from Rch_Before. The L subtractor 207a subtracts NL from Lch_Before, and outputs array data of 513 points from Lch_After[0] to Lch_After

[0512] . The R subtractor 207b subtracts NR from Rch_Before, and outputs array data of 513 points from Rch_After[0] to Rch_After

[0512] . In this embodiment, the subtraction processing unit 207 performs subtraction processing by a spectral subtraction method.

[0075] The iFFT unit 208 performs inverse fast Fourier transform (inverse Fourier transform) on the frequency domain digital audio signal input from the subtraction processing unit 207 to convert it into a time domain digital audio signal.

[0076] The audio processor 209 executes audio processing on the time domain digital audio signal, such as an equalizer, an auto level controller, and stereo enhancement processing, etc. The audio processor 209 outputs the processed audio data to the volatile memory 105.

[0077] In this embodiment, the imaging device 100 has two microphones as the first microphone, but the imaging device 100 may have one microphone or three or more microphones as the first microphone. For example, when the imaging device 100 has one microphone as the first microphone in the audio input unit 104, the imaging device 100 records audio data acquired by the one microphone in a monaural format. Also, for example, when the imaging device 100 has three or more microphones as the first microphone in the audio input unit 104, the imaging device 100 records audio data acquired by the three or more microphones in a surround format.

[0078] In this embodiment, the L microphone 201a, the R microphone 201b, and the noise microphone 201c are non-directional microphones, but these microphones may be directional microphones.

[0079] <Arrangement of microphones in the voice input unit 104> Here, a description will be given of an example of the arrangement of the microphones of the voice input unit 104 of this embodiment. Fig. 4 shows an example of the arrangement of the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

[0080] 4 is an example of a cross-sectional view of a portion of the imaging device 100 to which the L microphone 201a, the R microphone 201b, and the noise microphone 201c are attached. This portion of the imaging device 100 is composed of an exterior part 302, a microphone bushing 303, and a fixed part 304.

[0081] The exterior part 302 has a hole (hereinafter referred to as a microphone hole) for inputting environmental sound to the microphone. In this embodiment, the microphone hole is formed above the L microphone 201a and the R microphone 201b. On the other hand, the noise microphone 201c is provided to acquire driving sound generated within the housing of the image capture device 100 and within the housing of the optical lens 300, and does not need to acquire environmental sound. Therefore, in this embodiment, no microphone hole is formed in the exterior part 302 above the noise microphone 201c.

[0082] Drive sounds generated within the housings of the image capture device 100 and the optical lens 300 are picked up by the L microphone 201a and the R microphone 201b through the microphone holes. When drive sounds or the like are generated within the housings of the image capture device 100 and the optical lens 300 while the ambient sound is low, the sound picked up by each microphone is mainly this drive sound. Therefore, the sound level from the noise microphone 201c is higher than the sound levels from the L microphone 201a and the R microphone 201b. That is, in this case, the relationship between the levels of the sound signals output from each microphone is as follows: Lch ≒ Rch <Nch Furthermore, when the environmental sound becomes louder, the sound level of the environmental sound from the L microphone 201a and the R microphone 201b becomes louder than the sound level of the drive sound from the microphone 201c generated by the image capture device 100 or the optical lens 300. Therefore, in this case, the relationship between the levels of the audio signals output from each microphone is as follows: Lch ≒ Rch > Nch In this embodiment, the shape of the microphone hole formed in exterior part 302 is elliptical, but it may be other shapes such as circular or rectangular. Also, the shape of the microphone hole on microphone 201a and the shape of the microphone hole on microphone 201b may be different from each other.

[0083] In this embodiment, the noise microphone 201c is disposed close to the L microphone 201a and the R microphone 201b. Furthermore, in this embodiment, the noise microphone 201c is disposed between the L microphone 201a and the R microphone 201b. As a result, the audio signal generated by the noise microphone 201c from drive sounds and the like generated within the housing of the image capture device 100 and the housing of the optical lens 300 becomes similar to the audio signal generated by the L microphone 201a and the R microphone 201b from this drive sound and the like.

[0084] The microphone bushing 303 is a member for fixing the L microphone 201a, the R microphone 201b, and the noise microphone 201c. The fixing portion 304 is a member for fixing the microphone bushing 303 to the exterior portion 302.

[0085] In this embodiment, exterior part 302 and fixed part 304 are made of a molded material such as PC material. Also, exterior part 302 and fixed part 304 may be made of a metal material such as aluminum or stainless steel. Also, in this embodiment, microphone bushing 303 is made of a rubber material such as ethylene propylene diene rubber.

[0086] <Noise parameters> FIG. 5 is an example of noise parameters recorded in the noise parameter recording unit 205. The noise parameters shown in FIG. 5 are noise parameters for noise generated from a certain lens. The noise parameters are parameters for correcting an audio signal generated by the noise microphone 201c acquiring drive sounds generated in the housing of the image capture device 100 and in the housing of the optical lens 300. As shown in FIG. 5, in this embodiment, PLx and PRx are recorded in the noise parameter recording unit 205. In this embodiment, the source of the drive sounds will be described as being in the housing of the optical lens 300. The drive sounds generated in the housing of the optical lens 300 are transmitted to the housing of the image capture device 100 via the lens mount 301 and are acquired by the L microphone 201a, the R microphone 201b, and the noise microphone 201c.

[0087] The frequency of the driving sound varies depending on the type of the driving sound. Therefore, in this embodiment, the imaging device 100 records a plurality of noise parameters corresponding to the type of the driving sound (noise). Then, noise data is generated using one of the plurality of noise parameters. In this embodiment, the imaging device 100 records a noise parameter for white noise as a constant noise. The imaging device 100 also records a noise parameter for short-term noise generated by, for example, meshing of gears in the optical lens 300. The imaging device 100 also records a noise parameter for, for example, sliding noise in the housing of the lens 300 as a long-term noise. In addition, the imaging device 100 may record noise parameters for each type of the optical lens 300, and for each temperature in the housing of the imaging device 100 and tilt of the imaging device 100 detected by the information acquisition unit 103.

[0088] <How to generate noise data> 6 and 7, the noise data generation process in the noise data generator 204 will be described. Here, the noise data generation process for the Lch data will be described, but the method of generating noise data for the Rch data is similar.

[0089] First, a process of generating noise parameters in a situation where it is considered that there is no environmental sound will be described. Fig. 6(a) is an example of a frequency spectrum of Lch_Before when a driving sound occurs within the housing of the optical lens 300 in a situation where it is considered that there is no environmental sound. Fig. 6(b) is an example of a frequency spectrum of Nch_Before when a driving sound occurs within the housing of the optical lens 300 in a situation where it is considered that there is no environmental sound. The horizontal axis indicates the frequency from the 0th point to the 512th point, and the vertical axis indicates the amplitude of the frequency spectrum.

[0090] Since it is assumed that there is no environmental sound, the amplitude of the frequency spectrum in the same frequency band is large in Lch_Before and Nch_Before. In addition, since the drive sound is generated inside the housing of the optical lens 300, the amplitude of each frequency spectrum for the same drive sound tends to be larger in Nch_Before than in Lch_Before.

[0091] FIG. 6(c) is an example of PLx in this embodiment. In this embodiment, PLx is a coefficient of each frequency spectrum calculated by dividing the amplitude of each frequency spectrum of Lch_Before by the amplitude of each frequency spectrum of Nch_Before. The result of this division is written as Lch_Before / Nch_Before. That is, PLx is the ratio of the amplitudes of Lch_Before and Nch_Before. The noise parameter recording unit 205 records the value of Lch_Before / Nch_Before as the noise parameter PLx. As described above, the amplitude of the frequency spectrum for the same driving sound tends to be larger in Nch_Before than in Lch_Before, so the value of each coefficient of the noise parameter PLx tends to be smaller than 1. However, if the value of Nch_Before[n] is smaller than a predetermined threshold value, the noise parameter recording unit 205 records the noise parameter PLx as PLx[n]=0.

[0092] Next, a process of applying the generated noise parameters to Nch_Before will be described. Fig. 7(a) is an example of a frequency spectrum of Lch_Before when a drive sound occurs within the housing of the optical lens 300 in a situation where an environmental sound is present. Fig. 7(b) is an example of a frequency spectrum of Nch_Before when a drive sound occurs within the housing of the optical lens 300 in a situation where an environmental sound is present. The horizontal axis indicates the frequency from the 0th point to the 512th point, and the vertical axis indicates the amplitude of the frequency spectrum.

[0093] 7(c) is an example of NL when a driving sound occurs inside the housing of the optical lens 300 in a situation where an environmental sound exists. The noise data generating unit 204 multiplies each frequency spectrum of Nch_Before by each coefficient of PLx to generate NL. NL is the frequency spectrum generated in this way.

[0094] 7D is an example of Lch_After when a driving sound occurs inside the housing of the optical lens 300 in a situation where an environmental sound exists. The subtraction processing unit 207 subtracts NL from Lch_Before to generate Lch_After. Lch_After is a frequency spectrum generated in this way.

[0095] This allows the imaging device 100 to reduce noise caused by driving sounds within the housing of the optical lens 300, and record environmental sounds with less noise.

[0096] <Explanation of the noise parameter selection unit 206> FIG. 8 is a block diagram showing an example of a detailed configuration of the noise parameter selection unit 206. As shown in FIG.

[0097] The noise parameter selection unit 206 receives Lch_Before, Rch_Before, Nch_Before, lens information, and a lens control signal.

[0098] The Nch noise detection unit 2061 detects, from Nch_Before, noise caused by drive sounds generated within the housing of the optical lens 300. In this embodiment, the Nch noise detection unit 2061 detects noise by using data of 513 points of Nch_Before.

[0099] The environmental sound detection unit 2062 detects the level of the environmental sound from Lch_Before and Rch_Before.

[0100] The noise determination unit 2063 determines noise parameters to be used by the noise data generation unit 204 based on the lens information, the lens control signal, the data input from the Nch noise detection unit 2061, and the data input from the environmental sound detection unit. The noise determination unit 2063 determines, from among the noise parameters recorded in the noise parameter recording unit 205, noise parameters corresponding to the type of lens indicated by the lens information from the lens control unit 102, as noise parameters to be used by the noise data generation unit 204. The noise determination unit 2063 outputs data indicating the type of the determined noise parameter to the noise data generation unit 204.

[0101] The Nch differentiation unit 2064 performs differentiation processing on Nch_Before. The Nch differentiation unit 2064 outputs data indicating the result of differentiation processing on Nch_Before to the short-term noise detection unit 2065. The short-term noise detection unit 2065 detects whether or not short-term noise is occurring based on the data input from the Nch differentiation unit 2064. The short-term noise detection unit 2065 outputs data indicating whether or not short-term noise is occurring to the noise determination unit 2063. Note that the Nch differentiation unit 2064 and the short-term noise detection unit 2065 are included in the Nch noise detection unit 2061.

[0102] The Nch integrator 2066 executes integration processing on Nch_Before. The Nch integrator 2066 outputs data indicating the result of differentiation processing of Nch_Before to the long-term noise detector 2067. The long-term noise detector 2067 detects whether or not long-term noise is occurring based on the data input from the Nch integrator 2066. The long-term noise detector 2067 outputs data indicating whether or not long-term noise is occurring to the noise determination unit 2063. Note that the Nch integrator 2066 and the long-term noise detector 2067 are included in the Nch noise detector 2061.

[0103] The environmental sound extraction unit 2068 extracts environmental sounds. In this embodiment, the environmental sound extraction unit 2068 extracts data of frequencies that are less affected by noise based on a noise parameter. For example, the environmental sound extraction unit 2068 extracts data of frequencies where the noise parameter is equal to or less than a predetermined value. Then, the environmental sound extraction unit 2068 outputs data indicating the volume of the environmental sound based on the extracted frequency data. Note that the environmental sound extraction unit 2068 is included in the environmental sound detection unit 2062.

[0104] The environmental sound determination unit 2069 determines the volume of the environmental sound. The environmental sound determination unit 2069 inputs data indicating the determined volume of the environmental sound to the Nch noise detection unit 2061 and the noise determination unit 2063. The Nch noise detection unit 2061 changes a first threshold value and a second threshold value, which will be described later, based on the data indicating the volume of the environmental sound input from the environmental sound determination unit 2069. Note that the environmental sound determination unit 2069 is included in the environmental sound detection unit 2062.

[0105] <Timing chart for noise reduction processing> The noise reduction process in this embodiment will be described with reference to FIG.

[0106] 9(a) to (i) are examples of timing charts of audio processing in the noise data generator 204, the noise parameter selector 206, and the subtraction processor 207. In this embodiment, for the sake of simplicity, audio processing for the Lch will be described, but audio processing for the Rch is similar. The horizontal axis of the graphs in Figs. 9(a) to (i) is the time axis.

[0107] Fig. 9(a) shows an example of a lens control signal. The lens control signal is a signal that the lens control unit 102 uses to instruct the optical lens 300 to drive. In this embodiment, the level of the lens control signal is expressed as two values, High and Low. When the level of the lens control signal is High, the lens control unit 102 is instructing the optical lens 300 to drive. When the level of the lens control signal is Low, the lens control unit 102 is not instructing the optical lens 300 to drive.

[0108] FIG. 9(b) is a graph showing an example of the value of Lch_Before[n]. The vertical axis indicates the value of Lch_Before[n]. In this embodiment, Lch_Before[n] is a signal at the nth frequency point, among the Lch_Before output from the FFT unit 203, at which a signal indicating the driving sound of the optical lens 300 appears characteristically. Note that in this embodiment, the signal at the nth frequency point will be described, but audio processing is similarly performed for other frequencies. Furthermore, the signals indicated by signal X and signal Y are signals containing noise. In this embodiment, signal X indicates a signal containing short-term noise. Signal Y indicates a noise signal containing long-term noise.

[0109] 9(c) is a graph showing an example of the volume of the environmental sound extracted by the environmental sound extraction unit 2068. The vertical axis indicates the level of the audio signal generated from the acquired environmental sound. Threshold value Th1 and threshold value Th2 are two threshold values ​​used in the environmental sound determination unit 2069.

[0110] Fig. 9(d) is a graph showing an example of the value of Nch_Before[n]. Nch_Before[n] is a signal at the nth frequency point, among Nch_Before output from the FFT unit 203, at which a signal indicating the driving sound of the optical lens 300 appears characteristically. The vertical axis indicates the value of Nch_Before[n]. In Nch_Before[n], the noise signals indicated by the signals X and Y in Fig. 9(b) appear more characteristically than in Lch_Before.

[0111] FIG. 9(e) is a graph showing an example of the value of Ndiff[n]. Ndiff[n] indicates the value of the signal at the n-th frequency point among Ndiff output from the Nch differentiation unit 2064. The vertical axis indicates the value of Ndiff[n]. If the amount of change in the value of Nch_Before[n] per predetermined time is large, the value of Ndiff[n] becomes large. The short-term noise detection unit 2065 has a threshold value Th_Ndiff[n], which is a first threshold value, in order to detect short-term noise. The threshold value Th_Ndiff[n] changes between levels 1 to 3 based on the data indicating the volume of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold value Th_Ndiff[n] is set to level 2. The level of the threshold value Th_Ndiff[n] is represented by a horizontal dashed line.

[0112] FIG. 9(f) is a graph showing an example of the value of Nint[n]. In this embodiment, Nint[n] indicates the value of the signal at the n-th frequency point among Nint output from the Nch integration unit 2066. The vertical axis indicates the value of Nint[n]. If Nch_Before[n] is continuously large, the value of Nint[n] increases. The long-term noise detection unit 2067 has a second threshold, a threshold Th_Nint[n], in order to detect long-term noise. The threshold Th_Nint[n] changes between levels 1 to 3 based on the data indicating the volume of the environmental sound input from the environmental sound determination unit 2069 and the lens control signal. The initial value of the threshold Th_Nint[n] is set to level 2. The level of the threshold Th_Nint[n] is represented by a horizontal dashed line.

[0113] 9(g) shows an example of noise parameters selected by the noise parameter selection unit 206. In this embodiment, the solid part indicates that only the noise parameter of PL1 is selected. The diagonal line part indicates that the noise parameters of PL1 and PL2 are selected. The checkered part indicates that the noise parameters of PL1 and PL3 are selected.

[0114] 9(h) is a graph showing an example of the value of NL[n]. In this embodiment, NL[n] indicates the signal value of the n-th frequency point among the NL generated by the noise data generating unit 204. The vertical axis indicates the value of NL[n].

[0115] 9(i) is a graph showing an example of the value of Lch_After[n]. In this embodiment, Lch_After[n] indicates the value of the signal at the n-th frequency point among the Lch_After output from the subtraction processing unit 207. The vertical axis indicates the value of Lch_After[n].

[0116] Next, the timing of each operation will be described using times t701 to t709.

[0117] At time t701, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9(a)). At time t701, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the short-term noise detection unit 2065 lowers the threshold Th_Ndiff[n] to level 1 (FIG. 9(e)). Also at time t701, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold Th_Nint[n] to level 1 (FIG. 9(f)).

[0118] At time t702, the optical lens 300 is driven, generating short-term drive sound such as the sound of gears meshing. The noise microphone 201c picks up the short-term drive sound, causing the value of Ndiff[n] to exceed the threshold value Th_Ndiff[n] (FIG. 9(e)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL2 (FIG. 9(g)). The noise data generation unit 204 generates NL[n] based on Nch_Before[n] and the noise parameters PL1 and PL2 (FIG. 9(h)). The subtraction processing unit 207 subtracts NL[n] from Lch_Before[n] and outputs Lch_After[n] (FIG. 9(i)). In this case, Lch_After[n] becomes an audio signal in which constant noise and short-term noise have been reduced.

[0119] At time t703, the optical lens 300 starts to drive continuously, and a long-term driving sound such as a sliding sound is generated in the housing of the optical lens 300. The noise microphone 201c picks up the long-term driving sound, and the value of Nint[n] exceeds the threshold value Th_Nint[n] (FIG. 9(f)). In response to this, the noise parameter selection unit 206 selects the noise parameters PL1 and PL3 (FIG. 9(g)). The noise data generation unit 204 generates NL[n] based on Nch_Before[n] and the noise parameters PL1 and PL3 (FIG. 9(h)). The subtraction processing unit 207 subtracts NL[n] from Lch_Before[n] and outputs Lch_After[n] (FIG. 9(i)). In this case, Lch_After[n] becomes an audio signal in which constant noise and long-term noise have been reduced.

[0120] At time t704, the optical lens 300 stops continuous driving. Since the noise microphone 201c no longer picks up the long-term driving sound, the value of Nint[n] becomes equal to or less than the threshold value Th_Nint[n] (FIG. 9(f)). In response to this, the noise parameter selection unit 206 selects the noise parameter PL1 (FIG. 9(g)). The noise data generation unit 204 generates NL[n] based on Nch_Before[n] and the noise parameter PL1 (FIG. 9(h)). The subtraction processing unit 207 subtracts NL[n] from Lch_Before[n] and outputs Lch_After[n] (FIG. 9(i)). In this case, Lch_After[n] becomes an audio signal with constant noise reduced.

[0121] At time t705, the lens control unit 102 outputs a Low signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9(a)). In this case, since the possibility of drive sound occurring within the housing of the optical lens 300 is low, the short-term noise detection unit 2065 increases the threshold Th_Ndiff[n] to level 2 (FIG. 9(e)). Also, in this case, since the possibility of drive sound occurring within the housing of the optical lens 300 is low, the long-term noise detection unit 2067 increases the threshold Th_Nint[n] to level 2 (FIG. 9(f)).

[0122] At time t706, the volume of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold Th1. When the environmental sound is loud, the user is less likely to perceive the noise contained in the audio signal, so the short-term noise detection unit 2065 raises the threshold Th_Ndiff[n] to level 3 (FIG. 9(e)). Also, when the environmental sound is loud, the user is less likely to perceive the noise contained in the audio signal, so the long-term noise detection unit 2067 raises the threshold Th_Nint[n] to level 3 (FIG. 9(f)).

[0123] At time t707, the lens control unit 102 outputs a High signal as a lens control signal to the optical lens 300 and the noise parameter selection unit 206 (FIG. 9(a)). In this case, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the short-term noise detection unit 2065 lowers the threshold Th_Ndiff[n] to level 2 (FIG. 9(e)). Also, in this case, since there is a high possibility that drive sound will be generated within the housing of the optical lens 300, the long-term noise detection unit 2067 lowers the threshold Th_Nint[n] to level 2 (FIG. 9(f)).

[0124] At time t708, the volume of the environmental sound extracted by the environmental sound extraction unit 2068 exceeds the threshold value Th2. Here, the noise parameter selection unit 206 selects only the noise parameter PL1 regardless of the data input from the Nch noise detection unit 2061. In this way, when the environmental sound is very loud, the user is less likely to perceive the noise contained in the audio signal, so the imaging device 100 reduces only the constant noise, thereby recording environmental sound that is more natural than when processing is performed to further reduce short-term noise and long-term noise.

[0125] As described above, the imaging device 100 can record environmental sounds with reduced noise by performing noise reduction processing using the noise microphone 201c, which is the second microphone.

[0126] Furthermore, the image capturing device 100 detects the occurrence of noise using the output signal of the noise microphone 201c, and sets the noise parameters in accordance with the timing of detecting the occurrence of the noise. Therefore, the image capturing device 100 can appropriately set the noise parameters in synchronization with the occurrence of noise, and appropriately reduce the noise.

[0127] Furthermore, when the volume of the environmental sound is equal to or less than the threshold Th2, the imaging device 100 executes noise reduction processing according to the noise detected by the Nch noise detection unit 206, and when the environmental sound is greater than the threshold Th2, reduces only constant noise. This enables the imaging device 100 to record environmental sound with reduced noise according to the volume of the environmental sound so as to reduce discomfort to the user.

[0128] In this embodiment, the imaging device 100 reduces the driving sound generated within the housing of the optical lens 300, but the driving sound generated within the imaging device 100 may be reduced. The driving sound generated within the imaging device 100 is, for example, noise from the board and radio wave noise. The noise from the board is, for example, a sound generated by the creaking of the board when a voltage is applied to a capacitor on the board.

[0129] The thresholds Th1 and Th2 of the environmental sound determination unit 2069, the threshold Th_Ndiff[n] of the short-term noise detection unit 2065, and the threshold Th_Nint[n] of the long-term noise detection unit 2067 are determined based on the generated driving sound and environmental sound. Therefore, the imaging device 100 may change these thresholds depending on the type of the optical lens 300, the inclination of the imaging device 100, and the like.

[0130] <Noise parameters for each lens> First, a case where the optical lens 300 is replaced and the same noise parameters are applied to each lens will be described with reference to Fig. 7 and Fig. 11. It is assumed that different lenses are attached to the imaging device 100 in Fig. 7 and Fig. 11. It is assumed that the same environmental sound is occurring in Fig. 7 and Fig. 11.

[0131] 7(a) and 11(a) show the amplitude of each frequency spectrum of Lch_Before. FIG. 7(b) and 11(b) show the amplitude of each frequency spectrum of Nch_Before. Comparing the frequency spectra, it can be seen that they are at least partially different from each other. This is because the noises picked up by the L microphone 201a, the R microphone 201b, and the noise microphone 201c differ depending on the shape and structure of the lens, and the number and positions of noise sources such as driving bodies included in the lens.

[0132] 7(c) and 11(c) show the amplitude of each frequency spectrum of NL, where the same noise parameters are used to generate NL regardless of the lens attached to the imaging device 100.

[0133] In FIG. 7(d) and FIG. 11(d), the amplitude of each frequency spectrum of Lch_After is shown. Since Lch_After is an audio signal recorded as environmental sound, ideally the frequency spectrum shown in FIG. 7(d) and FIG. 11(d) will be the same. However, when comparing the frequency spectra of FIG. 7(d) and FIG. 11(d), it can be seen that they are at least partially different from each other. This is because the frequency spectrum of NL subtracted from Lch_Before is different from the frequency spectrum of noise generated in each lens. The reason why the frequency spectrum of NL is different from the frequency spectrum of noise is because the noise parameters are not parameters made to match the noise generated from the lens, but are general-purpose parameters for a certain type of noise. In other words, if the same noise parameters are applied to noise generated from various lenses, the audio data recorded on the recording medium 110 as environmental sound will be different for each lens attached to the imaging device 100.

[0134] Therefore, in this embodiment, the imaging device 100 effectively reduces noise and more accurately records environmental sounds by making the noise parameters different for each lens. FIG. 10 shows an example of noise parameters for each lens that can be attached to the imaging device 100 as the optical lens 300. In FIG. 10, the noise parameters of PL2 and PR2 are shown as default values ​​and for each lens. The noise parameter recording unit 205 of this embodiment records the noise parameters for each lens in advance. Note that the noise parameter recording unit 205 also records the other lens parameters PLx and PRx for each lens in the same manner.

[0135] In this embodiment, the noise parameter recording unit 205 records, as default values, noise parameters for lenses other than those for which noise parameters are recorded in advance, because the image capture device 100 can reduce, to a certain degree, the noise propagating from the lens by applying general-purpose noise parameters to noise generated from a lens for which noise parameters are not recorded in advance.

[0136] The process of the imaging device 100 when replacing the optical lens 300 will now be described with reference to the flowchart of Fig. 12. The process of the imaging device 100 is triggered, for example, by the user operating the power switch 72 to turn on the power.

[0137] In step S1201, the control unit 111 controls the supply of power to each component of the imaging device 100 from a power source (not shown).

[0138] In step S1202, the lens control unit 102 receives lens information from the optical lens 300. The lens information includes, for example, the type of lens, the model number of the lens, and the type of noise source. The control unit 111 records the received lens information in the non-volatile memory 117.

[0139] In step S1203, the lens control unit 102 determines whether the lens has been replaced. For example, the lens control unit 102 compares the lens information recorded in the non-volatile memory 117 before the process of step S1202 with the lens information received in step S1202. If it is determined that the two pieces of lens information match, the lens control unit 102 determines that the lens has not been replaced. If it is determined that the two pieces of lens information do not match, the lens control unit 102 determines that the lens has been replaced. Also, for example, if the lens information has not been recorded in the non-volatile memory 117 before the process of step S1202, the lens control unit 102 determines that the lens has been replaced. If the lens control unit 102 determines that the lens has been replaced, the process of step S1204 is executed. If the lens control unit 102 determines that the lens has not been replaced, the process of step S1208 is executed.

[0140] In step S1204, the noise parameter selection unit 206 receives the lens information from the lens control unit 102.

[0141] In step S1205, the noise parameter selection unit 206 determines whether or not noise parameters for the optical lens 300 attached to the imaging device 100 are recorded in the noise parameter recording unit 205. For example, the noise parameter selection unit 206 determines whether or not noise parameters for the lens are recorded in the noise parameter recording unit 205 based on the lens model number included in the lens information received in step S1204. If the noise parameter selection unit 206 determines that the noise parameters of the optical lens 300 are registered in the noise parameter recording unit 205, the process of step S1206 is executed. If the noise parameter selection unit 206 determines that the noise parameters of the optical lens 300 are not registered in the noise parameter recording unit 205, the process of step S1207 is executed.

[0142] In step S1206, the noise parameter selection unit 206 decides to use the noise parameters for the optical lens 300, and instructs the noise data generation unit 204 to use the noise parameters. For example, the noise parameter selection unit 206 transmits data indicating the model number of the optical lens 300 to the noise data generation unit 204. Note that in this case, when reading the noise parameters from the noise parameter recording unit 205, the noise data generation unit 204 reads the noise parameters for the optical lens 300 instructed by the noise parameter selection unit 206.

[0143] In step S1207, the noise parameter selection unit 206 decides to use default noise parameters, and instructs the noise data generation unit 204 to use the noise parameters. For example, the noise parameter selection unit 206 transmits data indicating the default values ​​to the noise data generation unit 204. In this case, when reading the noise parameters from the noise parameter recording unit 205, the noise data generation unit 204 reads the noise parameters of the default values ​​instructed by the noise parameter selection unit 206.

[0144] In step S1208, control unit 111 determines whether or not to turn off the power. For example, when the power switch is operated to turn off the power, control unit 111 determines to turn off the power. When control unit 111 determines to turn off the power, the process of step S1212 is executed. When control unit 111 determines not to turn off the power, the process of step S1209 is executed.

[0145] In step S1209, control unit 111 determines whether or not to start moving image recording. For example, control unit 111 determines to start moving image recording in response to pressing of release switch 61. Conversely, control unit 111 determines not to start moving image recording when release switch 61 is not pressed. If control unit 111 determines to start moving image recording, processing of step S1209 is executed. If control unit 111 determines not to start moving image recording, processing of step S1203 is executed.

[0146] In step S1210, control unit 111 records a moving image. For example, as described above, control unit 111 records moving image data with audio on recording medium 110. In the audio processing during this moving image recording, the noise parameters determined in the processing of step S1206 or the processing of step S1207 are used.

[0147] In step S1211, control unit 111 determines whether or not to end moving image recording. For example, control unit 111 determines to end moving image recording in response to pressing of release switch 61. Conversely, control unit 111 determines not to end moving image recording when release switch 61 is not pressed. If control unit 111 determines to end moving image recording, processing of step S1203 is executed. If control unit 111 determines not to end moving image recording, processing of step S1210 is executed.

[0148] In step S1212, the control unit 111 records the lens information in the non-volatile memory 117. The lens information recorded in this step is used the next time the process of step S1203 is executed.

[0149] The process of the imaging device 100 when replacing the optical lens 300 has been described above.

[0150] In this way, by changing the noise parameters depending on the optical lens 300 attached to the image capture device 100, effective noise reduction processing can be performed.

[0151] In this embodiment, the noise parameters are recorded for each lens, but they may be recorded for each type of noise source. For example, the image capture device 100 prepares noise parameters for each lens that uses USM and for each lens that uses STM. In this case, the noise parameter selection unit 206 instructs the noise data generation unit 204 to use the noise parameters for USM or the noise parameters for STM based on the type of noise source included in the lens information received from the lens control unit 102.

[0152] [Second Example] In the first embodiment, the image capture device 100 uses noise parameters according to the type of lens attached to the image capture device 100. In the second embodiment, a process will be described in which, in addition to using noise parameters according to the type of lens attached to the image capture device 100, noise parameters according to the attitude of the image capture device 100 and the state of the lens are used.

[0153] The second embodiment will be described using the same image pickup device 100 as in the first embodiment. Note that the configuration of the image pickup device 100 is the same as in the first embodiment, and therefore description thereof will be omitted.

[0154] FIG. 13 is a diagram showing the attitude information of the imaging device 100 in this embodiment. In this embodiment, the angle is 0° when the imaging direction of the imaging device 100 is horizontal to the ground. The angle is 90° when the imaging direction of the imaging device 100 is vertical upward to the ground. The angle is -90° when the imaging direction of the imaging device 100 is vertical downward to the ground. Here, the angle θ indicates the angle of the imaging direction of the imaging device 100. In this embodiment, the imaging device 100 acquires the attitude information of the imaging device 100 by the information acquisition unit 103. For example, the information acquisition unit 103 detects the inclination of the imaging device 100 by an acceleration sensor or a gyro sensor.

[0155] In this embodiment, the range of the imaging direction of the imaging device 100 is divided into five. Range a is 60° to 90°, range b is 30° to 60°, range c is -30° to 30°, range d is -60° to -30°, and range e is -90° to -60°. For example, when the angle θ, which is the imaging direction of the imaging device 100, is 50°, it falls into range b. Note that, although the range of the imaging direction of the imaging device 100 is divided into five in this embodiment, the imaging direction of the imaging device 100 may have multiple ranges.

[0156] Gravity acting on the motor in the lens, which is a noise source in this embodiment, changes according to the attitude of the image capture device 100. When gravity changes, the holding force of the holding mechanism that holds the lens changes, and the movement of the motor that drives this holding mechanism also changes. Therefore, the drive sound generated when the motor is driven also changes according to gravity, and the frequency characteristics of the noise picked up by each microphone also change according to gravity.

[0157] 14(a) shows Lch_Before when the angle θ indicating the attitude of the image capture device 100 is 0° and 50°. In this embodiment, the amplitude of the frequency spectrum changes from Lch_Before

[0400] to Lch_Before

[0450] . Here, in Lch_Before

[0400] to Lch_Before

[0450] , the peak value of the amplitude of the frequency spectrum when the angle θ is 0° is A, and the peak value of the amplitude of the frequency spectrum when the angle θ is 50° is A'.

[0158] 14B shows Nch_Before when the angle θ indicating the attitude of the image capture device 100 is 0° and 50°. In this embodiment, the amplitude of the frequency spectrum changes from Nch_Before

[0400] to Nch_Before

[0450] . In this embodiment, the position of the frequency spectrum where the amplitude changes is the same in Lch_Before and Nch_Before. Here, in Nch_Before

[0400] to Nch_Before

[0450] , the peak value of the amplitude of the frequency spectrum when the angle θ is 0° is B, and the peak value of the amplitude of the frequency spectrum when the angle θ is 50° is B'.

[0159] Here, the change amount |A'-A| in the peak value of the amplitude of the frequency spectrum of Lch_Before due to a change in the posture of the imaging device 100 is compared with the change amount |B'-B| in the peak value of the amplitude of the frequency spectrum of Nch_Before. Referring to FIG. 14, it can be seen that |B'-B|>|A'-A|. This indicates that the change amount of the volume of the noise picked up by the noise microphone 201c is larger than the change amount of the volume of the noise picked up by the L microphone 201a. Therefore, in this embodiment, in order to effectively reduce noise, the imaging device 100 corrects the noise parameters in a range where the amplitude of the frequency spectrum changes in response to a change in the posture of the imaging device 100.

[0160] FIG. 15 shows correction noise parameters for each posture information of the imaging device 100. In this embodiment, the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before are assumed to change in the range of N=400 to 450. On the other hand, in this embodiment, the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before are assumed to change sufficiently small in the other ranges of N so that the noise parameters do not need to be corrected. As shown in FIG. 15, only the frequency spectrum that needs to be corrected is recorded in the noise parameter recording unit 205 as the noise parameters for correction. For example, when the angle θ is 50°, the noise parameter selection unit 206 instructs the noise data generation unit 204 to use the correction noise parameters in the range b together with the noise parameters to be used. Note that, when the angle θ is in the range c, the noise parameter selection unit 206 may determine that the correction of the noise parameters is not necessary, and may not instruct the noise data generation unit 204 regarding the correction noise parameters.

[0161] FIG. 16 is a comparison of the length of a zoom lens when the lens magnification is different. As shown in FIG. 16, the focal length of a zoom lens can be changed by extending and contracting the lens barrel and moving the lens position. This changes the shape and structure of the lens, so that the propagation path and propagation distance of noise from the motor, which is the noise source, to each microphone changes, and the frequency spectrum of the noise picked up by the microphone changes. In this embodiment, the lens control unit 102 of the imaging device 100 receives the zoom magnification from the optical lens 300 as lens information, and thereby acquires the zoom magnification of the optical lens 300.

[0162] 17(a) shows Lch_Before when the zoom magnification of the optical lens 300 is 1x and 5x. In this embodiment, the amplitude of the frequency spectrum changes from Lch_Before

[0200] to Lch_Before

[0250] . Here, in Lch_Before

[0200] to Lch_Before

[0250] , the peak value of the amplitude of the frequency spectrum when the zoom magnification is 5x is denoted as C, and the peak value of the amplitude of the frequency spectrum when the zoom magnification is 1x is denoted as C'.

[0163] 17(b) shows Nch_Before when the zoom magnification of the optical lens 300 is 1x and 5x. In this embodiment, the amplitude of the frequency spectrum changes from Nch_Before

[0200] to Nch_Before

[0250] . Here, in Nch_Before

[0200] to Nch_Before

[0250] , the peak value of the amplitude of the frequency spectrum when the zoom magnification is 5x is D, and the peak value of the amplitude of the frequency spectrum when the zoom magnification is 1x is D'.

[0164] Here, the change amount |C'-C| in the peak value of the amplitude of the frequency spectrum of Lch_Before due to the change in the zoom magnification of the optical lens 300 is compared with the change amount |D'-D| in the peak value of the amplitude of the frequency spectrum of Nch_Before. Referring to FIG. 17, it can be seen that |D'-D|>|C'-C|. This indicates that the change amount of the volume of the noise picked up by the noise microphone 201c is larger than the change amount of the volume of the noise picked up by the L microphone 201a, as in the case where the attitude of the imaging device 100 changes. Therefore, in this embodiment, in order to effectively reduce noise, the imaging device 100 corrects the noise parameters in the range where the amplitude of the frequency spectrum changes in response to the change in the zoom magnification of the optical lens 300.

[0165] FIG. 18 shows the correction noise parameters for each zoom magnification of the optical lens 300. In this embodiment, the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before are assumed to change in the range of N=200 to 250. On the other hand, in this embodiment, the amplitudes of the frequency spectra of Lch_Before, Rch_Before, and Nch_Before are assumed to change sufficiently small in the other ranges of N so that the noise parameters do not need to be corrected. As shown in FIG. 18, only the frequency spectrum that needs to be corrected is recorded in the noise parameter recording unit 205 as the correction noise parameters. For example, when the zoom magnification is 2.3 times, the noise parameter selection unit 206 instructs the noise data generation unit 204 to use the correction noise parameters for a zoom magnification of 2 to 3 times together with the noise parameters to be used. Note that, when the zoom magnification is 1 time, the noise parameter selection unit 206 may determine that the correction of the noise parameters is not necessary, and may not instruct the noise data generation unit 204 on the correction noise parameters.

[0166] Now, the noise parameter correction process of the image capture device 100 during moving image recording will be described with reference to the flowchart of Fig. 19. This process of the image capture device 100 is started when triggered by, for example, the user pressing the release switch 61.

[0167] In step S1901, the noise parameter selection unit 206 determines to use the noise parameters for the optical lens 300 attached to the image capture device 100.

[0168] In step S1902, the noise parameter selection unit 206 determines whether or not noise has been detected. If the noise parameter selection unit 206 determines that noise has been detected, the process proceeds to step S1903. If the noise parameter selection unit 206 determines that noise has not been detected, the process proceeds to step S1909.

[0169] In step S 1903 , the noise parameter selection unit 206 acquires the posture information of the image capturing device 100 from the information acquisition unit 103 .

[0170] In step S1904, the noise parameter selection unit 206 acquires the zoom magnification of the optical lens 300 from the lens control unit .

[0171] In step S1905, the noise parameter selection unit 206 determines the correction noise parameters based on the posture information acquired in step S1903 and the zoom magnification acquired in step S1904.

[0172] In step S1906, the noise parameter selection unit 206 instructs the noise data generation unit 204 on the correction noise parameters to be used in the noise reduction process. The noise data generation unit 204 generates noise data using the noise parameters and correction parameters for the optical lens 300.

[0173] In step S1907, the audio input unit 104 records the audio data that has been subjected to noise reduction processing in the volatile memory 105. The control unit 111 records moving image data with audio in the recording medium 110 using this audio data.

[0174] In step S1908, control unit 111 determines whether to end movie recording. For example, control unit 111 determines to end movie recording in response to pressing of release switch 61. Conversely, control unit 111 determines not to end movie recording when release switch 61 is not pressed. If control unit 111 determines to end movie recording, processing of step S1909 is executed. If control unit 111 determines not to start movie recording, processing of this flowchart ends.

[0175] The noise parameter correction process of the image capturing device 100 has been described above.

[0176] In this way, by correcting the noise parameters during video recording, effective noise reduction processing can be performed.

[0177] In this embodiment, as an example, the amplitudes of the frequency spectra of Lch_Before

[0200] to Lch_Before

[0250] and Nch_Before

[0200] to Nch_Before

[0250] change depending on the posture of the imaging device 100. However, the position of the frequency spectrum that changes depending on the posture of the imaging device 100 depends on the shape and structure of the lens, and therefore differs for each type of optical lens 300. Similarly, the position of the frequency spectrum that changes depending on the zoom magnification of the optical lens 300 depends on the shape and structure of the lens, and therefore the position of the frequency spectrum differs for each type of optical lens 300. In other words, the correction noise parameters become noise parameters at positions other than the frequency spectrum shown in this embodiment depending on the type of the optical lens 300.

[0178] Additionally, the noise parameters of the imaging device 100 may be changed depending on whether the device is held in the user's hand or fixed to a tripod, etc. This is a correction for the fact that the frequency spectrum of the noise input to the microphone differs due to differences in transfer functions, etc., between the state where the imaging device 100 is held in the user's hand and the state where the imaging device is fixed.

[0179] Alternatively, the configuration may be such that the state of the device that affects the frequency spectrum of the noise picked up by each microphone is detected, and the noise parameters are corrected according to the detection result. Specifically, the state of the audio processing device that affects the sound transmission path and transfer function from the noise source to each microphone is detected, and the noise parameters are corrected according to the detected state.

[0180] [Third Example] Fig. 20 is a block diagram showing an example of the configuration of the voice input unit 104 in the second embodiment. The parts different from the configuration of the voice input unit 104 shown in Fig. 3 are a subtraction processing unit 207 and an iFFT unit 208. Here, a description of the processing units similar to those in Fig. 3 will be omitted.

[0181] The iFFT unit 208a performs an inverse fast Fourier transform on Lch_Before and Rch_Before input from the FFT unit 203, respectively, to convert the frequency domain digital audio signals into time domain digital audio signals. The iFFT unit 208b also performs an inverse fast Fourier transform on NL and NR, respectively, to convert the frequency domain digital audio signals into time domain digital audio signals.

[0182] The subtraction processing unit 207 subtracts the digital audio signal input from the iFFT unit 208b from the digital audio signal input from the iFFT unit 208a. The calculation process in the subtraction processing unit 207 is a waveform subtraction method that subtracts the digital audio signal in the time domain.

[0183] The waveform subtraction method in the third embodiment can be applied in place of the spectral subtraction method in the first or second embodiment.

[0184] When performing waveform subtraction, the image capturing apparatus 100 may also record parameters related to the phase of the digital audio signal as noise parameters.

[0185] [Other Examples] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a recording medium, and one or more processors in a computer of the system or device read and execute the program. Also, the present invention can be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0186] The present invention is not limited to the above-mentioned embodiment, and the components can be modified without departing from the spirit of the invention. Various inventions can be formed by appropriately combining the components disclosed in the above-mentioned embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

Claims

1. a mounting means for mounting a lens having a noise source; a first microphone for acquiring environmental sounds; a second microphone for acquiring sound from the noise source; a first transforming means for Fourier transforming the audio signal from the first microphone to generate a first audio signal in the frequency domain; a second transforming means for Fourier transforming the audio signal from the second microphone to generate a second audio signal in the frequency domain; a generating means for calculating the second audio signal and a parameter related to the noise of the noise source corresponding to the type of the lens attached to the attachment means, and generating noise data; a subtraction means for subtracting the noise data from the first audio signal; a third transform means for performing an inverse Fourier transform on the audio signal output from the subtraction means; having The parameters include a coefficient for each of a plurality of frequency points that constitute a frequency spectrum, the generating means generates the noise data by multiplying a frequency spectrum of a plurality of frequency points of the second audio signal by the coefficient of the parameter; The subtraction means subtracts a frequency spectrum of a plurality of frequency points of the noise data from a frequency spectrum of a plurality of frequency points of the first audio signal.

13. A voice processing device comprising:

2. 2. The audio processing device according to claim 1, wherein the generating means determines the parameters to be used for calculating the noise data from a plurality of parameters corresponding to a plurality of lens types based on information regarding the type of the lens attached to the mounting means.

3. A detection means for detecting that the lens has been replaced; 2. The audio processing device according to claim 1, further comprising a determination means for determining, in response to detection by the detection means that the lens has been replaced, from a plurality of parameters corresponding to a plurality of lens types, the parameter corresponding to the type of lens attached to the mounting means by the replacement as the parameter to be used in the calculation by the generation means.

4. a recording means for recording a plurality of parameters corresponding to the plurality of lens types; 4. The audio processing device according to claim 3, wherein the determination means determines, from the plurality of parameters recorded in the recording means, the parameter corresponding to the type of lens attached to the mounting means as the parameter to be used in the calculation by the generation means.

5. The audio processing device according to claim 4, characterized in that, when the parameters corresponding to the type of lens attached to the attachment means are not recorded in the recording means, the determination means determines the default values ​​of the parameters as the parameters to be used in calculations by the generation means.

6. 6. The audio processing device according to claim 1, wherein the noise source is a motor, and the type of the motor included in the lens varies depending on the type of the lens.

7. 7. The audio processing device according to claim 1, wherein the type of the lens is a model number of the lens.

8. a mounting means for mounting a lens having a noise source; a first microphone for acquiring environmental sounds; A method for controlling a sound processing device having a second microphone for acquiring a sound from the noise source, the method comprising: performing a Fourier transform on the audio signal from the first microphone to generate a first audio signal in the frequency domain; performing a Fourier transform on the audio signal from the second microphone to generate a second audio signal in the frequency domain; a generating step of calculating the second audio signal and a parameter related to the noise of the noise source corresponding to the type of the lens attached to the attachment means to generate noise data; a subtraction step of subtracting the noise data from the first audio signal; performing an inverse Fourier transform on the audio signal output by the subtraction step; having The parameters include a coefficient for each of a plurality of frequency points that constitute a frequency spectrum, the generating step generates the noise data by multiplying a frequency spectrum of a plurality of frequency points of the second audio signal by the coefficient of the parameter; The subtraction step subtracts a frequency spectrum of a plurality of frequency points of the noise data from a frequency spectrum of a plurality of frequency points of the first audio signal. A control method comprising:

9. A computer-readable program for causing a computer to function as each of the means of the voice processing device according to any one of claims 1 to 7.

Images