AUDIO SIGNAL PROCESSING APPARATUS, AUDIO SIGNAL PROCESSING METHOD, AND PROGRAM

Abstract

This acoustic signal processing device includes: noise cancellation processing units which are provided for each of a plurality of microphones, and which generate a signal for canceling noise, on the basis of an input audio signal from the microphone; and a digital filter for processing an external input signal.　Figure 7

Description

[Technical field]

[0001] The present disclosure relates to an audio signal processing device, an audio signal processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art There is known a technology relating to so-called noise canceling, which cancels external noise when playing music or the like through a headphone device (see, for example, Patent Document 1 below). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-25918 A Summary of the Invention [Problem to be solved by the invention]

[0004] In such fields, it is desirable to be able to more effectively cancel external noise leaking into the inside of a headphone device.

[0005] An object of the present disclosure is to provide an audio signal processing device, an audio signal processing method, and a program that more effectively cancel noise. [Means for solving the problem]

[0006] The present disclosure relates to, for example, a noise cancellation processing unit provided for each of the plurality of microphones, the noise cancellation processing unit generating a signal for canceling noise based on an input audio signal from the microphone; A control unit that generates control parameters for a noise cancellation processing unit; an analysis unit that analyzes an input audio signal from a microphone; A digital filter for processing an external input signal received from an external device; An acoustic signal processing device comprising: the external input signal includes audio objects and meta information corresponding to the audio objects; The analysis of the input audio signal includes changing a playback position of the audio object based on noise arrival direction information and meta information generated based on the input audio signal from the microphone; The control unit further generates control parameters for the digital filter. An audio signal processing device.

[0007] The present disclosure relates to, for example, A noise canceling processor provided for each of the plurality of microphones generates a signal for canceling noise based on an input audio signal from the microphone; A processor generates control parameters for a noise cancellation processing unit, analyzes an input audio signal from a microphone, A digital filter processes an external input signal received from the outside. 1. A method for processing an acoustic signal, comprising: the external input signal includes audio objects and meta information corresponding to the audio objects; The processor further changes a playback position of the audio object and generates a control parameter for the digital filter based on the noise arrival direction information and the meta information generated based on the input audio signal from the microphone. An acoustic signal processing method.

[0008] The present disclosure relates to, for example, A noise canceling processor provided for each of the plurality of microphones generates a signal for canceling noise based on an input audio signal from the microphone; A processor generates control parameters for a noise cancellation processing unit, analyzes an input audio signal from a microphone, A digital filter processes an external input signal received from the outside. A program for causing a computer to execute an acoustic signal processing method, the external input signal includes audio objects and meta information corresponding to the audio objects; The processor further changes a playback position of the audio object and generates a control parameter for the digital filter based on the noise arrival direction information and the meta information generated based on the input audio signal from the microphone. This is a program for causing a computer to execute an acoustic signal processing method. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the relationship between a typical noise-canceling headphone and the direction from which noise comes. [Diagram 2] FIG. 2 is a diagram to which reference is made when describing the issues to be considered in this disclosure. [Diagram 3] FIG. 3 is a diagram that is referred to when explaining how the reproducibility of 3D audio is reduced due to the masking effect of noise. [Figure 4] FIG. 4 is a diagram that will be referred to when explaining the outline of the embodiment. [Diagram 5] FIG. 5 is a diagram that will be referred to when explaining the outline of the embodiment. [Figure 6] FIG. 6 is a diagram that will be referred to when explaining the outline of the embodiment. [Figure 7]FIG. 7 is a diagram illustrating an example of the configuration of headphones according to the first embodiment. [Figure 8] 8A and 8B are diagrams for explaining an overview of the second embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of the configuration of headphones according to the second embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of the configuration of an analysis unit according to the second embodiment. [Figure 11] FIG. 11 is a diagram for explaining an example of a noise arrival direction to be searched. [Figure 12] FIG. 12 is a diagram illustrating a specific example of the configuration of the noise arrival direction estimating unit. [Figure 13] FIG. 13 is a diagram illustrating an example of noise arrival direction information. [Figure 14] FIG. 14 is a diagram showing a specific example of the configuration of the audio object optimum placement position calculation unit. [Figure 15] FIG. 15 is a flowchart illustrating a first example of a sound source determination process performed by the sound source direction determining unit. [Figure 16] FIG. 16 is a flowchart illustrating a second example of the sound source determination process performed by the sound source direction determining unit. [Figure 17] 17A and 17B are diagrams to be referred to when explaining a second example of the sound source determination process performed by the sound source direction determining unit. [Figure 18] FIG. 18 is a flowchart illustrating a third example of the sound source determination process performed by the sound source direction determining unit. [Figure 19] 19A to 19D are diagrams to be referred to when explaining a third example of the sound source determination process performed by the sound source direction determining unit. [Figure 20] FIG. 20 is a diagram to be referred to when explaining the processing performed by the optimum NC filter calculation unit. [Figure 21] FIG. 21 is a diagram illustrating an example of the configuration of headphones according to the third embodiment. [Figure 22]FIG. 22 is a diagram illustrating an example of the configuration of a smartphone according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The description will be made in the following order. <First embodiment> <Second embodiment> <Third embodiment> <Modification> The embodiments and the like described below are preferred specific examples of the present disclosure, and the contents of the present disclosure are not limited to these embodiments and the like.

[0011] <First embodiment> [overview] First, in order to facilitate understanding of the present disclosure, an overview of the present embodiment will be described while explaining problems to be considered in the present embodiment.

[0012] In recent years, 3D audio playback via headphones has been attracting attention as a form of music playback. 3D audio can achieve a higher degree of reproducibility of the direction from which the sound source comes than conventional stereo content playback, enabling more realistic headphone playback. If there is a lot of ambient noise when listening to audio, the accuracy of 3D audio reproduction decreases due to masking from the noise. For this reason, playback using digital noise canceling (hereinafter referred to as DNC) technology is very effective.

[0013] FIG. 1 is a diagram showing the relationship between a general noise-canceling headphone and the direction from which noise comes. FIG. 1 shows an example in which a listener L listens to 3D audio using headphones 1. Noise-canceling microphones LM and RM are provided in left and right housings 2 and 3 of the headphones 1, respectively. In the method shown in FIG. 1, feed-forward noise canceling (hereinafter, appropriately referred to as FFNC) is realized using one microphone per ear. In FIG. 1, noise that may be mixed in from the surroundings of the headphones 1 is typically shown by a sinusoidal waveform and arrows.

[0014] In the case shown in FIG. 1, one microphone is provided on each side, so it is highly effective against noise from the left and right (for example, noises N1 and N2 in FIG. 1). However, for noise from the front and rear directions (for example, noises N3 to N6 in FIG. 1), the noise leaks into the ears faster than when the noise is picked up by the microphones LM and RM and a noise cancellation signal is reproduced. For this reason, the noise cannot be cancelled using an accurate anti-phase signal via the noise cancelling system, and the noise cancellation effect is deteriorated compared to the left and right directions. This effect is particularly significant as the wavelength becomes shorter as the frequency increases. In addition, since noise actually arrives from all directions, it is not possible to improve the cancellation performance for high frequencies with a general method.

[0015] Consider the case where 3D audio is played back in the system shown in Figure 1. As shown in Figure 2, if an audio object AO that has been processed to be localized to the left front of the listener L overlaps with a direction in which cancellation performance is weak, the audio object AO cannot be correctly perceived due to masking by noise. Even if the audio object AO and the direction from which the noise comes do not overlap, it is important that the head-related transfer characteristics can be correctly reproduced in order to perceive three-dimensional direction, which is a characteristic of 3D audio. If the effect of noise canceling is weak as in the frequency characteristics shown in Figure 3, and the high-frequency reproduction of the audio object AO that has been convoluted with the head-related transfer characteristics is affected by ambient noise, the reproduction of 3D audio will decrease due to the masking effect of the noise.

[0016] In consideration of this problem, as shown in Fig. 4, in headphones 1A according to this embodiment, multiple microphones LM are provided in a left housing 2A, and multiple microphones RM are provided in a right housing 3A. Surrounding noise is picked up by multiple microphones including a feedback (FB) microphone provided in the housing (headphone case), and a noise cancellation signal is generated by performing signal processing in a DNC filter block for each. The generated noise cancellation signal is then output from each of the left and right headphone drivers together with an audio signal.

[0017] Figure 5 is a diagram showing the case where 3D audio is played in a system that applies a multi-microphone FFNC, which is an FFNC that uses multiple microphones. The multi-microphone FFNC system can also deal with noise directionality, which was a weakness of single-microphone FFNC, which uses one microphone on each side. As a result, even if noise arrives from various directions, the FF microphones can pick up the noise before it reaches the ears, and the leaking noise can be cancelled out by reproducing an anti-phase signal. Since noise can be cancelled robustly from any direction, the frequency band of the cancellation effect can also be made wider.

[0018] As shown in Fig. 6, when 3D audio is played back using a multi-microphone FFNC, it is possible to robustly accommodate the direction of noise arrival, so that even if the direction of noise arrival overlaps with the placement position of the audio object AO in the same direction, the effect of masking by noise can be reduced, improving the reproduction accuracy of 3D audio. Note that the present disclosure is also effective for audio content in monaural or stereo format.

[0019] [Example of configuration of acoustic signal processing device] 7 is a diagram showing an example of the configuration of an acoustic signal processing device according to this embodiment. The acoustic signal processing device according to this embodiment is configured as headphones 1A. The headphones 1A include microphones LM1 to LM N , DNC filter 11, microphone LFB, DNC filter 12, adder 13, driver 14, adder 15, microphones RM1 to RM N , DNC filter 21, microphone RFB, DNC filter 22, adder 23, driver 24, adder 25, DNC filter 22, digital filter 31, digital filter 32, and control unit 35.

[0020] Audio data is supplied to the headphones 1A as an external input signal. The audio data is supplied by wire or wirelessly. The audio data may be music data, or may be data including only the speaker's voice. In this embodiment, the audio data will be described as 3D audio music data MS. The music data MS may be monaural audio data or stereo audio data. In addition, the headphones 1A will be described as being subject to external noise N. Examples of external noise N include noise emitted by moving objects such as airplanes and vehicles, and noise emitted by air conditioning equipment, etc.

[0021] Microphone LM1 ~ Microphone LM N(where N is any natural number) is a microphone for FFNC, and is provided in the housing 2A on the left side of the headphones 1A. When it is not necessary to distinguish between the individual microphones, they will be referred to as microphone LM as appropriate. The number and positions of the microphones LM can be any appropriate number and position, but the number and positions are preferably such that external noise N that may be mixed in from around the listener L can be detected.

[0022] DNC filter 11 is DNC filter 111 ~ DNC filter 11 N (where N is any natural number.) Each microphone LM is connected to a DNC filter 11. For example, a DNC filter 111 is connected to microphone LM1, and a DNC filter 112 is connected to microphone LM2.

[0023] The DNC filters 11 generate noise cancellation signals that have the effect of canceling external noise N and allowing the listener to hear only the sound of the audio signal when the sound output by the driver 14 reaches the listener's ear. In other words, the DNC filters 11 generate noise cancellation signals that have the opposite phase characteristics of the external noise N (audio signal picked up by the corresponding microphone LM) that reaches the listener's ear. Each DNC filter 11 outputs the generated noise cancellation signal to the adder 13.

[0024] The DNC filter 11 is configured as, for example, a finite impulse response (FIR) filter or an infinite impulse response (IIR) filter. In this embodiment, the DNC filter 11 to be used and the filter coefficients of the DNC filter 11 can be changed by control parameters generated by the control unit 35.

[0025] The microphone LFB is a feedback microphone provided inside the housing 2 A. The microphone LFB is provided near the driver 14.

[0026] The DNC filter 12 generates a noise cancellation signal for canceling the external noise N based on the audio signal input to the microphone LFB. The DNC filter 12 is configured as, for example, an FIR filter or an IIR filter. In this embodiment, the filter coefficients of the DNC filter 12 are fixed, but the filter coefficients may be changed by control parameters generated by the control unit 35.

[0027] The adder 13 adds the noise cancellation signal generated by the DNC filter 11, the noise cancellation signal generated by the DNC filter 12, and the music data MS that has been processed by the digital filter 31. The added signal is supplied to the driver 14.

[0028] The driver 14 outputs the music data MS and the noise cancellation signal supplied from the addition unit 13. The signal output from the driver 14 is supplied to the addition unit 15.

[0029] The adder 15 adds the music data MS, the noise cancellation signal, and the external noise N. As a result, the music data MS from which the external noise N has been cancelled reaches the left ear of the listener.

[0030] Microphone RM1 ~ Microphone RM N (where N is any natural number) is a microphone for FFNC, and is provided in the housing 3A on the right side of the headphones 1A. When it is not necessary to distinguish between the individual microphones, they will be referred to as microphone RM as appropriate. The number and positions of the microphones RM can be any appropriate number and location, but the number and positions are preferably such that external noise N that may be mixed in from around the listener L can be detected.

[0031] DNC filter 21 is DNC filter 211 to DNC filter 21 N (where N is any natural number.) Each microphone RM is connected to a DNC filter 21. For example, a DNC filter 211 is connected to microphone RM1, and a DNC filter 212 is connected to microphone RM2.

[0032] The DNC filters 21 generate noise cancellation signals that have the effect of canceling external noise N and allowing the listener to hear only the sound of the audio signal when the sound output by the driver 24 reaches the listener's ear. In other words, the DNC filters 21 generate noise cancellation signals that have the opposite phase characteristics of the external noise N (audio signal picked up by the corresponding microphone RM) that reaches the listener's ear. Each DNC filter 21 outputs the generated noise cancellation signal to the adder 23.

[0033] The DNC filter 21 is configured as, for example, an FIR filter or an IIR filter. In this embodiment, the DNC filter 21 to be used and the filter coefficients of the DNC filter 21 can be changed by control parameters generated by the control unit 35.

[0034] The microphone LRB is a feedback microphone provided inside the housing 3 A. The microphone RFB is provided near the driver 24.

[0035] The DNC filter 22 generates a noise cancellation signal for canceling the external noise N based on the audio signal input to the microphone RFB. The DNC filter 22 is configured as, for example, an FIR filter or an IIR filter. In this embodiment, the filter coefficients of the DNC filter 22 are fixed, but the filter coefficients may be changed by control parameters generated by the control unit 35.

[0036] The adder 23 adds together the noise cancellation signal generated by the DNC filter 21, the noise cancellation signal generated by the DNC filter 22, and the music data MS that has been processed by the digital filter 32. The added signal is supplied to the driver 24.

[0037] The driver 24 outputs the music data MS and the noise cancellation signal supplied from the addition unit 23. The signal output from the driver 24 is supplied to an addition unit 25.

[0038] The adder 25 adds the music data MS, the noise cancellation signal, and the external noise N. As a result, the music data MS from which the external noise N has been cancelled reaches the right ear of the listener.

[0039] The digital filters 31 and 32 process an external input signal (music data MS in this embodiment). The digital filters 31 and 32 are filters having, for example, an equalizing function for changing the frequency characteristics of the music data MS converted into a digital format by an A / D (Analog to Digital) conversion unit (not shown), and a rendering function for localizing an audio object at a predetermined position by appropriately controlling the phase and delay of the audio object. The filter characteristics such as the filter coefficients of the digital filters 31 and 32 are set by the control parameters in the control unit 35.

[0040] The control unit 35 generates and supplies control parameters for the DNC filters 11 and 21 to control the operation of the DNC filters 11 and 21. The control unit 35 also generates and supplies control parameters for the digital filters 31 and 32 to control the operation of the digital filters 31 and 32.

[0041] In this embodiment, the microphones correspond to microphones LM, LFB, RM, and RFB. DNC filters 11, 12, and DNC filters 21, 22 are provided for each microphone and correspond to a noise cancellation processing unit that generates a signal for canceling noise based on an input audio signal picked up by each microphone. Although not shown, the headphones 1A may have a gain adjustment unit that adjusts the volume.

[0042] [Headphone operation example] Next, an operation example of the headphones 1A will be described. A noise cancellation signal for canceling the external noise N is generated by the DNC filter 11 based on an input audio signal picked up by the microphone LM. Also, a noise cancellation signal for canceling the external noise N is generated by the DNC filter 21 based on an input audio signal picked up by the microphone RM.

[0043] The noise cancellation signal is added to the music data MS to cancel the external noise N. Therefore, a sound corresponding to the music data MS from which the external noise N has been cancelled is reproduced for the listener.

[0044] According to the first embodiment described above, multiple microphones are arranged on the headphone housing, so that even if noise arrives from various directions, the noise can be effectively cancelled.

[0045] <Second embodiment> Next, the second embodiment will be described. In the description of the second embodiment, the same or similar configurations in the above description are given the same reference symbols, and duplicated descriptions are omitted as appropriate. In addition, unless otherwise specified, the matters described in the first embodiment can be applied to the second embodiment.

[0046] [overview] 8A and 8B are diagrams for explaining an outline of the second embodiment. For example, as shown in FIG. 8A, consider a case where external noise N coming from the right side is dominant with respect to the listener L. The music data MS is a 3D audio content, and a predetermined audio object is localized at a predetermined position VP1 on the right side of the listener L. In this way, if the localization position of the audio object, that is, the sound source direction, is the same direction as the noise arrival direction, there is a risk that the clarity and localization feeling of the reproduced sound will be impaired. Therefore, in this embodiment, the sound source direction is dynamically changed. Specifically, as shown in FIG. 8B, the position where the music data MS is localized is changed to a position VP2 in a direction with less noise, thereby improving the localization feeling and clarity of the reproduced sound. Hereinafter, this embodiment will be described in detail.

[0047] [Headphone configuration example] (Overall configuration example) 9 is a diagram showing an example of the configuration of headphones (headphones 1B) according to the second embodiment. The headphones 1B differ in configuration from the headphones 1A according to the first embodiment in that the headphones 1B includes an analysis unit 41 connected to the control unit 35. The analysis unit 41 is supplied with an audio signal collected by the microphone LM, an audio signal collected by the microphone RM, and music data MS. The analysis unit 41 analyzes the audio signals from the microphones LM and RM and an external input signal.

[0048] (Example of analysis unit configuration) 10 is a diagram showing an example of the configuration of the analysis unit 41. The analysis unit 41 has, for example, a noise arrival direction estimation unit 401, an audio object optimum arrangement position calculation unit 402, and an optimum NC filter calculation unit 403.

[0049] An audio signal corresponding to external noise N picked up by the microphones LM and RM is input to the noise arrival direction estimation unit 401. The noise arrival direction estimation unit 401 generates noise arrival direction information indicating the noise arrival direction based on the audio signal input thereto. Specifically, the noise arrival direction information is an index indicating the intensity of noise from each of a plurality of directions. The noise arrival direction information is supplied to each of an audio object optimum arrangement position calculation unit 402 and an optimum NC filter calculation unit 403.

[0050] The audio object optimum placement position calculation unit 402 calculates the optimum placement position of the audio object based on the noise arrival direction information. Although details will be described later, the audio object optimum placement position calculation unit 402 also refers to information described in meta information corresponding to the audio object to calculate the optimum placement position of the audio object.

[0051] Based on the noise arrival direction information, the optimum NC filter calculation unit 403 calculates optimum control parameters for the DNC filters 11 and 21. Then, the optimum NC filter calculation unit 403 outputs the calculation result to the control unit .

[0052] (Noise arrival direction estimation section) Next, a specific example of the process performed by the noise arrival direction estimation unit 401 will be described. Fig. 11 is a diagram for explaining an example of the noise arrival direction to be searched. As shown in Fig. 11, a horizontal angle θ and an elevation angle φ are defined with a listener L using headphones 1B as the center. The noise arrival direction estimation unit 401 calculates the noise intensity for each three-dimensional direction while changing the horizontal angle θ and the elevation angle φ, and generates noise arrival direction information based on the calculation result.

[0053] 12 is a diagram showing a specific example of the configuration of the noise arrival direction estimation unit 401. The noise arrival direction estimation unit 401 includes filters 45 (filters 451 to 45) corresponding to three-dimensional directions. N(where N is a natural number). For example, filter 451 is a filter that directs zero sensitivity directivity in the vertical 90 degree direction, filter 452 is a filter that directs zero sensitivity directivity in the horizontal angle 0 degree and elevation angle 0 degree direction, and filter 453 is a filter that directs zero sensitivity directivity in the horizontal angle 30 degree and elevation angle 0 degree direction. Audio signals collected by microphones LM and RM are input to each filter constituting filter 45.

[0054] The output of the filter 45 is supplied to the dB calculation unit 46. The dB calculation unit 46 calculates the level (dB) of the input audio signal. The calculation results of each filter are supplied to the average calculation unit 47, which calculates the average value. Then, the adder 48 calculates the difference from the average value, and the result is set as the noise intensity in the three-dimensional direction corresponding to the specified filter. For example, the output of the filter 451 is supplied to the dB calculation unit 461. The calculation result of the dB calculation unit 461 is supplied to the average calculation unit 47 and the adder 48. The adder 48 calculates the difference between the output of the dB calculation unit 461 and the output of the average calculation unit 47. The output of the adder 48 becomes the noise intensity index corresponding to the direction corresponding to the filter 451, that is, φ=90 degrees. In this manner, the noise intensity index corresponding to each three-dimensional direction is obtained.

[0055] The noise arrival direction estimating unit 401 generates noise arrival direction information based on the obtained noise intensity index. Fig. 13 is a diagram showing an example of the noise arrival direction information. As shown in Fig. 13, the noise arrival direction information specifies the noise level corresponding to a predetermined horizontal angle θ and elevation angle φ.

[0056] (Audio object optimal placement position calculation part) As shown in FIG. 14, the audio object optimum placement position calculation unit 402 has a sound source direction determination unit 402A and a filter coefficient conversion unit 402B.

[0057] The sound source direction determination unit 402A determines the direction in which the audio object is localized, that is, the sound source direction, based on the noise arrival direction information. The sound source direction may be a specific position in three dimensions or may be specified as a direction relative to the listener L. The sound source direction determination unit 402A supplies the determined sound source direction to the filter coefficient conversion unit 402B.

[0058] The filter coefficient conversion unit 402B converts the sound source direction determined by the sound source direction determination unit 402A into a filter coefficient by performing a conversion process on the sound source direction determined by the sound source direction determination unit 402A. For example, the filter coefficient conversion unit 402B holds filter coefficients corresponding to a plurality of sound source directions as a table. The filter coefficient conversion unit 402B reads out the filter coefficient corresponding to the sound source direction supplied from the sound source direction determination unit 402A from the table. The filter coefficient conversion unit 402B then supplies the read filter coefficient to the control unit 35. The control unit 35 sets the filter coefficient as a control parameter in the digital filters 31 and 32. As a result, the audio object is localized in the sound source direction determined by the audio object optimal arrangement position calculation unit 402.

[0059] Hereinafter, several examples of the sound source determination process performed by the sound source direction determination unit 402A will be described. Fig. 15 is a flowchart for explaining a first example of the sound source determination process performed by the sound source direction determination unit 402A. The first example (pattern PT1) is an example in which a single audio object is played back. The meta information corresponding to the audio object includes an identification number of the audio object, a sound source direction recommended for playback (recommended playback position information), and changeability information indicating whether the sound source direction may be changed from the recommended playback position information.

[0060] In step ST11, the sound source direction determining unit 402A judges whether or not the change of the sound source direction is permitted based on the change permission information of the meta information. If the judgment result is No, the process proceeds to step ST12.

[0061] In step ST12, since the sound source direction cannot be changed, the sound source direction determining unit 402A outputs the recommended sound source direction to the filter coefficient converting unit 402B. As a result, the audio object is played back in the recommended sound source direction. Then, the process ends.

[0062] If the result of the determination in step ST13 is Yes, the process proceeds to step ST13. In step ST13, a calculation is performed to convolute the noise arrival direction information with a predetermined smoothing filter. Then, the process proceeds to step ST14.

[0063] In step ST14, the sound source direction determining unit 402A outputs the direction (θ, φ) in which the noise intensity index is the smallest based on the smoothed noise arrival direction information. As a result, the audio object is played back in the direction (θ, φ) in which the noise intensity index is the smallest. Then, the process ends.

[0064] It should be noted that the calculation of convolution with a smoothing filter in step ST13 does not have to be performed.

[0065] 16 is a flowchart for explaining a second example of the sound source determination process performed by the sound source direction determining unit 402A. The second example (pattern PT2) is an example of the case where a plurality of audio objects whose relative positions are determined are played back.

[0066] The meta information includes an identification number for identifying a group of audio objects, recommended playback position information for a reference audio object (also referred to as a reference object, if appropriate), changeability information, and a list of partial audio objects belonging to the same group. The partial audio object list includes an identification number for identifying each partial audio object (also referred to as a partial object, if appropriate), and the relative sound source direction of the partial object (relative angle from the playback position of the reference object).

[0067] In step ST21, the sound source direction determining unit 402A judges whether or not the change of the sound source direction of the group of audio objects is permitted based on the change permission information of the meta information. If the judgment result is No, the process proceeds to step ST22.

[0068] In step ST22, the sound source direction of the reference object is set to the sound source direction indicated by the recommended playback position information, and the process proceeds to step ST26.

[0069] The meta information describes the relative sound source direction of the partial object. Therefore, since the sound source direction of the reference object is set, it is possible to determine the sound source direction of the partial object as well. Therefore, in step ST26, the sound source direction determination unit 402A outputs a list indicating the sound source directions of the reference object and all partial objects. The reference object and each partial object are played back in the sound source direction indicated in the list. Then, the process ends.

[0070] If the determination result in step ST21 is Yes, the process proceeds to step ST23. In step ST23, a convolution operation is performed on the noise arrival direction information. For example, FIG. 17A shows an example of the noise arrival direction information. A smoothing comb filter as shown in FIG. 17B is prepared for each time the relative sound source direction (θ, φ) of the partial object with respect to the reference object is set to (120, 0). The smoothing comb filter is a two-dimensional filter that has positive values ​​only around the angle of the relative sound source direction of the partial object. The two-dimensional filter is subjected to a circular convolution operation on the noise arrival direction information. Then, the process proceeds to step ST24.

[0071] In step ST24, the sound source direction determining unit 402A sets the sound source direction of the reference object to the direction (θ, φ) in which the noise intensity index is minimum in the calculated noise arrival direction information, and then the process proceeds to step ST25.

[0072] In step ST25, since the sound source direction of the reference object is set, the sound source direction of the partial object is set to (sound source direction angle of the reference object+relative angle of the partial object).Then, the process proceeds to step ST26.

[0073] In step ST26, the sound source direction determining unit 402A outputs a list of the sound source directions of the reference object and all the partial objects. The reference object and each partial object are played back in the sound source direction indicated in the list. Then, the process ends.

[0074] 18 is a flowchart for explaining a third example of the sound source determination process performed by the sound source direction determining unit 402A. The third example (pattern PT3) is an example in which a plurality of audio objects are arranged.

[0075] An order is defined for each of the multiple audio objects. The order may be random or may be a priority order based on the importance of the audio objects. For example, the importance of an audio object is high for an audio object such as a person's voice, and low for an audio object such as background music. In addition, when the content type is described in the meta information, the order may be based on the content type. For example, a priority order may be defined for each content type in advance, and the audio objects may be sorted according to the priority order.

[0076] In step ST31, the sound source direction determining unit 402A determines the order in which to process audio objects or audio object groups (hereinafter, appropriately abbreviated as audio objects or the like), and then the process proceeds to step ST32.

[0077] In step ST32, a loop of processing related to audio objects, etc. is started, and the process then proceeds to step ST33.

[0078] In step ST33, the sound source direction determining unit 402A performs the process related to the above-mentioned pattern PT1 or pattern PT2 for each audio object etc. in the order corresponding to the determined order. Then, the process proceeds to step ST34.

[0079] In step ST34, the noise coming direction information is updated every time the playback position of a predetermined audio object is determined. FIG. 19A shows the noise coming direction information before updating. FIG. 19B shows an example of a smoothing filter convoluted with the noise coming direction information. For example, assume that the arrangement position of a predetermined audio object is set to about 70 degrees by the process related to pattern PT2. The average level of the audio object corresponding to the angle is obtained (FIG. 19C). This average level is added to the noise coming direction information shown in FIG. 19A (FIG. 19D). In the next process, the updated noise coming direction information is used. This process makes it possible to reflect the change in the noise coming direction information accompanying the rearrangement of the audio object in the process related to each pattern. Then, the process proceeds to step ST35.

[0080] In step ST35, it is determined whether or not there are no more audio objects to be processed, If there are no more audio objects to be processed, the process ends.

[0081] (Optimal NC filter calculation section) The optimum NC filter calculation unit 403 uses the noise arrival direction information and meta information to select the optimum filter coefficient for canceling noise and the DNC filter 11 to be operated. For example, the optimum NC filter calculation unit 403 calculates the DNC filter 11 to be operated and the noise canceling strength for each DNC filter 11 based on the noise arrival direction information. Then, as shown in FIG. 20, the optimum NC filter calculation unit 403 generates optimum control parameters based on the calculation results. The control parameters generated by the optimum NC filter calculation unit 403 are set in an appropriate DNC filter 11 by the control unit 35. Note that although the DNC filter 11 is shown in FIG. 20, the optimum NC filter calculation unit 403 also performs the same process for the DNC filter 21.

[0082] Furthermore, if the residual noise in the listener's ears is e(t), the residual noise in the listener's ears can be expressed by the following formula (1). (In formula (1), l(t) is the leakage noise measured in advance, x m (t) is the microphone input for all FFNC, f m (t) is the characteristic of the DNC filter, and d(t) is the acoustic characteristic inside the headphones.)

[0083]

number

[0084] The optimum NC filter calculation unit 403 may calculate the control parameters for the DNC filters 11 and 21 so as to minimize the residual noise in the ear in the formula (1).

[0085] <Third embodiment> Next, the third embodiment will be described. In the description of the third embodiment, the same or similar configurations in the above description are given the same reference symbols, and duplicated descriptions are omitted as appropriate. In addition, unless otherwise specified, the matters described in the first and second embodiments can be applied to the third embodiment.

[0086] The third embodiment is generally an embodiment in which part of the processing that has been performed by the headphones is performed by an external device (for example, a smartphone or a server device that can communicate with the headphones).

[0087] [Headphone configuration example] 21 is a diagram showing an example of the configuration of headphones (headphones 1C) according to the third embodiment. The headphones 1C have a communication unit 51 and a storage unit 52 such as a memory. The headphones 1C also have only a noise arrival direction estimation unit 401 among the functional blocks of the analysis unit 41. The headphones 1C do not have the digital filters 31 and 32. However, the headphones 1C have EQs 53 and 54 that perform an equalizing function among the functions of the digital filters 31 and 32.

[0088] The communication unit 51 has a modulation / demodulation circuit, an antenna, and the like corresponding to the communication method. The communication is assumed to be wireless communication, but may be wired communication. Examples of wireless communication include LAN (Local Area Network), Bluetooth (registered trademark), Wi-Fi (registered trademark), and WUSB (Wireless USB). The headphones 1C are paired with an external device such as a smartphone through communication performed by the communication unit 51.

[0089] [Example of smartphone configuration] 22 is a diagram showing a configuration example of a smartphone 81, which is an example of an external device. The smartphone 81 has a CPU (Central Processing Unit) 82, a DSP (Digital Signal Processor) 83, a first communication unit 84, a second communication unit 85, (an audio object optimum arrangement position calculation unit and an optimum NC filter calculation unit) 86, an object filter control circuit 87, and a storage unit 88. The DSP 83 has digital filters 83A and 83B.

[0090] The CPU 82 performs overall control of the smartphone 81. Digital filters 83A and 83B included in the DSP 83 perform, for example, a rendering process for localizing an audio object at a predetermined position.

[0091] The first communication unit 84 communicates with the server device 71. Through this communication, data of an audio object is downloaded from the server device 71 to the smartphone 81.

[0092] The second communication unit 85 communicates with the communication unit 51 of the headphones 1C. Through this communication, noise arrival direction information is supplied from the headphones 1C to the smartphone 81. In addition, an audio object that has been subjected to processing described later is supplied from the smartphone 81 to the headphones 1C.

[0093] An (audio object optimum placement position calculation unit and optimum NC filter calculation unit) 86 has the functions of the audio object optimum placement position calculation unit 402 and optimum NC filter calculation unit 403 described above.

[0094] The object filter control circuit 87 is a circuit for setting filter coefficients for realizing the placement positions of the audio objects calculated by the (audio object optimum placement position calculation unit and optimum NC filter calculation unit) 86 in the digital filters 83A and 83B.

[0095] The storage unit 88 is a storage unit that stores various data. For example, filter coefficients for realizing the arrangement positions of audio objects are stored in the storage unit 88. Examples of the storage unit 88 include a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, and a magneto-optical storage device.

[0096] [Processing between headphones and smartphone] Next, processing is performed between the headphones 1C and the smartphone 81. First, short-range wireless communication or the like is performed between the headphones 1C and the smartphone 81, whereby the headphones 1C and the smartphone 81 are paired.

[0097] The headphones 1C generate noise coming direction information as described in the second embodiment. The noise coming direction information is supplied from the communication unit 51 of the headphones 1C to the second communication unit 85 of the smartphone 81. The noise coming direction information is supplied from the second communication unit 85 to (an audio object optimum arrangement position calculation unit and an optimum NC filter calculation unit) 86.

[0098] A first communication unit 84 of the smartphone 81 communicates with the server device 71 to acquire audio objects and meta information corresponding to the audio objects from the server device 71. The data of the audio objects is supplied to a DSP 83, and the meta information is supplied to an audio object optimum arrangement position calculation unit and an optimum NC filter calculation unit 86.

[0099] An (audio object optimum placement position calculation unit and optimum NC filter calculation unit) 86 determines the placement position (sound source direction) of the audio object in the same manner as in the second embodiment. The (audio object optimum placement position calculation unit and optimum NC filter calculation unit) 86 supplies the determined sound source direction to an object filter control circuit 87. The object filter control circuit 87 reads out filter coefficients for realizing the sound source direction from a storage unit 88, and sets the read out coefficients in each of the digital filters 83A and 83B.

[0100] Digital filters 83A and 83B perform filtering on the audio object data, and the processed data is transmitted to headphones 1C via second communication unit 85. In addition, optimal control parameters for DNC filters 11 and 21 calculated by (audio object optimal placement position calculation unit and optimal NC filter calculation unit) 86 are transmitted to headphones 1C via second communication unit 85.

[0101] The data of the audio object received by the communication unit 51 of the headphones 1C is subjected to an equalization process by the EQ 53 and then supplied to the addition unit 13. Also, the data of the audio object received by the communication unit 51 of the headphones 1C is subjected to an equalization process by the EQ 54 and then supplied to the addition unit 23.

[0102] Moreover, optimal control parameters for the DNC filters 11, 21 received by the communication unit 51 of the headphones 1C are supplied to the control unit 35 and set for each of the DNC filters 11, 21. Other processing is similar to the processing described in the first or second embodiment.

[0103] As described above, a part of the headphones described in the first or second embodiment may be implemented by an external device such as a smartphone. That is, the acoustic signal processing device according to the present disclosure is not limited to headphones, but may be realized by an electronic device such as a smartphone. Note that the functions to be performed by the external device may be changed as appropriate. For example, in the third embodiment described above, the smartphone 81 may have the function of the noise arrival direction estimation unit 401 that generates noise arrival direction information.

[0104] <Modification> Although several embodiments of the present disclosure have been specifically described above, the contents of the present disclosure are not limited to the above-described embodiments, and various modifications based on the technical ideas of the present disclosure are possible.

[0105] The arrangement of the above-mentioned headphone configurations in the headphones can be changed as appropriate. For example, in the case of on-ear type, overhead type headphones, or neckband type headphones, a circuit configuration such as a digital filter, a control unit, an analysis unit, etc. is mounted in the housing of either the left or right L side or the right side, and data is transmitted to and received from the side not mounted with the circuit configuration via a data cable connecting both housings. In addition, in neckband type headphones, the circuit may be mounted in one housing as described above, or the circuit configuration such as the control unit, the analysis unit, etc. may be arranged in the neckband part. On the other hand, in so-called left and right independent canal type or open ear type headphones, although not shown, it is desirable to mount circuits such as a digital filter, a control unit, an analysis unit, etc. independently on both the left and right sides.

[0106] The above-mentioned DNC filter, digital filter, and EQ 53 can be configured as a part of the DSP. Also, the control unit and the analysis unit can be configured as a part of the circuit of the DSP or the processor, or can be configured to operate by a computer program (software) that operates on the DSP or the processor.

[0107] The configurations, methods, steps, shapes, materials, and values ​​given in the above-mentioned embodiment and modified examples are merely examples, and different configurations, methods, steps, shapes, materials, and values ​​may be used as necessary, and may be replaced with known ones. Furthermore, the configurations, methods, steps, shapes, materials, and values ​​in the embodiment and modified examples may be combined with each other to the extent that no technical contradiction occurs.

[0108] It should be noted that the contents of the present disclosure should not be construed as being limited to the effects exemplified in this specification.

[0109] The present disclosure may also have the following configurations. (1) a noise cancellation processing unit provided for each of the plurality of microphones, the noise cancellation processing unit generating a signal for canceling noise based on an input audio signal from the microphone; Has a digital filter to process external input signals Acoustic signal processing device. (2) A control unit that generates a control parameter for the noise cancellation processing unit; an analysis unit that analyzes an input audio signal from the microphone; have An acoustic signal processing device as described in (1). (3) The analysis unit has a noise arrival direction estimation unit that generates noise arrival direction information that is information indicating the arrival direction of noise based on the input voice signal from the microphone. An acoustic signal processing device as described in (2). (4) the noise cancellation processing unit further generates a control parameter for the digital filter, and the analysis unit further analyzes the external input signal, the external input signal including an audio object and meta information corresponding to the audio object; The audio object optimum placement position calculation unit includes an audio object optimum playback position calculation unit that calculates an optimum playback position of the audio object based on the noise arrival direction information. An acoustic signal processing device according to (3). (5) the audio object is a single audio object, The meta information includes recommended playback position information and changeability information indicating whether or not a direction of a sound source can be changed; When the changeability information is changeable, a process is performed to play the audio object at the optimal playback position, and when the changeability information is not changeable, a process is performed to play the audio object at the recommended playback position. An acoustic signal processing device according to (4). (6) The audio object includes a plurality of audio objects having relative playback positions defined; The analysis unit calculates the optimal playback position based on the noise arrival direction information so that a noise intensity index for the plurality of audio objects is minimized. An acoustic signal processing device according to (4). (7) The audio object includes a plurality of audio objects having a specified order; When the changeability information is changeable, a process is performed to play the audio object at the optimal playback position, and when the changeability information is not changeable, a process is performed to play the audio object at the recommended playback position. The processing is performed in a sequence corresponding to the order, and the noise arrival direction information is updated every time the processing is performed. An acoustic signal processing device according to (4). (8) The order is any one of a random order, an order based on priority, and an order based on the type of content. An acoustic signal processing device according to (7). (9) The analysis unit generates optimal control parameters for the noise cancellation processing unit based on the noise arrival direction information. An acoustic signal processing device according to any one of (3) to (8). (10) The digital filter performs processing to localize the audio object at a predetermined position. An acoustic signal processing device according to any one of (4) to (8). (11) The plurality of microphones An acoustic signal processing device according to any one of (1) to (10). (12) The plurality of microphones includes a feedforward microphone and a feedback microphone. An acoustic signal processing device according to (11). (13) The external input signal is audio data supplied via a wired or wireless connection. An acoustic signal processing device according to any one of (1) to (12). (14) Configured as a headphone device An acoustic signal processing device according to any one of (1) to (13). (15) a noise cancellation processing unit provided for each of the plurality of microphones, generating a signal for canceling noise based on an input audio signal from the microphone; Digital filters process external input signals Acoustic signal processing method. (16) a noise cancellation processing unit provided for each of the plurality of microphones, generating a signal for canceling noise based on an input audio signal from the microphone; Digital filters process external input signals A program for causing a computer to execute an acoustic signal processing method. [Explanation of symbols]

[0110] 1A, 1B, 1C... Headphones 11,12,21,22...DNC Filter 31, 32... Digital filter 35 Control section 41...Analysis Department 401 Noise direction estimation unit 402 Audio object optimal placement position calculation unit 403 Optimal NC filter calculation part LM, RM...Microphone

Claims

1. a noise cancellation processing unit provided for each of the plurality of microphones, the noise cancellation processing unit generating a signal for canceling noise based on an input audio signal from the microphone; A control unit that generates a control parameter for the noise cancellation processing unit; an analysis unit that analyzes an input audio signal from the microphone; A digital filter for processing an external input signal received from an external device; An acoustic signal processing device comprising: the external input signal includes audio objects and meta information corresponding to the audio objects; the analysis of the input audio signal includes noise arrival direction information generated based on the input audio signal from the microphone and changing a playback position of the audio object based on the meta information; The control unit further generates a control parameter for the digital filter. Acoustic signal processing device.

2. the audio object is a single audio object, and the meta information includes playback position information and changeability information indicating whether a direction of a sound source can be changed; If the changeability information is changeable, a process is performed to play the audio object at the changed playback position, and if the changeability information is not changeable, a process is performed to play the audio object at the playback position of the meta information. The audio signal processing device according to claim 1 .

3. The audio object includes a plurality of audio objects, each of which has a relative sound source direction from a reference object defined in the meta information; The analysis of the input audio signal changes the playback position based on the noise arrival direction information so as to minimize a noise intensity index for the plurality of audio objects. The audio signal processing device according to claim 1 .

4. The audio object includes a plurality of audio objects having a specified order; If the changeability information is changeable, a process is performed to play the audio object at the changed playback position; If the changeability information is not changeable, the audio object is reproduced at the reproduction position of the meta information. The processing is performed in a sequence corresponding to the order, and the noise arrival direction information is updated every time the processing is performed. The audio signal processing device according to claim 2 .

5. The order is any one of a random order, an order based on priority, and an order based on the type of content. The audio signal processing device according to claim 4 .

6. The analysis unit generates a control parameter for the noise cancellation processing unit based on the noise arrival direction information. The audio signal processing device according to claim 1 .

7. The digital filter performs processing to localize the audio object at a predetermined position. The audio signal processing device according to claim 1 .

8. The plurality of microphones The audio signal processing device according to claim 1 .

9. The plurality of microphones includes a feedforward microphone and a feedback microphone. The audio signal processing device according to claim 8 .

10. The external input signal is audio data supplied via a wired or wireless connection. The audio signal processing device according to claim 1 .

11. Configured as a headphone device The audio signal processing device according to claim 1 .

12. a noise canceling processor provided for each of the plurality of microphones generates a signal for canceling noise based on an input audio signal from the microphone; A processor generates control parameters for the noise cancellation processing unit and analyzes an input audio signal from the microphone; A digital filter processes an external input signal received from the outside.

1. A method for processing an acoustic signal, comprising: the external input signal includes audio objects and meta information corresponding to the audio objects; The processor further changes a playback position of the audio object based on the meta information and noise arrival direction information generated based on the input audio signal from the microphone, and generates a control parameter for the digital filter. Acoustic signal processing method.

13. a noise canceling processor provided for each of the plurality of microphones generates a signal for canceling noise based on an input audio signal from the microphone; A processor generates control parameters for the noise cancellation processing unit and analyzes an input audio signal from the microphone; A digital filter processes an external input signal received from the outside. A program for causing a computer to execute an acoustic signal processing method, the external input signal includes audio objects and meta information corresponding to the audio objects; The processor further changes a playback position of the audio object based on the meta information and noise arrival direction information generated based on the input audio signal from the microphone, and generates a control parameter for the digital filter. A program for causing a computer to execute an acoustic signal processing method.

Images