Acoustic signal processing device, acoustic signal processing method, and program

Abstract

The acoustic signal processing device includes: a noise cancellation processing unit configured for each of a plurality of microphones, and generating a signal for noise cancellation based on input audio signals from the microphones; and a digital filter for processing external input signals.

Description

Technical Field

[0001] This disclosure relates to acoustic signal processing apparatus, acoustic signal processing methods and programs. Background Technology

[0002] Techniques for eliminating external noise when reproducing music using headphone devices are known, or techniques involving so-called noise cancellation (see, for example, Patent Document 1).

[0003] [List of Citations]

[0004] [Patent Literature]

[0005] [Patent Document 1]

[0006] Japanese Patent Publication No. 2007-25918 Summary of the Invention

[0007] [Technical Issues]

[0008] In this field, it is desirable to eliminate external noise that leaks into headphone devices.

[0009] The purpose of this disclosure is to provide an acoustic signal processing apparatus, method, and program for effectively eliminating noise.

[0010] [Solution to the problem]

[0011] For example, this disclosure is an acoustic signal processing apparatus, including: a noise cancellation unit configured for each of a plurality of microphones and generating a signal for noise cancellation based on an acoustic signal input from the respective microphone; and a digital filter for processing external input signals.

[0012] For example, this disclosure is an acoustic signal processing method, which includes generating a noise cancellation signal based on an audio signal input from a corresponding microphone among a plurality of microphones by a noise cancellation unit configured for each microphone, and processing an external input signal by a digital filter.

[0013] For example, this disclosure is a program for causing a computer to perform an acoustic signal processing method, including: generating a signal for eliminating noise based on an acoustic signal input from a corresponding microphone among a plurality of microphones by means of a noise cancellation unit configured for each microphone; and processing an external input signal by means of a digital filter. Attached Figure Description

[0014] Figure 1 This is a diagram illustrating the relationship between typical noise-cancelling headphones and the direction of noise arrival.

[0015] Figure 2This is a diagram for reference when explaining the issues to be considered in this disclosure.

[0016] Figure 3 This is a diagram to refer to when explaining how the masking effect provided by noise degrades the reproducibility of 3D audio.

[0017] Figure 4 These are illustrations to be referenced when providing a summary description of the implementation method.

[0018] Figure 5 These are illustrations to be referenced when providing a summary description of the implementation method.

[0019] Figure 6 These are illustrations to be referenced when providing a summary description of the implementation method.

[0020] Figure 7 This is a diagram depicting an example configuration of headphones according to the first embodiment.

[0021] Figure 8 A and Figure 8 B is a diagram illustrating the outline of the second embodiment.

[0022] Figure 9 This is a diagram illustrating an example configuration of the headphones according to the second embodiment.

[0023] Figure 10 This is a diagram illustrating an example configuration of the analysis unit according to the second embodiment.

[0024] Figure 11 This is an example diagram used to illustrate the direction of arrival of the noise to be searched.

[0025] Figure 12 This is a diagram illustrating a specific configuration example of the noise arrival direction estimation unit.

[0026] Figure 13 This is an example diagram illustrating the direction of arrival information for noise.

[0027] Figure 14 This is a diagram illustrating a specific configuration example of the optimal audio object placement calculation unit.

[0028] Figure 15 This is a flowchart illustrating the first embodiment of the sound source determination process performed by the sound source direction determination unit.

[0029] Figure 16 This is a flowchart illustrating a second example of the sound source determination process performed by the sound source direction determination unit.

[0030] Figure 17 A and Figure 17Figure B is the diagram referred to when explaining the second example of the sound source determination process performed by the sound source direction determination unit.

[0031] Figure 18 This is a flowchart illustrating a third example of the sound source determination process performed by the sound source direction determination unit.

[0032] Figure 19 A to Figure 19 Figure D is a diagram used to illustrate the third embodiment of the sound source determination process performed by the sound source direction determination unit.

[0033] Figure 20 This is a diagram used to explain the processing performed by the optimal NC filter calculation unit.

[0034] Figure 21 This is a diagram illustrating an example configuration of headphones according to a third embodiment.

[0035] Figure 22 This is a schematic diagram illustrating a configuration example of a smartphone according to a third embodiment. Detailed Implementation

[0036] In the following description, embodiments for carrying out this disclosure will be illustrated with reference to the accompanying drawings. It should be noted that the explanations will be given in the following order.

[0037] <First Implementation Method>

[0038] <Second Implementation Method>

[0039] <Third Implementation Method>

[0040] <Variation Example>

[0041] The embodiments described below are preferably examples of this disclosure. However, this disclosure is not limited to these embodiments.

[0042] <First Implementation Method>

[0043] <Invention>

[0044] First, in order to facilitate understanding of the present invention, an outline of this embodiment will be described, and the problems to be considered in this embodiment will be explained.

[0045] In recent years, headphone reproduction of music and 3D audio has attracted attention. Compared with traditional stereoscopic content reproduction, headphone reproduction of 3D audio can provide a more realistic experience due to its enhanced reproducibility in the direction of sound source arrival. However, if there is significant ambient noise during audio listening, the accuracy of 3D audio reproduction deteriorates due to the masking effect of the noise. Therefore, reproduction using digital noise cancellation (hereinafter appropriately referred to as DNC) technology is highly effective.

[0046] Figure 1 This is a diagram illustrating the relationship between typical noise-cancelling headphones and the direction of noise arrival. Figure 1 In the example, listener L uses headphones 1 to listen to 3D audio. The left shell 2 and right shell 3 of headphones 1 are equipped with noise-canceling microphones LM and RM, respectively. Figure 1 In this process, one microphone per ear is used to perform feedforward noise cancellation (appropriately referred to as FFNC below). Figure 1 In the diagram, each sine wave and arrow schematically represent noise that can intrude into the ear from the surrounding environment of the earphone 1.

[0047] exist Figure 1 In the case described, the configuration of setting one microphone for each of the left and right sides is suitable for noise from the left and right sides (e.g., Figure 1 The noise (N1 and N2) in the signal is very effective. However, before the microphone LM and RM have finished collecting the noise and reproducing the noise-cancelled signal, noise from the front and rear directions (e.g., Figure 1 Noise (N3 to N6) reaches the inside of each ear. Therefore, noise cancellation using a suitable inverted signal by a noise cancellation system cannot be performed. Noise cancellation in the front-to-back direction is less effective than in the left-to-right direction. In particular, this effect becomes more pronounced at higher frequencies with shorter wavelengths. Furthermore, since the actual noise arrives from all directions, a typical system cannot enhance cancellation performance at high frequencies.

[0048] The discussion here is approved. Figure 1 The system described herein reproduces 3D audio. For example... Figure 2 As shown, when an audio object AO, which has been processed to be positioned to the left front of the listener L, is superimposed in a direction with low cancellation performance, the audio object AO is masked by noise and therefore cannot be accurately perceived. Furthermore, in order to achieve perception in three-dimensional directions as a characteristic of 3D audio, it is important to properly reproduce the head transmission characteristics, regardless of whether the audio object AO is superimposed in the direction of noise arrival. If provided... Figure 3 The high-frequency range reproduction of the audio object AO, which is shown as a low-noise cancellation effect and whose head transmission characteristics have been convolved, is affected by ambient noise. Therefore, the reproducibility of 3D audio is degraded due to the noise masking effect.

[0049] In view of the above problems, such as Figure 4As shown, the earphone 1A according to this embodiment is equipped with multiple microphones LM in the left housing 2A and multiple microphones RM in the right housing 3A. The multiple microphones, including a feedback (FB) microphone disposed inside the housing (earphone housing), collect ambient noise, and a noise cancellation signal is generated by performing signal processing on the noise at a DNC filter block. The generated noise cancellation signal and the audio signal are output from the left and right earphone drivers.

[0050] Figure 5 This diagram illustrates the reproduction of 3D audio by a multi-microphone FFNC system using multiple microphones. The multi-microphone FFNC system addresses the directionality of noise, a weakness of single-microphone FFNC, which uses one microphone on each of the left and right sides. Therefore, even when noise arrives from any direction, it is collected by the FF microphones and reproduced as an inverted signal before leaking into the ear, thus eliminating leakage noise. Because noise from any direction can be robustly canceled, a higher frequency band of cancellation effect can be achieved.

[0051] like Figure 6 As shown, when reproducing 3D audio using a multi-microphone FFNC, the direction of noise arrival can be robustly handled. Therefore, even if the arrangement of the audio object AO is superimposed on the direction of noise arrival, the effect of noise masking can be reduced, thereby enhancing the accuracy of 3D audio reproduction. It should be noted that this disclosure is effective for both mono and stereo audio content.

[0052] <Configuration example of an acoustic signal processing device>

[0053] Figure 7 This diagram illustrates a configuration example of the acoustic signal processing apparatus according to this embodiment. The acoustic signal processing apparatus according to this embodiment is configured as an earphone 1A. The earphone 1A includes microphones LM1 to LM2. N 11. DNC filter, 12. Microphone LFB, 13. Adder section, 14. Driver, 15. Adder section, RM1 to RM1 N The components include: DNC filter 21, microphone RFB, DNC filter 22, adder unit 23, driver 24, adder unit 25, DNC filter 22, digital filter 31, digital filter 32 and control unit 35.

[0054] Audio data, as an external input signal, is provided to the headphones 1A. The audio data can be provided wirelessly or via a wired connection. The audio data can be music data or data consisting only of the speaker's voice. This embodiment will assume that the audio data is 3D audio music data MS. It should be noted that the music data MS can be mono audio data or stereo audio data. Furthermore, the explanation will be given under the assumption that external noise N can intrude into the headphones 1A. For example, the external noise N is noise generated by a moving object (such as an airplane or vehicle), or noise generated by an air conditioner, etc.

[0055] Microphone LM1 to LM N (N represents an optional natural number) is an FFNC microphone, and it is located in the left housing 2A of the headset 1A. Additionally, there is no need to distinguish between microphones LM1 to LM2. N In this case, it is appropriately referred to as a microphone LM. The number and placement of the microphone LMs can be appropriately determined, but preferably, in a way that allows the detection of external noise N that can intrude from the listener L's surrounding environment.

[0056] DNC filter 11 includes DNC filter 111-11 N (N represents an optional natural number). The microphone LM is connected to the corresponding DNC filter 11. For example, microphone LM1 is connected to DNC filter 111, and microphone LM2 is connected to DNC filter 112.

[0057] When the sound output by the driver 14 reaches the listener's ear, each DNC filter 11 eliminates the external noise N and generates a noise cancellation signal. This noise cancellation signal has the effect of allowing only the audio signal audible to the listener. Specifically, each DNC filter 11 generates a noise cancellation signal with the characteristic of being out of phase with the external noise N (the sound signal collected by the corresponding microphone LM) reaching the listener's ear. The DNC filter 11 outputs the generated noise cancellation signal to the adder unit 13.

[0058] The DNC filter 11 is configured as, for example, an FIR (Finite Impulse Response) filter or an IIR (Infinite Impulse Response) filter. Furthermore, in this embodiment, the used DNC filter 11 and its filter coefficients can be changed according to control parameters generated by the control unit 35.

[0059] The microphone LFB is a feedback microphone located in housing 2A. The microphone LFB is positioned near driver 14.

[0060] The DNC filter 12 generates a noise cancellation signal for eliminating external noise N based on the audio signal input to the microphone LFB. The DNC filter 12 is configured as, for example, an FIR filter or an IIR filter. Furthermore, although the filter coefficients are fixed in this embodiment, the filter coefficients of the DNC filter 12 can be changed according to control parameters generated by the control unit 35.

[0061] Adder 13 adds the noise cancellation signal generated by DNC filter 11, the noise cancellation signal generated by DNC filter 12, and the music data MS processed by digital filter 31. The resulting signal is provided to driver 14.

[0062] Driver 14 outputs music data MS and noise cancellation signal provided from adder section 13. The signal output from driver 14 is provided to adder section 15.

[0063] The adder unit 15 adds the music data MS, the noise cancellation signal, and the external noise N. Therefore, the music data MS, with the external noise N eliminated, reaches the listener's left ear.

[0064] Microphone RM1 to RM N (N represents an optional natural number) is an FFNC microphone, and it is located in the right housing 3A of the headset 1A. It should be noted that it is not necessary to connect microphones RM1 to RM... N When distinguished from one another, the microphones are appropriately referred to as microphones RM. The number and placement of the microphones RM can be appropriately determined, but preferably, in a manner that allows for the detection of external noise N that may intrude from the listener L's surroundings.

[0065] DNC filter 21 includes DNC filters 211 to 21. N (N represents an optional natural number). Microphones RM are connected to the corresponding DNC filter 21. For example, microphone RM1 is connected to DNC filter 211, and microphone RM2 is connected to DNC filter 212.

[0066] When the sound output from driver 24 reaches the listener's ear, each DNC filter 21 cancels external noise N and generates a noise cancellation signal that makes only the audio signal audible to the listener. That is, each DNC filter 21 generates a noise cancellation signal that is out of phase with the external noise N (the sound signal collected by the corresponding microphone RM) reaching the listener's ear. The DNC filter 21 outputs the generated noise cancellation signal to adder unit 23.

[0067] The DNC filter 21 is configured as, for example, an FIR filter or an IIR filter. Furthermore, in this embodiment, the used DNC filter 21 and its filter coefficients can be changed according to control parameters generated by the control unit 35.

[0068] Microphone LRB is a feedback microphone located in housing 3A. Microphone RFB is positioned near driver 24.

[0069] The DNC filter 22 generates a noise cancellation signal for eliminating external noise N based on the audio signal input to the microphone RFB. The DNC filter 22 may be configured as, for example, an FIR filter or an IIR filter. Furthermore, although the filter coefficients are fixed in this embodiment, the filter coefficients of the DNC filter 22 can be changed according to control parameters created by the control unit 35.

[0070] Adder 23 adds the noise cancellation signal generated by DNC filter 21, the noise cancellation signal generated by DNC filter 22, and the music data MS processed by digital filter 32. The resulting signal is provided to driver 24.

[0071] Driver 24 outputs music data MS and noise cancellation signal provided from adder section 23. The signal output from driver 24 is provided to adder section 25.

[0072] The adder unit 25 adds the music data MS, the noise cancellation signal, and the external noise N. Therefore, the music data MS, with the external noise N eliminated, reaches the listener's right ear.

[0073] Digital filters 31 and 32 each process the external input signal (music data MS in this embodiment). For example, both digital filters 31 and 32 have an equalization function that changes the frequency characteristics of the music data MS converted into digital form by an A / D (analog-to-digital) converter (not shown), and a rendering function that positions the audio object at a predetermined position by controlling the delay or phase of the audio object. The filter characteristics, such as the filter coefficients of digital filters 31 and 32, are defined by control parameters from the control unit 35.

[0074] The control unit 35 controls the operation of DNC filters 11 and 21 by generating and providing control parameters for DNC filters 11 and 21. Furthermore, the control unit 35 controls the operation of digital filters 31 and 32 by generating and supplying control parameters for digital filters 31 and 32.

[0075] In this embodiment, microphones LM, LFB, RM, and RFB serve as multiple microphones. Furthermore, DNC filters 11 and 12, and DNC filters 21 and 22, serve as noise cancellation units for each microphone, generating noise cancellation signals based on the input sound signals collected by the corresponding microphones. It should be noted that, for example, the headset 1A may include a gain adjustment unit (not shown) for adjusting the volume of the sound.

[0076] <Example of headphone operation>

[0077] Next, an operational example of the headphone 1A will be explained. The DNC filter 11 generates a noise reduction signal to cancel out external noise N based on the input sound signal collected by the microphone LM. The DNC filter 21 generates a noise cancellation signal to eliminate external noise N based on the input sound signal collected by the microphone RM.

[0078] A noise cancellation signal is added to the music data MS. As a result, the external noise N is eliminated. Therefore, the sound corresponding to the music data MS, whose external noise N has been eliminated, is reproduced to the listener.

[0079] According to the first embodiment described so far, multiple microphones are disposed within the housing of the earphone. Therefore, noise arriving from any direction can be effectively eliminated.

[0080] <Second Implementation Method>

[0081] Next, the second embodiment will be described. It should be noted that in the description of the second embodiment, parts that are the same as or similar to those in the above description are indicated by the same reference numerals, and their repeated descriptions will be appropriately omitted. Furthermore, unless otherwise stated, the content already described in the first embodiment can be applied to the second embodiment.

[0082] <Invention>

[0083] Figure 8 A and Figure 8 Figure B is a diagram illustrating the outline of the second embodiment. For example, as shown... Figure 8 As shown in Figure A, the case where external noise N, arriving from the right side of the listener L, dominates will be discussed. The music data MS is 3D audio content. A specified audio object is located at a specified position VP1 to the right of the listener L. In this case, since the location of the audio object or the direction of the sound source is the same as the direction of noise arrival, the clarity and perception of the reproduced sound localization may deteriorate. Therefore, in this embodiment, the direction of the sound source is dynamically changed. Specifically, as in… Figure 8As illustrated in section B, the position of the music data MS is changed to position VP2 along a direction with almost no noise, thereby improving the clarity and feel of the reproduced sound's localization. Details of this embodiment will be described below.

[0084] <Headphone Configuration Example>

[0085] (General configuration example)

[0086] Figure 9 This diagram illustrates an example configuration of the headphones (headphone 1B) according to the second embodiment. The headphone 1B differs from the headphone 1A according to the first embodiment in that it includes an analysis unit 41 connected to the control unit 35. Sound signals collected by the microphone LM, sound signals collected by the microphone RM, and music data MS are supplied to the analysis unit 41. The analysis unit 41 analyzes the sound signals provided from the microphone LM and RM, as well as external input signals.

[0087] (Analysis Department Configuration Example)

[0088] Figure 10 This is a diagram illustrating an example configuration of the analysis unit 41. The analysis unit 41 includes, for example, a noise arrival direction estimation unit 401, an optimal audio object placement calculation unit 402, and an optimal NC filter calculation unit 403.

[0089] The sound signal corresponding to the external noise N collected by the microphones LM and RM is input to the noise arrival direction estimation unit 401. Based on the input sound signal, the noise arrival direction estimation unit 401 generates noise arrival direction information representing the arrival direction of the noise. Specifically, the noise arrival direction information is an index representing the intensity of each noise in multiple directions. The noise arrival direction information is provided to the optimal audio object placement calculation unit 402 and the optimal NC filter calculation unit 403.

[0090] The optimal audio object placement calculation unit 402 calculates the optimal placement position of the audio object based on the noise arrival direction information. The optimal audio object placement calculation unit 402 calculates the optimal placement position of the audio object by additionally referring to information written in the metadata corresponding to the audio object. The method of referring information will be explained in detail later.

[0091] The optimal NC filter calculation unit 403 calculates the optimal control parameters for DNC filters 11 and 21 based on the noise arrival direction information. Then, the optimal NC filter calculation unit 403 outputs the calculation results to the control unit 35.

[0092] (Noise arrival direction estimation unit)

[0093] Next, a specific example of the processing performed by the noise arrival direction estimation unit 401 will be described. Figure 11 This is an example diagram used to illustrate the direction of arrival of the noise to be searched. For example... Figure 11 As shown, the horizontal angle θ and elevation angle are defined using headphones 1B relative to the listener L. Noise arrival direction estimation unit 401 changes the horizontal angle θ and the elevation angle Simultaneously, the noise intensity is calculated for each direction in three dimensions, and noise arrival direction information is generated based on the calculation results.

[0094] Figure 12 This is a diagram illustrating a specific configuration example of the noise arrival direction estimation unit 401. The noise arrival direction estimation unit 401 includes filters 45 (filters 451-45) for three-dimensional directions. N (N represents a natural number). For example, filter 451 directs the zero-sensitivity directionality to a 90-degree vertical angle. Filter 452 directs the zero-sensitivity directionality to a 0-degree horizontal angle and a 0-degree elevation angle. Filter 453 directs the zero-sensitivity directionality to a 30-degree horizontal angle and a 0-degree elevation angle. The sound signals collected by microphones LM and RM are input to the filters that constitute filter 45.

[0095] The output from filter 45 is provided to each dB calculation unit 46. Each dB calculation unit 46 calculates the level (dB) of the input sound signal. The calculation results obtained from the filter are provided to the average calculation unit 47. The average calculation unit 47 calculates the average of the calculation results. Then, the adder unit 48 calculates the difference from the average value and uses the calculation result as the noise intensity in the three-dimensional direction corresponding to the specified filter. For example, the output from filter 451 is provided to dB calculation unit 461. The calculation results obtained from dB calculation unit 461 are provided to the average calculation unit 47 and the adder unit 48. The adder unit 48 calculates the difference between the output of dB calculation unit 461 and the output of average calculation unit 47. The output of adder unit 48 is used as the noise intensity index in the direction corresponding to filter 451, i.e., In this way, the noise intensity index corresponding to each three-dimensional direction is obtained.

[0096] The noise arrival direction estimation unit 401 generates noise arrival direction information based on the obtained noise intensity index. Figure 13 This is an example diagram illustrating the direction of arrival information of noise. For example... Figure 13 As shown, the noise arrival direction information is defined to correspond to a specified horizontal angle θ and a specified elevation angle. The noise level.

[0097] (Optimal audio object placement calculation department)

[0098] like Figure 14 As shown, the optimal audio object placement calculation unit 402 includes a sound source direction determination unit 402A and a filter coefficient conversion unit 402B.

[0099] The sound source direction determination unit 402A determines the direction in which the audio object will be located, i.e., the sound source direction, based on the noise arrival direction information. The sound source direction can be defined by a specific three-dimensional position, or it can be defined by a direction relative to the listener L. The sound source direction determination unit 402A provides the determined sound source direction to the filter coefficient conversion unit 402B.

[0100] The filter coefficient conversion unit 402B converts the sound source direction determined by the sound source direction determination unit 402A into filter coefficients by performing a sound source direction conversion process. For example, the filter coefficient conversion unit 402B maintains filter coefficients corresponding to multiple sound source directions in a table. The filter coefficient conversion unit 402B reads the filter coefficients corresponding to the sound source directions provided by the sound source direction determination unit 402A from this table. Thereafter, the filter coefficient conversion unit 402B provides the read filter coefficients to the control unit 35. The control unit 35 defines the filter coefficients as control parameters for digital filters 31 and 32. Therefore, the audio object is positioned in the sound source direction determined by the optimal audio object placement calculation unit 402.

[0101] The following describes an example of the sound source determination process performed by the sound source direction determination unit 402A. Figure 15 This is a flowchart illustrating a first embodiment of the sound source determination process performed by the sound source direction determination unit 402A. In the first example (mode PT1), a single audio object is reproduced. The metadata corresponding to the audio object includes an identifier of the audio object, a recommended sound source direction for reproducing the audio object (recommended playback position information), and change permission / prohibition information indicating whether the sound source direction is allowed to change from the recommended playback position information.

[0102] In step ST11, the sound source direction determination unit 402A determines whether to allow a change in the sound source direction based on the change permission / prohibition information contained in the metadata. If the determination result is negative, the process proceeds to step ST12.

[0103] In step ST12, since changes in the sound source direction are not allowed, the sound source direction determination unit 402A outputs a recommended sound source direction to the filter coefficient conversion unit 402B. As a result, the audio object is reproduced in the recommended sound source direction. Then, the process ends.

[0104] If the determination result in step ST13 is yes, the process proceeds to step ST13. In step ST13, the calculation of convolving the specified smoothing filter into noise arrival direction information is performed. Then, the process proceeds to step ST14.

[0105] In step ST14, the sound source direction determination unit 402A outputs the direction in which the noise intensity index is minimized based on the smoothed noise arrival direction information. Therefore, in the direction where the noise intensity index is minimized The audio object is then reproduced. The processing then ends.

[0106] It should be noted that the calculation for convolving the smoothing filter in step ST13 can be omitted.

[0107] Figure 16 This is a flowchart illustrating a second example of the sound source determination process performed by the sound source direction determination unit 402A. In the second example (mode PT2), multiple audio objects whose relative positions have been defined are reproduced.

[0108] Meta-information includes identifiers for identifying groups of audio objects, recommended reproduction location information for reference audio objects (hereinafter referred to as reference objects), change permission / prohibition information, and a list of constituent audio objects belonging to the same group. The list of constituent audio objects includes identifiers for identifying each constituent audio object (also referred to as constituent object), and the relative sound source orientation of the constituent audio object (the relative angle relative to the reproduction location of the reference object).

[0109] In step ST21, the sound source direction determination unit 402A determines whether the audio object group is allowed to change the sound source direction based on the change permission / prohibition information included in the metadata. If the result is negative, the process proceeds to step ST22.

[0110] In step ST22, the sound source direction of the reference object is set to the sound source direction represented by the recommended reproduction location information. Then, the process proceeds to step ST26.

[0111] The relative sound source directions of the constituent objects are described in the metadata. When the sound source direction of the reference object is set, the sound source direction of each constituent object can also be determined. Therefore, in step ST26, the sound source direction determination unit 402A outputs a list of the sound source directions of the reference object and all constituent objects. The reference object and constituent objects are reproduced along the respective sound source directions indicated by the list. Then, the process ends.

[0112] If the determination result in step ST21 is yes, the process proceeds to step ST23. In step ST23, convolution calculation is performed on the noise arrival direction information. For example, Figure 17 A describes an example of noise arrival direction information. This refers to the relative sound source direction of the constituent object compared to a reference object. Given (120, 0), prepare as follows Figure 17The smooth comb filter shown in B is a two-dimensional filter that has positive values ​​only around the angle of the object relative to the sound source object. This two-dimensional filter is cyclically convolved with the noise arrival direction information. Then, the processing proceeds to step ST24.

[0113] In step ST24, the sound source direction determination unit 402A sets the sound source direction of the reference object to the direction in which the noise intensity index is minimized from the noise arrival direction information obtained through calculation. Then, the process proceeds to step ST25.

[0114] Since the sound source direction of the reference object has been set, in step ST25, the sound source direction of each component object is set as (the angle of the sound source direction of the reference object + the relative angle of the component object). Then, the process proceeds to step ST26.

[0115] In step ST26, the sound source direction determination unit 402A outputs a list of sound source directions for the reference object and all constituent objects. The reference object and constituent objects are reproduced along the respective sound source directions indicated by the list. Then, the process ends.

[0116] Figure 18 This is a flowchart illustrating a third example of the sound source determination process performed by the sound source direction determination unit 402A. In the third example (mode PT3), multiple audio objects are arranged.

[0117] Define the order of multiple audio objects. The order can be determined randomly or based on the importance of each audio object. For example, audio objects representing human speech have high importance, while audio objects representing background music (BGM) have low importance. Alternatively, if content categories are described in the metadata, the order can be based on those categories. For example, audio objects can be ordered according to a priority order previously defined for each content category.

[0118] In step ST31, the sound source direction determination unit 402A determines the order in which audio objects or groups of audio objects (hereinafter appropriately referred to as audio objects, etc.) are processed. Then, the process proceeds to step ST32.

[0119] In step ST32, a processing loop for audio objects, etc., begins. Then, the process proceeds to step ST33.

[0120] In step ST33, the sound source direction determination unit 402A performs processing on the audio object, etc., in the order corresponding to the determined sequence, related to the aforementioned mode PT1 or PT2. Then, the process proceeds to step ST34.

[0121] In step ST34, whenever the reproduction location of the specified audio object is determined, the noise arrival direction information is updated. Figure 19 A indicates the noise arrival direction information that has not been updated. Figure 19 B describes an example of a smoothing filter that convolves into information about the direction of arrival of noise. For example, as a result of processing mode PT2, it is assumed that the placement of a specified audio object is defined by an angle of approximately 70 degrees. The average level of the audio object corresponding to this angle is obtained ( Figure 19 (C). Add the average level to Figure 19 The noise arrival direction information shown in A ( Figure 19 (D). The next process uses the updated noise arrival direction information. Thus, the change in noise arrival direction information as a result of the rearrangement of audio objects can be reflected in the process regarding either of the modes. The process then proceeds to step ST35.

[0122] In step ST35, it is determined whether there are any more audio objects to be processed. If there are no more audio objects to be processed, the processing is complete.

[0123] (Optimal NC filter calculation unit)

[0124] The optimal NC filter calculation unit 403 selects the optimal filter coefficients for noise cancellation or selects the DNC filter 11 to be operated by using noise arrival direction information and metadata. For example, the optimal NC filter calculation unit 403 calculates the noise cancellation intensity of the DNC filter 11 to be operated or each DNC filter 11 based on the noise arrival direction information. Then, as... Figure 20 As shown, the optimal NC filter calculation unit 403 generates optimal control parameters based on the calculation results. The control unit 35 sets the control parameters generated by the optimal NC filter calculation unit 403 for the appropriate DNC filter 11. It should be noted that the DNC filter 11 in Figure 20 As shown in the figure, the optimal NC filter calculation unit 403 performs similar processing on the DNC filter 21.

[0125] With the residual noise in the listener's ear defined as e(t), the residual noise in the ear can be represented by the following expression (1) (in expression (1), l(t) represents the previously measured leakage noise, x...). m (t) represents the input to all FFNC microphones, f m (t) represents the characteristics of the DNC filter, and d(t) represents the acoustic characteristics in the headphones.

[0126] [Mathematical Expression 1]

[0127]

[0128] The optimal NC filter calculation unit 403 can calculate the control parameters of DNC filters 11 and 21 in a way that minimizes the residual noise in the ear in expression (1).

[0129] <Third Implementation Method>

[0130] Next, the third embodiment will be described. It should be noted that in the description of the third embodiment, parts that are the same as or similar to those in the above description are indicated by the same reference numerals, and their repeated descriptions will be appropriately omitted. Furthermore, unless otherwise stated, the content already described in the first or second embodiment can be applied to the third embodiment.

[0131] In the third embodiment, generally speaking, a portion of the processing performed in the headset in the first or second embodiment is performed in an external device (e.g., a smartphone or server device that can communicate with the headset).

[0132] <Headphone Configuration Example>

[0133] Figure 21 This is a diagram illustrating a configuration example of an earphone (earphone 1C) according to a third embodiment. Earphone 1C includes a communication unit 51 and a storage unit 52, such as a memory. Furthermore, earphone 1C only includes a noise arrival direction estimation unit 401, a functional block of the analysis unit 41. Additionally, earphone 1C does not include digital filters 31 or 32. However, earphone 1C includes equalization functions EQs 53 and EQ 54 that perform the functions of digital filters 31 and 32.

[0134] The communication unit 51 includes an antenna and a modulation / demodulation circuit suitable for a communication system. Wireless communication is assumed to be performed, but wired communication can also be performed. For example, wireless communication can be performed via LAN (Local Area Network), Bluetooth, Wi-Fi, or WUSB (Wireless USB). As a result of the communication performed by the communication unit 51, the headset 1C is paired with an external device such as a smartphone.

[0135] <Configuration Example of a Smart Phone>

[0136] Figure 22 This is a diagram illustrating a configuration example of a smartphone 81 as an example of an external device. The smartphone 81 includes a CPU (Central Processing Unit) 82, a DSP (Digital Signal Processor) 83, a first communication unit 84, a second communication unit 85, an optimal audio object placement calculation unit and an optimal NC filter calculation unit 86, an object filter control circuit 87, and a storage unit 88. The DSP 83 includes digital filters 83A and 83B.

[0137] CPU 82 typically controls smartphone 81. For example, digital filters 83A and 83B of DSP 83 perform rendering processes that position audio objects at predetermined locations.

[0138] The first communication unit 84 communicates with the server device 71. As a result of this communication, data about the audio object is downloaded from the server device 71 to the smartphone 81.

[0139] The second communication unit 85 communicates with the communication unit 51 of the headset 1C. As a result of this communication, noise arrival direction information is provided from the headset 1C to the smartphone 81. In addition, audio data that undergoes processing, which will be described later, is supplied from the smartphone 81 to the headset 1C.

[0140] The (Optimal Audio Object Placement Calculation Unit and Optimal NC Filter Calculation Unit) 86 has the functions of the aforementioned optimal audio object placement position calculation unit 402 and the aforementioned optimal NC filter calculation unit 403.

[0141] The object filter control circuit 87 sets the filter coefficients for digital filters 83A and 83B to realize the configuration position of the audio object calculated by the (optimal audio object placement calculation unit and optimal NC filter calculation unit) 86.

[0142] Storage unit 88 stores various types of data. For example, storage unit 88 stores filter coefficients used to determine the placement of audio objects. Storage unit 88 is, for example, a magnetic storage device such as an HDD (hard disk drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device.

[0143] Processing performed between the headset and the smartphone

[0144] Next, the processing between the headset 1C and the smartphone 81 will be performed. First, for example, short-range wireless communication will be performed between the headset 1C and the smartphone 81, thereby pairing the headset 1C and the smartphone 81.

[0145] The headset 1C generates noise arrival direction information, as previously described in the second embodiment. The noise arrival direction information is provided from the communication unit 51 of the headset 1C to the second communication unit 85 of the smartphone 81. The noise arrival direction information is then provided from the second communication unit 85 to the (optimal audio object placement calculation unit and optimal NC filter calculation unit) 86.

[0146] By communicating with server device 71, the first communication unit 84 of the smartphone 81 obtains audio objects and corresponding metadata from server device 71. Data about the audio objects is provided to DSP 83. The metadata is then provided to (optimal audio object placement calculation unit and optimal NC filter calculation unit) 86.

[0147] The optimal audio object placement calculation unit and the optimal NC filter calculation unit 86 determine the placement position (sound source direction) of the audio object in a manner similar to that in the second embodiment. The optimal audio object placement calculation unit and the optimal NC filter calculation unit 86 provide the determined sound source direction to the target filter control circuit 87. The target filter control circuit 87 reads the filter coefficients used to realize the sound source direction from the storage unit 88 and sets the read coefficients for the digital filters 83A and 83B.

[0148] Digital filters 83A and 83B are used to filter the data on the audio object. The resulting data is transmitted to the headset 1C via the second communication unit 85. In addition, the optimal control parameters of DNC filters 11 and 21 calculated by the (optimal audio object placement calculation unit and optimal NC filter calculation unit) 86 are sent to the headset 1C via the second communication unit 85.

[0149] Data about an audio object received by the communication unit 51 of the headset 1C undergoes equalization processing at EQ 53 and is then provided to the adder unit 13. Furthermore, data about an audio object received by the communication unit 51 of the headset 1C undergoes equalization processing at EQ 54 and is then provided to the adder unit 23.

[0150] Furthermore, the optimal control parameters for DNC filters 11 and 21 received by the communication unit 51 of the earphone 1C are provided to the control unit 35, and settings are made for DNC filters 11 and 21 respectively. The remaining processes are similar to those previously described in the first or second embodiments.

[0151] As described so far, a portion of the headset according to the first or second embodiment can be implemented by an external device (e.g., a smartphone). That is, the acoustic signal processing apparatus according to this disclosure is not limited to headsets and can be implemented by electronic devices such as smartphones. It should be noted that which function is implemented by an external device can be appropriately changed. For example, in the third embodiment described above, the smartphone 81 may have the function of a noise arrival direction estimation unit 401 that generates noise arrival direction information.

[0152] <Variation Example>

[0153] Some embodiments according to this disclosure have been described above. This disclosure is not limited to the above embodiments, and various modifications can be made based on the technical concept of this disclosure.

[0154] The arrangement of the aforementioned components in the headphones can be appropriately modified within the headphones. For example, in the case of using over-ear or neckband headphones, the circuit configuration, including a digital filter, control unit, analysis unit, etc., is installed in either the left or right housing, while the data cable connecting the two housings is configured to allow data to be sent / received from / to one side without the circuit configuration. Furthermore, in neckband headphones, the circuit can be installed in one of the housings as described above, or the circuit configuration, including a digital filter, control unit, analysis unit, etc., can be located in the neckband portion. Meanwhile, in so-called left-right independent headphones (e.g., in-ear or open-ear types), it is desirable that the circuit, including a digital filter, control unit, analysis unit, etc., is independently installed on each of the left and right sides, which is not described.

[0155] The aforementioned DNC filter, digital filter, and EQ53 can be incorporated into a DSP. Furthermore, the control and analysis units can be included in the circuitry of the DSP or processor. Alternatively, the control and analysis units can be configured to operate according to a computer program (software) operated by the DSP or processor.

[0156] The configurations, methods, steps, shapes, materials, and values ​​described in the above embodiments and variations are merely examples. Any other configurations, methods, steps, shapes, materials, and values ​​may be used instead if necessary. The structures, methods, steps, shapes, materials, and values ​​described in the above embodiments and variations can be replaced with well-known structures, methods, steps, shapes, materials, and values. Furthermore, the configurations, methods, steps, shapes, materials, and values ​​described in the above embodiments and variations can be combined as long as no technical inconsistencies arise.

[0157] It should be noted that the interpretation of this disclosure should not be limited to the effects described herein.

[0158] This technology may also have the following configurations. (1)

[0160] An acoustic signal processing device, comprising:

[0161] A noise cancellation unit, wherein the noise cancellation unit is configured for each of a plurality of microphones and generates a signal for noise cancellation based on an audio signal input from the corresponding microphone; and

[0162] Digital filters are used to process external input signals. (2)

[0164] The acoustic signal processing apparatus according to (1) further includes:

[0165] The control unit generates control parameters for the noise cancellation unit; and

[0166] The analysis unit analyzes the sound signal input from the microphone. (3)

[0168] According to the acoustic signal processing device of (2), wherein,

[0169] The analysis unit includes a noise arrival direction estimation unit, which generates noise arrival direction information representing the arrival direction of the noise based on the sound signal input from the microphone. (4)

[0171] According to the acoustic signal processing apparatus described in (3), the noise cancellation unit further generates control parameters for the digital filter, and the analysis unit further analyzes the external input signal, which includes an audio object and metadata corresponding to the audio object.

[0172] The optimal audio object placement calculation unit includes an optimal audio object reproduction position calculation unit, which calculates the optimal reproduction position of the audio object based on the noise arrival direction information. (5)

[0174] According to the acoustic signal processing device of (4), wherein,

[0175] The audio object is a single audio object.

[0176] The metadata includes recommended reproduction location information and permission / prohibition information indicating whether a change in the sound source direction is allowed, as well as...

[0177] If the change permission / prohibition information indicates that the change is permitted, the process of reproducing the audio object at the optimal reproduction position is performed; if the change permission / prohibition information indicates that the change is not permitted, the process of reproducing the audio object at the recommended reproduction position is performed. (6)

[0179] According to the acoustic signal processing device described in (4), wherein

[0180] The audio object includes multiple audio objects that define their relative reproduction positions, and

[0181] Based on the noise arrival direction information, the analysis unit calculates the optimal reproduction position to minimize the noise intensity index for multiple audio objects. (7)

[0183] According to the acoustic signal processing device described in (4), wherein

[0184] An audio object consists of multiple audio objects in a defined order.

[0185] If the change permission / prohibition information indicates that the change is permitted, the audio object is reproduced at the optimal reproduction position; if the change permission / prohibition information indicates that the change is not permitted, the audio object is reproduced at the recommended reproduction position.

[0186] The processing is performed in the order corresponding to the defined order, and the noise arrival direction information is updated each time the processing is performed. (8)

[0188] According to the acoustic signal processing device of (7), wherein,

[0189] The order is determined randomly, based on priority, or based on content classification. (9)

[0191] The acoustic signal processing apparatus according to any one of (3) to (8), wherein

[0192] The analysis unit generates the optimal control parameters for the noise cancellation unit based on the noise arrival direction information. (10)

[0194] The acoustic signal processing apparatus according to any one of (4) to (8), wherein

[0195] The digital filter performs the process of locating the audio object at a specified position. (11)

[0197] The acoustic signal processing apparatus according to any one of (1) to (10) further includes:

[0198] The multiple microphones. (12)

[0200] According to the acoustic signal processing device of (11), wherein,

[0201] The plurality of microphones includes feedforward microphones and feedback microphones. (13)

[0203] The acoustic signal processing apparatus according to any one of (1) to (12), wherein

[0204] The external input signal includes audio data provided wirelessly or via wired connection. (14)

[0206] The acoustic signal processing apparatus according to any one of (1) to (13) is configured as an earphone device. (15)

[0208] An acoustic signal processing method, comprising:

[0209] A noise cancellation unit, configured for each microphone, generates a signal for canceling noise based on the sound signals input from the corresponding microphones among the multiple microphones; and

[0210] External input signals are processed using digital filters. (16)

[0212] A program for causing a computer to perform an acoustic signal processing method, the acoustic signal processing method comprising:

[0213] A noise cancellation unit, configured for each microphone, generates a signal for canceling noise based on the sound signals input from the corresponding microphones among the multiple microphones; and

[0214] External input signals are processed using digital filters.

[0215] [List of Reference Numbers]

[0216] 1A, 1B, 1C headphones; 11, 12, 21, 22 DNC filters

[0217] 31, 32 Digital Filters

[0218] 35 Control Department

[0219] 41 Analysis Department

[0220] 401 Noise Arrival Direction Estimation Unit

[0221] 402 Optimal Audio Object Placement Calculation Unit; 403 Optimal NC Filter Calculation Unit; LM and RM Microphones

Claims

1. An acoustic signal processing device, comprising: A noise cancellation unit is provided for each of a plurality of microphones and generates a signal for noise cancellation based on the sound signal input from the corresponding microphone. The control unit generates control parameters for the noise cancellation unit; The analysis unit analyzes the sound signal input from the microphone; as well as Digital filters process externally received input signals, where... The external input signal includes an audio object and corresponding metadata, including reproduction location information and permission / prohibition information indicating whether a change in the direction of the sound source is permitted. Analyzing the input audio signal involves altering noise direction-of-arrival information and changing the reproduction position of the audio object based on said metadata, the noise direction-of-arrival information being generated from the audio signal input from the microphone. The control unit further generates control parameters for the digital filter.

2. The acoustic signal processing apparatus according to claim 1, wherein, The audio object is a single audio object, and If the change permission / prohibition information indicates that the change is permitted, the process of reproducing the audio object at the changed reproduction location is performed; if the change permission / prohibition information indicates that the change is not permitted, the process of reproducing the audio object at the reproduction location of the metadata is performed.

3. The acoustic signal processing apparatus according to claim 1, wherein, The audio object includes multiple audio objects, and in the metadata, the relative sound source direction from the reference object is defined. Based on the noise arrival direction information, the input sound signal is analyzed to change the reproduction position, so that the noise intensity index for the plurality of audio objects is minimized.

4. The acoustic signal processing apparatus according to claim 2, wherein, The audio object includes multiple audio objects that define the order of the audio objects. If the change permission / prohibition information indicates that change is permitted, the process of reproducing the audio object at the changed reproduction location is performed; and if the change permission / prohibition information indicates that change is not permitted, the process of reproducing the audio object at the reproduction location of the metadata is performed. Processing is performed in the order corresponding to the defined order, and the noise arrival direction information is updated each time processing is performed.

5. The acoustic signal processing apparatus according to claim 4, wherein, The order is determined randomly, based on priority, or based on content classification.

6. The acoustic signal processing apparatus according to claim 1, wherein, The analysis unit generates control parameters for the noise cancellation unit based on the noise arrival direction information.

7. The acoustic signal processing apparatus according to claim 1, wherein, The digital filter performs the process of locating the audio object at a specified position.

8. The acoustic signal processing apparatus according to claim 1, further comprising: The multiple microphones.

9. The acoustic signal processing apparatus according to claim 8, wherein, The plurality of microphones includes feedforward microphones and feedback microphones.

10. The acoustic signal processing apparatus according to claim 1, wherein, The external input signal includes audio data provided wirelessly or via wired connection.

11. The acoustic signal processing apparatus according to claim 1, wherein the acoustic signal processing apparatus is configured as an earphone device.

12. An acoustic signal processing method, comprising: A noise cancellation signal is generated by a digital noise cancellation filter based on the sound signal input from a corresponding microphone among a plurality of microphones, the digital noise cancellation filter being configured for each of the plurality of microphones; The processor generates control parameters for the noise cancellation unit and analyzes the sound signal input from the microphone. as well as The externally received input signal is processed by a digital filter, wherein... The external input signal includes an audio object and corresponding metadata, the metadata including reproduction location information and permission / prohibition information indicating whether a change in the direction of the sound source is permitted. The processor also changes the reproduction position of the audio object based on the metadata and noise arrival direction information, and generates control parameters for the digital filter, wherein the noise arrival direction information is generated based on the sound signal input from the microphone.

13. A computer-readable storage medium storing a program for causing a computer to perform an acoustic signal processing method, comprising: A noise cancellation signal is generated by a digital noise cancellation filter based on the sound signal input from a corresponding microphone among a plurality of microphones, the digital noise cancellation filter being configured for each of the plurality of microphones; The processor generates control parameters for the noise cancellation unit and analyzes the sound signal input from the microphone. as well as The externally received input signal is processed by a digital filter, wherein... The external input signal includes an audio object and corresponding metadata, the metadata including reproduction location information and permission / prohibition information indicating whether a change in the direction of the sound source is permitted. The processor also changes the reproduction position of the audio object based on the metadata and noise arrival direction information, and generates control parameters for the digital filter, wherein the noise arrival direction information is generated based on the sound signal input from the microphone.

Images