Wearable audio device with enhanced voice pickup

Abstract

A wearable bidirectional communication audio device includes a first microphone that provides a first microphone signal, a second microphone that provides a second microphone signal, and a third microphone that provides a third microphone signal. The device also includes one or more processors configured to process the first microphone signal and the second microphone signal to form a first beamformed signal. The one or more processors compare energy in the first beamformed signal to energy in the first microphone signal, and if the energy in the first beamformed signal exceeds the energy in the first microphone signal, the one or more processors mix the first microphone signal and the third microphone signal to provide a mixed signal. The one or more processors can also generate a speech output signal for transmission to a far-end recipient using the mixed signal.

Description

Background Technology

[0001] This disclosure relates to wearable audio devices. More specifically, this disclosure relates to wearable audio devices that enhance a user's speech signals. Summary of the Invention

[0002] All examples and features mentioned below can be combined in any technically possible way.

[0003] In one aspect, a wearable two-way communication audio device includes: a first microphone providing a first microphone signal; a second microphone providing a second microphone signal; and a third microphone providing a third microphone signal. The device also includes one or more processors configured to process the first microphone signal and the second microphone signal to form a first beamforming signal. The one or more processors compare the energy in the first beamforming signal with the energy in the first microphone signal, and if the energy in the first beamforming signal exceeds the energy in the first microphone signal, the one or more processors mix the first microphone signal and the third microphone signal to provide a mixed signal. The one or more processors may also use the mixed signal to generate a voice output signal for transmission to a remote receiver.

[0004] Specific implementations may include one of the following features, or any combination thereof.

[0005] In some specific implementations, mixing the first microphone signal and the third microphone signal includes calculating the energy ratio between the first microphone signal and the third microphone signal, and selecting a mixing coefficient for the first microphone signal and the third microphone signal based on the calculated energy ratio.

[0006] In some specific implementations, generating a speech output signal using a mixed signal includes using the mixed signal to generate a first signal component in a first frequency range for the speech output signal, and using a beamforming signal to generate a second signal component in a second frequency range for the speech output signal, and combining the first signal component and the second signal component to provide the speech output signal.

[0007] In some cases, one or more processors are configured to mix the first microphone signal and the third microphone signal to provide a mixed signal only when the energy in the first beamforming signal exceeds the energy in the first microphone signal by a predetermined threshold.

[0008] In some cases, one or more processors are configured such that the first beamforming signal is used to generate a speech output signal if the energy in the first beamforming signal does not exceed a predetermined threshold in the energy of the first microphone signal.

[0009] In some examples, one or more processors are configured such that if the energy in the first beamforming signal does not exceed a predetermined threshold in the energy of the first microphone signal, the first microphone signal and the third microphone signal are not mixed.

[0010] In some examples, one or more processors are configured such that if the energy in the beamforming signal does not exceed the energy in the first microphone signal, the first beamforming signal is used to generate the speech output signal, and a mixed signal is not used to generate the speech output signal.

[0011] In some implementations, one or more processors are configured such that if the energy in the first beamforming signal exceeds the energy in the first microphone signal, the first microphone signal and the third microphone signal are mixed to provide a mixed signal, and the combination of the mixed signal and the first beamforming signal is used to generate a speech output signal.

[0012] In some implementations, one or more processors are configured such that a first beamforming signal is used to provide a first signal component, the first signal component including frequency content above a predetermined frequency, and a mixed signal is used to provide a second signal component, the second signal component including frequency content below the predetermined frequency. The first signal component and the second signal component are combined to provide a voice output signal.

[0013] Another feature is a wearable two-way communication audio device. The device includes multiple microphones and one or more processors. The one or more processors are configured to process signals from the multiple microphones to form a first beamforming signal and estimate wind energy based on the first beamforming signal. The one or more processors are further configured to adjust a high-pass filter based on the estimated wind energy and use the high-pass filter to filter other signals to provide a voice output signal.

[0014] Specific implementations may include one of the features described above and / or below, or any combination thereof.

[0015] In some cases, one or more processors are configured to filter the first beamforming signal using a bandpass filter to provide a bandpass-filtered signal, and use the bandpass-filtered signal to estimate wind energy.

[0016] In some cases, one or more processors are configured to adjust the high-pass filter by mapping the estimated wind energy to one of a plurality of high-pass filters, each with a different turn frequency.

[0017] In some examples, one or more processors are configured to select a first high-pass filter with a higher corner frequency when the estimated wind energy is high, and a second high-pass filter with a lower corner frequency when the estimated wind energy is low.

[0018] In some examples, multiple high-pass filters include at least five high-pass filters.

[0019] In some implementations, multiple high-pass filters include at least 10 high-pass filters.

[0020] In some implementations, one or more processors are configured to adjust the high-pass filter by adjusting its corner frequency.

[0021] In some cases, one or more processors are configured to process signals from multiple microphones to form a second beamforming signal, and use the second beamforming signal to generate another signal.

[0022] According to another aspect, a wearable two-way communication audio device includes: a first earpiece including a first plurality of microphones; and a second earpiece including a second plurality of microphones. The device further includes one or more processors configured to process signals from the first plurality of microphones to form a first beamforming signal, and to process signals from the first plurality of microphones to form a second beamforming signal. The one or more processors are further configured to process signals from the second plurality of microphones to form a third beamforming signal, and to process signals from the second plurality of microphones to form a fourth beamforming signal. The one or more processors compare a first wind signal derived from the second beamforming signal with a second wind signal derived from the fourth beamforming signal, and select one of the first earpiece or the second earpiece based on the comparison of the first wind signal and the second wind signal to provide a voice output signal for transmission to a remote receiver.

[0023] Specific implementations may include one of the features described above and / or below, or any combination thereof.

[0024] In some cases, one or more processors are further configured to compare a third wind signal derived from the first beamforming signal with a fourth wind signal derived from the third beamforming signal, and to select either the first or second earpiece to provide a voice output signal based at least in part on the comparison between the third and fourth wind signals.

[0025] In some examples, one or more processors are further configured to calculate a first wind energy estimate based on a first beamforming signal and set a first wind flag based on the first wind energy estimate, and to calculate a second wind energy estimate based on a third beamforming signal and set a second wind flag based on the second wind energy estimate. A third wind signal may correspond to the first wind flag, and a fourth wind signal may correspond to the second wind flag.

[0026] In some examples, if the first wind indicator indicates no wind on the first earpiece and the second wind indicator indicates wind on the second earpiece, then the first earpiece is selected to provide the voice output signal.

[0027] In some specific implementations, one or more processors are further configured to calculate a third wind energy estimate based on a second beamforming signal, calculate a fourth wind energy estimate based on a fourth beamforming signal, and select either a first or a second earpiece to provide the voice output signal based on a comparison of the third and fourth wind energy estimates.

[0028] In some specific implementations, the first wind signal corresponds to the third wind energy estimate, and the second wind signal corresponds to the fourth wind energy estimate.

[0029] In some cases, one or more processors are configured such that if both the first wind indicator and the second indicator indicate wind conditions, the one or more processors compare a third wind energy estimate and a fourth wind energy estimate, and if the third wind energy estimate is lower than the fourth wind energy estimate, the first earpiece is selected to provide the voice output signal.

[0030] In certain situations, where there is no wind, the second earpiece is selected by default to provide the voice output signal.

[0031] Specific implementation may provide one or more of the following beneficial effects.

[0032] The systems and methods described in this paper can reduce wind noise, especially clustered wind noise.

[0033] Some implementation schemes can help reduce low-frequency wind noise to below 1 kHz without significantly compromising speech intelligibility.

[0034] Certain specific implementations can provide improved noise reduction. In this regard, the systems and methods described herein can use spectral noise subtraction and / or steady-state noise reduction algorithms to reduce severe high-frequency noise leakage.

[0035] Some implementations can provide reduced ambient noise, such as HVAC or fan noise, in relatively quiet environments.

[0036] Some implementations can provide a smoother transition in noise level between when the user is speaking and when the user stops speaking.

[0037] Some configurations can provide more natural voice and more bandwidth than regular headphones under quiet conditions.

[0038] Certain configurations can provide a significant reduction in the popping / crackling sounds that would otherwise be perceived as distortion in regular headphones.

[0039] Some specific implementations can reduce the effects of user speech becoming very quiet or spectrally unbalanced when the earpiece is rotated away from the nominal orientation or / and when the user is speaking next to a hard surface such as a wall or has their hands behind their head. Attached Figure Description

[0040] Figure 1 The block diagram depicts an exemplary wearable audio device based on various disclosed specific implementations.

[0041] Figure 2 It is a block diagram based on various specific implementations of audio processing systems.

[0042] Figure 3 It is possible to be with Figure 2 The audio processing system uses a block diagram of the output stage processing.

[0043] Figure 4 It is a description that can be used with Figure 1 A block diagram of wind-based role-switching logic used in conjunction with wearable audio devices.

[0044] Figure 5 It is a description of the source Figure 2 A block diagram of an exemplary steady-state noise reduction device for an audio processing system.

[0045] It should be noted that the accompanying drawings for various specific embodiments are not necessarily drawn to scale. The drawings are intended only to illustrate typical aspects of this disclosure and should not be considered as limiting the scope of the specific embodiments. In the drawings, similar numbers indicate similar elements between the figures. Detailed Implementation

[0046] The aspects and embodiments disclosed herein are applicable to a wide variety of wearable audio devices of various form factors, but generally relate to devices having at least one internal microphone that is substantially shielded from ambient noise (i.e., acoustically coupled to the environment inside the user's ear canal) and at least one external microphone that is substantially exposed to ambient noise (i.e., acoustically coupled to the environment outside the user's ear canal). Furthermore, various embodiments relate to wearable audio devices supporting bidirectional communication and may include, for example, in-ear, over-ear, and near-ear devices. Form factors may include, for example, earbuds, headphones, hearing aids, and other wearable devices. Additional configurations may include headphones with one or two earpieces, over-ear headphones, neckband headphones, in-ear or behind-the-ear hearing aids, wireless headsets, audio glasses, single earpieces or earpiece pairs, and hats, helmets, clothing, or any other physical configuration including one or two earpieces to enable audio communication and / or ear protection. Furthermore, the disclosure herein applies to wearable audio devices that are wirelessly connected to other devices, connected to other devices via conductive and / or light-guiding cables, or not connected to any other device at all.

[0047] It should be noted that although specific embodiments of wearable audio devices are presented in some degree of detail, such presentation of specific embodiments is intended to facilitate understanding by providing examples and should not be construed as limiting the scope of this disclosure or the scope of the claims.

[0048] Figure 1This is a block diagram of an example shape factor of an in-ear wearable audio device 100 having two earpieces 102A and 102B, each earpiece configured to direct sound toward the user's ear. (The reference numerals “A” or “B” indicate the correspondence between the identified features and a specific earpiece of the two earpieces. However, for simplicity, letter designations are omitted from the following discussion; for example, earpiece 102 refers to either or both of earpieces 102A and 102B.) Each earpiece 102 includes a housing 104 that defines a cavity 106 that houses an electroacoustic transducer 108 for outputting audio signals to the user. Additionally, at least one internal microphone 110 (also referred to as a “feedback microphone” or “FB microphone”) is also disposed within the cavity 106. In a specific embodiment where the wearable audio device 100 can be mounted on the ear, an ear coupling 112 (e.g., an ear tip or ear pad) attached to the housing 104 surrounds the opening of the cavity 106. Channel 114 is formed to pass through ear coupling member 112 and communicate with an opening leading to cavity 106. In various embodiments, one or more external microphones (e.g., first external microphone 116, second external microphone 118, and third external microphone 120) are disposed on the housing in a manner that allows acoustic coupling to the environment outside the housing 102. The first external microphone 116 may also be referred to as a "first communication microphone" or simply a "COM1 microphone". The second external microphone 118 may also be referred to as a "second communication microphone" or simply a "COM2 microphone". And the third external microphone 120 may also be referred to as a "feedforward microphone", or simply a "FF microphone" or "ear concha microphone".

[0049] The audio output from transducer 108 and the speech captured by external microphones 116, 118 in each earpiece are controlled by audio processing system 122. Audio processing system 122 may be integrated into one or both earpieces 102 or implemented by an external system. When audio processing system 122 is implemented by an external system, each earpiece 102 may be coupled to audio processing system 122 in a wired or wireless configuration. In various specific implementations, audio processing system 122 may include hardware, firmware, and / or software to provide various features to support the operation of wearable audio device 100, including, for example, providing power, amplification, input / output, network interface, user control functions, active noise cancellation (ANR), signal processing, data storage, data processing, speech detection, etc.

[0050] Wearable audio device 100 is configured to provide two-way communication, wherein a user’s voice or speech is captured and then output to an external node via audio processing system 122. In this regard, external microphones 116, 118 (alone or in combination with external microphone 120) can be used to capture the user’s voice, and audio processing system 122 can be used to process those microphone signals to provide a voice signal (also referred to as a “voice output signal”) to the remote end of the two-way communication (telephone call).

[0051] Therefore, the audio processing system 122 may include a left earpiece processing system 124 for processing signals from microphones 110A, 116A, 118A, and 120A of the left earpiece 102A, and a right earpiece processing system 126 for processing signals from microphones 110B, 116B, 118B, and 12B of the right earpiece 102B. The audio processing system 122 may also include a combined earpiece processing system 128 for processing signals from the left earpiece processing system 124 and the right earpiece processing system 126. For example, the wearable audio device 100 may be configured such that the microphone input from only one of the earpieces 102A and 102B (the main earpiece) is used to provide a voice output signal (e.g., ...). Figure 3 (Item 302), and as described below, the audio processing system 122 can be used to dynamically select which earpiece 102A, 102B will be used to provide the far-end voice signal based on the signals received from the left earpiece processing system 124 and the right earpiece processing system 126.

[0052] The left earpiece processing system 124 may be executed by a first processor in the left earpiece 102A, and the right earpiece processing system 126 may be executed by a second processor in the right earpiece 102B. The combined earpiece processing system 128 may be executed by one of the first or second processors or by a third processor that may reside in the left earpiece 102A, the right earpiece 102B, or an external system (such as a mobile device coupled to one or both of the earpieces 102A and 102B).

[0053] In a specific implementation including ANR for enhancing audio signals, the internal microphone 110 may act as a feedback microphone, and the external microphone 120 (alone or in combination with microphones 116 and 118) may act as a feedforward microphone. In such an implementation, each earpiece 102 may utilize ANR circuitry communicating with both the internal microphone 110 and the external microphone 120. The ANR circuitry receives an internal signal generated by the internal microphone 110 and an external signal generated by the external microphone 120 (alone or in combination with microphones 116 and 118), and performs an ANR process for the corresponding earpiece 102. This process includes providing a signal to an electroacoustic transducer (e.g., a speaker) 108 disposed in the cavity 106 to generate an anti-noise acoustic signal that reduces or substantially prevents sound from one or more acoustic noise sources outside the earpiece 102 from being heard by the user. The external microphone 120 may be arranged to face the user's concha when the device is worn, for example, to protect the microphone 27 from wind. This configuration is disclosed in U.S. Patent Application No. 17 / 362,625 (now Patent No. 11,540,043), filed on December 27, 2022, entitled “ACTIVE NOISE REDUCTION EARBUD”, the entire disclosure of which is incorporated herein by reference.

[0054] Figure 2 An exemplary embodiment of an exemplary earpiece processing system 124 (e.g., a left earpiece processing system 124 or a right earpiece processing system 126) is depicted, which receives voice and other inputs from a set of microphones 110, 116, 118, 120 on the earpiece 102, processes the inputs, and outputs an enhanced speech signal 202 for transmission or further processing. Figure 2 The processing illustrated herein is performed simultaneously by each of the two earpieces 102A and 102B. In this embodiment, earpiece 102 is configured to capture a corresponding microphone signal from each of the external microphones 116, 118 and 120, as well as at least one internal microphone signal, in this example, the at least one internal microphone signal originating from the internal feedback (FB) microphone 110.

[0055] System 124 typically includes a domain converter 204 that converts the microphone signal from the time domain to the frequency domain. Domain converter 204 also separates the spectral components of each microphone signal into multiple sub-bands. For example, domain converter 204 can process the microphone signal to provide frequencies limited to a specific range, and within that range, multiple sub-bands can be provided in combination to cover the entire range. In a specific example, a sub-band filter can provide 64 sub-bands in the frequency range of 0 to 8,000 Hz, each sub-band covering 125 Hz. Domain converter 204 can, for example, be configured to use Weighted Overlap Addition (WOLA) analysis to convert the time-domain signal into sub-bands.

[0056] Figure 2 Each of the subsequent components in the region labeled “subband processing” of the exemplary system 124 can logically represent multiple such components for processing multiple subbands.

[0057] Domain converter 204 provides frequency domain signals 206 and 208 from the first external microphone 116 and the second external microphone 118 to each of the two beamformers 210 and 212, respectively. Beamformers 210 and 212 employ array processing techniques, such as phased arrays and delay subtraction, and can utilize minimum variance distortion-free response (MVDR) and linearly constrained minimum variance (LCMV) techniques to adjust the responsivity of the group of microphones 116 and 118 to enhance or reject acoustic signals from various directions. Beamforming enhances acoustic signals from a specific direction or directional range, while null-guiding reduces or rejects acoustic signals from a specific direction or directional range.

[0058] The first beamformer 210 is a beamformer used to maximize the acoustic response of the group of microphones 116, 118 in the direction of the user's mouth (e.g., in front of and slightly below the earpiece) and to provide a first beamforming signal 214. Because the first beamformer 210 performs beamforming, the first beamforming signal 214 contains higher signal energy due to the user's voice than any single microphone signal.

[0059] The second beamformer 212 directs the null point toward the user's mouth and provides a second beamforming signal 216. Because the null point toward the user's mouth, the second beamforming signal 216 includes the minimum (if any) signal energy due to the user's speech. Therefore, the second beamforming signal 216 is essentially composed of components due to background noise and acoustic sources not due to the user's speech; that is, the second beamforming signal 216 is a signal associated with an acoustic environment without user speech.

[0060] In some examples, the first beamformer 210 is a super-directional near-field beamformer that enhances the acoustic response in the direction of the user's mouth, and the second beamformer 212 is a delay subtraction algorithm that guides the zero point (i.e. reduces the acoustic response) in the direction of the user's mouth.

[0061] A first beamforming signal 214 and a frequency-domain first external microphone signal 206 (also referred to as the "frequency-domain COM1 microphone signal") are provided to a wind detector 218, which analyzes those signals to identify the presence of wind. The wind detector 218 calculates the energy difference between the first beamforming signal 214 and the frequency-domain COM1 microphone signal. In this respect, the wind detector 218 may calculate the energy in each of the first beamforming signal 214 and the frequency-domain COM1 microphone signal 206 on a sub-band basis, and then sum the calculated sub-band energies to determine the total wind energy of each of those signals before determining the difference between the two sums. In some cases, the wind detector 218 may only calculate the energy within a specific frequency band (e.g., 125 Hz to 2 kHz).

[0062] If the energy difference between the first beamforming signal 214 and the frequency domain COM1 microphone signal 206 exceeds a threshold, the wind detector 218 identifies that wind has been detected. Based on this analysis, the wind detector 218 generates a wind indicator signal 220. The wind indicator signal 220 can be a binary signal (0 or 1) indicating whether there is wind or no wind.

[0063] The frequency domain signal 222 from the third external microphone 120 (also referred to as a "forward microphone," "FF microphone," or "ear concha microphone") is equalized via an equalization (EQ) filter 224 to produce an equalized FF microphone signal 226. This equalized FF microphone signal, along with the frequency domain COM1 microphone signal 206, the first beamforming signal 214, and the wind indicator signal 220, is provided to the dynamic wind mixer 228. The EQ filter 224 equalizes the FF microphone signal 222 to have the same speech spectrum as the COM1 microphone signal 206 or the first beamforming signal 214 before providing the equalized signal 226 to the dynamic wind mixer 228. The COM1 microphone signal 206 and the first beamforming signal 214 are assumed to have the same speech spectrum by design.

[0064] Dynamic wind mixer 228 generates a wind mixer output signal 230 based on wind conditions, as indicated by wind indicator signal 220. When wind indicator signal 220 indicates that wind has been detected, dynamic wind mixer 228 switches to dynamic mixing of frequency domain COM1 microphone signal 206 and FF microphone signal 222. The mixing coefficients of COM1 206 and FF microphone signal 226 are determined based on an estimated wind energy ratio between the two signals. In this respect, wind mixer 228 can calculate the energy in each of the frequency domain COM1 microphone signal 206 and the equalized FF microphone signal 226 on a sub-band basis, and then sum the calculated sub-band energies to determine the total energy of each of those signals before determining the ratio between the two sums. In some cases, wind mixer 228 can calculate only the energy within a specific frequency band (e.g., 125 Hz to 2 kHz).

[0065] In some implementations, the mixing of the COM1 microphone signal and the equalized FF microphone signal occurs only below a specific frequency (e.g., 2 kHz), and above that frequency, the dynamic wind mixer 228 crosses over to the first beamforming signal 214. Therefore, depending on the wind conditions, the wind mixer output signal 230 corresponds to either the first beamforming signal 214 or a mixed signal comprising a mixture of the COM1 microphone signal 206 at a lower frequency (e.g., below 2 kHz) and the equalized FF microphone signal 226, and this mixed signal crosses over to the first beamforming signal 214 at a higher frequency (e.g., 2 kHz and above).

[0066] The wind mixer output signal 230, together with the second beamforming signal (or its equalized version, discussed below), is provided to the spectrum enhancer 232 (also referred to as a "noise spectrum subtractor" or "NSS"). The spectrum enhancer 232 uses the wind mixer output signal 230 as a speech estimate and the second beamforming signal as a noise estimate, and enhances the short-time spectral amplitude (STSA) of the user's speech / utterance, thereby reducing noise in the spectrum-enhanced output signal 234. Examples of spectrum enhancement techniques that can be implemented in the spectrum enhancer 232 include spectral subtraction, minimum mean square error, and Wiener filter techniques. The spectrum enhancement via the spectrum enhancer 232 improves the speech-to-noise ratio of the output signal 234. Spectrum enhancement can further improve system performance in the presence of additional noise sources or by altering noise characteristics. The spectrum enhancer 232 can operate on both estimated signals, using their spectral contents to further enhance the user speech component of the output signal 234.

[0067] The output of spectrum enhancer 232 (i.e., spectrum-enhanced output signal 234) is passed through inverse domain converter 236 to generate a time-domain output signal. As described above, inverse domain converter 236 can be configured to perform the opposite function of domain converter 204. That is, inverse domain converter is used to recombine all sub-bands into a single output signal (enhanced speech signal 202) using WOLA synthesis. In some cases, the spectrum-enhanced output signal can first be provided to steady-state noise reduction (SSNR) 238, which helps remove certain ambient noise (such as HVAC noise) and noise in front of the user, and can remove high-frequency noise residue from spectrum enhancement (spectral subtraction). The output of SSNR 238 (“noise reduction output” 238) can then be provided to inverse domain converter 236 to generate output signal 202. Refer to below. Figure 5 Additional details describing SSNR 238.

[0068] In some implementations, output signal 202 may be provided as a voice output signal to be sent to a remote location. In other implementations, the output of inversion converter 236 may be executed. Figure 3 An additional output stage (time domain) processing 300 is used to generate a speech output signal 302. (See reference) Figure 3 Additional output stage processing features may include a sliding high-pass filter 304, etc. The sliding high-pass filter 304 dynamically adjusts how much low-frequency (wind noise) energy is cut from the speech output signal 302. For example, in strong winds, frequencies below 1 kHz may be cut. This reduces wind noise, but it can make the user's voice sound weak. This can be an acceptable trade-off when the wind is strong. However, when wind noise is low, a filter with a lower corner frequency can be applied, allowing the speech output to include more low-frequency energy and therefore sound more natural.

[0069] refer to Figure 2 and Figure 3 In order to select an appropriate high-pass filter, a sliding high-pass filter 304 is provided from the wind energy estimator 242. Figure 2 The wind energy estimator 242 obtains the bandpass (e.g., 250 Hz to 2 kHz) of the second beamforming signal 216 from the second (delay-subtracting) beamformer 212 and calculates the energy of that bandpass as an estimate of the wind energy. The wind energy estimator 242 can calculate the energy on a sub-band basis (for frequencies within the passband) and then sum the calculated sub-band energies to determine the total energy of the bandpass version of the second beamforming signal 216.

[0070] The wind energy estimate 244 is shared with a sliding high-pass filter 304, which maps the energy estimate to one of several different high-pass filters for application, thus striking a trade-off between wind noise reduction and speech naturalness. When wind energy is high, the system selects a high-pass filter with a higher corner frequency. When wind energy is low, the system selects a high-pass filter with a lower corner frequency.

[0071] In some cases, the wearable audio device 100 may provide voice output signals to a distant end only from one of the handsets 102A or 102B. In this regard, the wearable audio device 100 may, for example, use... Figure 2 The illustrated system detects and estimates wind noise on two handsets 102A, 102b and transmits this wind noise to a core processor running a combined handset processing system 128. The combined handset processing system 128 determines which of the handsets 102A, 102B has lower wind power and selects that handset to provide the voice output signal 302 to the remote end. In some cases, one handset (e.g., the right handset 102B) may be designated as the master handset by default, for example, in the absence of wind, and the other handset (e.g., the left handset 102A) will be designated as the slave handset. The master handset provides its voice output signal to the remote end, and the combined handset processing system 128 can be configured to switch handset roles based on which handset has lower wind power—the handset with lower wind power is expected to provide a clearer voice output signal.

[0072] Figure 4 A block diagram illustrating this wind-based role-switching function is shown. The handset switching logic 400 of the combined handset processing system 128 receives data from the wind detector 218. Figure 2 The wind indicator signal 220 and the wind energy estimator 242 ( Figure 2 The wind energy estimation signal 244 is used for both the left and right earpieces. By default, the right earpiece 102B can be designated as the master earpiece to provide its voice output signal to the remote end (state = default right). The combined earpiece processing system 128 checks to determine whether the right earpiece has been set as the master earpiece in the previous state (state_previous == right). If so, and if the wind indicator signal from the right earpiece indicates no wind (wind_right = 0), the state is set to the right earpiece (state = right). Otherwise, if the wind indicator signal from the right earpiece indicates wind conditions (wind_right == 1) and the wind indicator signal from the left earpiece indicates no wind conditions (wind_left == 0), a counter is started, and if those conditions continue for a predetermined amount of time (counter 1 > threshold), the earpiece roles are switched and the left earpiece 102A is set as the master earpiece to provide its voice output signal to the remote end.

[0073] Otherwise, if the wind indicator signals from both the left and right earpieces indicate wind conditions (wind_right == 1 and wind_right == 1), the combined earpiece processing system 128 examines the wind energy estimation signals from both the left and right earpieces. Furthermore, if the estimated wind energy on the left earpiece 102A is less than the estimated wind energy on the right earpiece 102B, this will trigger a role switch, causing the left earpiece 102A to be set as the primary earpiece.

[0074] Refer again Figure 2 In some implementations, the earpiece processing system 124 can be used to estimate the ambient noise level and use that estimate to select one of several different equalization filters to add to the spectrum enhancer 232. The goal here is to broaden the noise spectrum subtraction to obtain more speech bandwidth when the user is in a quiet environment. This has the effect of reducing speech artifacts when the user has an unusual fit or is near a hard surface such as a wall. When the user is in a noisy environment, the system becomes more aggressive in noise reduction.

[0075] In this respect, the handset processing system 124 may include a noise level estimator 246. For example... Figure 2 As shown, the noise level estimator 246 can receive the frequency domain COM2 microphone signal 208 to estimate the ambient noise level by calculating the energy in the signal. In this respect, the noise level estimator 246 can calculate the energy in the frequency domain COM2 microphone signal 208 on a sub-band basis and then sum the calculated sub-band energies to determine the total energy in the signal. In some cases, the noise level estimator 246 can calculate only the energy within a specific frequency band (e.g., 375 Hz to 11025 Hz).

[0076] The calculated ambient noise level is compared to a threshold. When the estimated ambient noise level exceeds the threshold, the noise level estimator 246 determines that the user is in a noisy environment, and when the estimated ambient noise level is below the threshold, the noise level estimator 246 determines that the user is in a quiet environment. When the user is in a noisy environment, the system becomes more proactive in noise reduction.

[0077] Noise level estimator 246 provides a noise flag signal to noise equalizer (EQ) 250. The noise flag signal 248 can be a binary signal (0 or 1) indicating a quiet (0) or noisy (1) condition. Noise EQ 250 also receives a wind flag signal 220 from wind detector 218. Depending on whether the user is in a quiet, noisy, or windy condition, noise EQ 250 smoothly transitions between different equalization filters to favor different noise characteristics, enabling improved noise reduction performance and speech spectrum in each scenario. In some implementations, if wind flag signal 220 indicates a windy condition (the user is in a windy environment), noise EQ 250 will select an equalization filter designed for improved performance in windy conditions. In such a specific implementation, if the wind indicator signal 220 instead indicates a windless condition (the user is not in a windy environment), the noise EQ 250 will look at the noise indicator signal 248 and apply an equalization filter designed for improved performance in noisy conditions or an equalization filter designed for improved performance in quiet conditions, depending on whether the noise indicator signal 248 indicates a noisy condition or a quiet condition.

[0078] The noise EQ 250 applies the selected EQ filter from the EQ filters to the second beamforming signal 216 and provides the equalized beamforming signal 252 to the spectrum enhancer 232 for processing. The equalized beamforming signal 252 is actually the noise reference signal for the spectrum enhancer 232.

[0079] For noise conditions, the noise spectrum is kept in the low frequencies to help ensure that the spectrum enhancer 232 attenuates low-frequency noise, but retains high frequencies to obtain higher speech bandwidth. For quiet conditions, a much more attenuating equalization filter (relative to the noise filter) can be used because there is not much noise to reduce. For windy conditions, a wind EQ filter is selected so that the spectrum enhancer 232 attenuates high-frequency noise, but broadens the low frequencies.

[0080] To achieve consistent and smooth noise estimates, the Voice Activity Detector (VAD) 254 can be used to freeze the ambient noise level estimate while the user is speaking.

[0081] In some cases, VAD 254 can use signal 256 from internal (feedback) microphone 110 to detect speech activity. In some specific implementations, internal microphone signal 110 may be filtered, for example, via acoustic echo canceller (AEC) 258 to provide clean feedback (FB) microphone signal 260 to domain converter 204, and frequency domain clean FB microphone signal 262 (from domain converter 204) may be input to VAD 254. VAD 254 then provides VAD flag signal 264 to noise level estimator 246. VAD flag signal 264 may be a binary signal (0 or 1) indicating a speech (user is speaking) or no speech (user is not speaking) state. When VAD flag signal 264 indicates that the user is speaking, noise level estimator 246 freezes the ambient noise level estimate until that state disappears.

[0082] As described above, some specific implementations may include a steady-state noise reduction (SSNR) 238 that receives a spectrum-enhanced output signal 234 from a spectrum enhancer 232 and provides further noise reduction before providing a noise-reduced output signal 240 (a noise-reduced version of the enhanced output signal 234) to an inverse domain converter 236. The SSNR 238 removes certain noises, such as HVAC noise, noise in front of the user (e.g., from a computer fan), and clears high-frequency noise residue from the spectrum enhancer 232. (See reference...) Figure 5 Each sub-band of the spectrum-enhanced output signal 234 passes through two energy trackers: a speech tracker 502 and a noise tracker 504. The speech tracker 502 has fast start and slow release, and the noise tracker 504 has slow start and fast release to estimate the band-wise SNR via a band-wise signal-to-noise ratio (SNR) estimator 506. The SNR is then mapped to a negative attenuation coefficient for each band (via a band-wise gain selector 508) and subsequently applied to the signal 234 via a band-wise gain 510.

[0083] Refer again Figure 3 In some implementations, the additional output stage processing 300 may include a speech equalizer (EQ) 306. The speech EQ 306 receives a noise flag signal 248 from the noise level estimator 246 and applies different equalization filters (e.g., a "quiet" EQ filter or a "noise" EQ filter) to the output of the inverse domain converter 236 to generate a speech output signal 302. Because the system is able to detect whether the user is in a quiet or noisy environment, it can smoothly switch between the two speech EQ filters to improve spectral naturalness.

[0084] The equalized output signal 308 is provided to a sliding high-pass filter 304, which applies a selected high-pass filter based on the wind energy estimate 244 to provide a filtered output signal 310. In some implementations, the filtered output signal 310 may pass through a limiter 312 before being sent to the remote end.

[0085] Depending on the specific implementation, wearable audio devices offer technical benefits for enhancing voice pickup during challenging environmental conditions, such as strong winds or noise.

[0086] It should be noted that the specific implementations described herein are particularly useful for two-way communications such as telephone calls, especially when using an earpiece. However, the beneficial effects extend beyond telephone calling applications. These techniques are also applicable to aerospace and military applications where high-noise pickup is desired using an earpiece. Further potential uses include peer-to-peer applications where voice pickup can be freed from the often-present echo problem. Other use cases may involve automotive "car-wearable" applications, wake words or other human-machine interfaces in environments where external microphones would not work reliably, self-voice recording / analysis applications that provide a discreet environment without picking up external conversations, and any application where multiple external microphones are not feasible. Furthermore, by avoiding the pickup of nearby conversations, these implementations can be useful in work-from-home or call center applications, thus providing user privacy.

[0087] It should be understood that one or more functions of the system may be implemented as hardware and / or software, and various components may include communication paths connecting components via any conventional means (e.g., hardwired and / or wireless connections). For example, one or more non-volatile devices (e.g., centralized or distributed devices such as flash memory devices) may store and / or execute programs, algorithms, and / or parameters for one or more of said devices. Furthermore, the functions described herein, or portions thereof, and various modifications thereof (hereinafter referred to as "functions"), may be implemented at least in part via computer program products, such as computer programs tangibly implemented in an information carrier, such as one or more non-transitory machine-readable media, for performing or controlling the operation of one or more data processing devices, such as programmable processors, computers, multiple computers, and / or programmable logic components.

[0088] Computer programs can be written in any programming language (including compiled or interpreted languages) and can be deployed in any form (including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment). Computer programs can be deployed on a single computer, distributed across a site or multiple sites, or executed on multiple computers interconnected by a network.

[0089] Actions associated with implementing all or part of the functionality can be performed by one or more programmable processors executing one or more computer programs to perform the functionality. All or part of the functionality can be implemented as special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) and / or ASICs (Application-Specific Integrated Circuits). Processors suitable for executing computer programs include, for example, both general-purpose microprocessors and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Generally, a processor can receive instructions and data from read-only memory or random access memory, or both. The components of a computer include a processor for executing instructions and one or more memory devices for storing instructions and data.

[0090] It should be noted that although the specific implementation described herein utilizes a microphone system to collect input signals, it should be understood that input signals can be collected using any type of sensor, such as accelerometers, thermometers, optical sensors, cameras, etc., either alone or in conjunction with a microphone system.

[0091] Additionally, one or more networked computing devices may perform actions associated with implementing all or part of the functions described herein. Networked computing devices may be connected via networks such as one or more wired and / or wireless networks such as local area networks (LANs), wide area networks (WANs), personal area networks (PANs), internet-connected devices and / or networks and / or cloud-based computing (e.g., cloud-based servers).

[0092] In various specific implementations, electronic components described as "coupled" can be linked via conventional hardwired and / or wireless devices, enabling these electronic components to transmit data to each other. Additionally, sub-components within a given component can be considered to be linked via conventional paths, which may not necessarily be illustrated.

[0093] Several specific embodiments have been described. However, it should be understood that additional modifications may be made without departing from the scope of the inventive concept described herein, and therefore, other specific embodiments are within the scope of the following claims.

Claims

1. A wearable two-way communication audio device, comprising: A first microphone, which provides a first microphone signal; A second microphone, which provides a second microphone signal; A third microphone, which provides a third microphone signal; One or more processors, said one or more processors being configured to: The first microphone signal and the second microphone signal are processed to form a first beamforming signal; The energy in the first beamforming signal is compared with the energy in the first microphone signal; If the energy in the first beamforming signal exceeds the energy in the first microphone signal, then the first microphone signal and the third microphone signal are mixed to provide a mixed signal; and The mixed signal is used to generate a voice output signal for transmission to a remote receiver.

2. The wearable two-way communication audio device according to claim 1, wherein mixing the first microphone signal and the third microphone signal comprises: Calculate the energy ratio between the first microphone signal and the third microphone signal, and select a mixing coefficient for the first microphone signal and the third microphone signal based on the calculated energy ratio.

3. The wearable two-way communication audio device of claim 1, wherein generating the voice output signal using the mixed signal comprises: The mixed signal is used to generate a first signal component within a first frequency range for the speech output signal, and the beamforming signal is used to generate a second signal component within a second frequency range for the speech output signal; as well as The first signal component and the second signal component are combined to provide the speech output signal.

4. The wearable two-way communication audio device of claim 1, wherein the one or more processors are configured to mix the first microphone signal and the third microphone signal to provide the mixed signal only when the energy in the first beamforming signal exceeds the energy in the first microphone signal by a predetermined threshold.

5. The wearable two-way communication audio device of claim 4, wherein the one or more processors are configured such that the first beamforming signal is used to generate the voice output signal if the energy in the first beamforming signal does not exceed the energy in the first microphone signal by a predetermined threshold.

6. The wearable two-way communication audio device of claim 5, wherein the one or more processors are configured such that if the energy in the first beamforming signal does not exceed the energy in the first microphone signal by a predetermined threshold, the first microphone signal and the third microphone signal are not mixed.

7. The wearable two-way communication audio device of claim 1, wherein the one or more processors are configured such that if the energy in the beamforming signal does not exceed the energy in the first microphone signal, the first beamforming signal is used to generate the voice output signal, and the mixed signal is not used to generate the voice output signal.

8. The wearable two-way communication audio device of claim 1, wherein the one or more processors are configured such that if the energy in the first beamforming signal exceeds the energy in the first microphone signal, the first microphone signal and the third microphone signal are mixed to provide the mixed signal, and the voice output signal is generated using the combination of the mixed signal and the first beamforming signal.

9. The wearable two-way communication audio device of claim 9, wherein the one or more processors are configured such that the first beamforming signal is used to provide a first signal component, and the mixed signal is used to provide a second signal component, the first signal component including frequency content above a predetermined frequency, and the second signal component including frequency content below the predetermined frequency; and The first signal component and the second signal component are combined to provide the speech output signal.

10. A wearable two-way communication audio device, comprising: Multiple microphones; as well as One or more processors, said one or more processors being configured to: The signals from the plurality of microphones are processed to form a first beamforming signal; Wind energy is estimated based on the first beamforming signal; The high-pass filter is adjusted based on the estimated wind energy; and The high-pass filter is used to filter another signal to provide a voice output signal.

11. The wearable two-way communication audio device of claim 10, wherein the one or more processors are configured to: The first beamforming signal is filtered using a bandpass filter to provide a bandpass filtered signal, and The bandpass filtered signal is used to estimate the wind energy.

12. The wearable two-way communication audio device of claim 10, wherein the one or more processors are configured to adjust the high-pass filter by mapping the estimated wind energy to one of a plurality of high-pass filters, each having a different corner frequency.

13. The wearable two-way communication audio device of claim 12, wherein the one or more processors are configured to select a first high-pass filter with a higher corner frequency when the estimated wind energy is high, and to select a second high-pass filter with a lower corner frequency when the estimated wind energy is low.

14. The wearable two-way communication audio device of claim 12, wherein the plurality of high-pass filters comprises at least five high-pass filters.

15. The wearable two-way communication audio device of claim 14, wherein the plurality of high-pass filters comprises at least 10 high-pass filters.

16. The wearable two-way communication audio device of claim 10, wherein the one or more processors are configured to adjust the high-pass filter by adjusting the corner frequency of the high-pass filter.

17. The wearable two-way communication audio device of claim 10, wherein the one or more processors are configured to process signals from the plurality of microphones to form a second beamforming signal, and to use the second beamforming signal to generate the other signal.

18. A wearable two-way communication audio device, comprising: A first earpiece, the first earpiece including a plurality of microphones; The second earpiece includes a second plurality of microphones; One or more processors, said one or more processors being configured to: The signals from the first plurality of microphones are processed to form a first beamforming signal; The signals from the first plurality of microphones are processed to form a second beamforming signal; The signals from the second plurality of microphones are processed to form a third beamforming signal; The signals from the second plurality of microphones are processed to form a fourth beamforming signal; The first wind signal derived from the second beamforming signal is compared with the second wind signal derived from the fourth beamforming signal; and The first earpiece or the second earpiece is selected based on a comparison between the first wind signal and the second wind signal to provide a voice output signal for transmission to a remote receiver.

19. The wearable two-way communication audio device of claim 18, wherein the one or more processors are further configured to: The third wind signal derived from the first beamforming signal is compared with the fourth wind signal derived from the third beamforming signal; and The first or second earpiece is selected, at least in part, based on a comparison of the third wind signal and the fourth wind signal, to provide the voice output signal.

20. The wearable two-way communication audio device of claim 19, wherein the one or more processors are further configured to: A first wind energy estimate is calculated based on the first beamforming signal, and a first wind indicator is set based on the first wind energy estimate; and A second wind energy estimate is calculated based on the third beamforming signal, and a second wind flag is set based on the second wind energy estimate. The third wind signal corresponds to the first wind sign, and the fourth wind signal corresponds to the second wind sign.

21. The wearable two-way communication audio device of claim 20, wherein the first earpiece is selected to provide the voice output signal if the first wind indicator indicates a windless condition on the first earpiece and the second wind indicator indicates a windy condition on the second earpiece.

22. The wearable two-way communication audio device of claim 20, wherein the one or more processors are further configured to: The third wind energy estimate is calculated based on the second beamforming signal; The fourth wind energy estimate is calculated based on the fourth beamforming signal; and The first or the second earpiece is selected based on a comparison between the third and fourth wind energy estimates to provide the voice output signal.

23. The wearable two-way communication audio device of claim 22, wherein the first wind signal corresponds to the third wind energy estimate, and the second wind signal corresponds to the fourth wind energy estimate.

24. The wearable two-way communication audio device of claim 22, wherein the one or more processors are configured such that if both the first wind indicator and the second indicator indicate wind conditions, the one or more processors compare the third wind energy estimate with the fourth wind energy estimate, and if the third wind energy estimate is lower than the fourth wind energy estimate, the first earpiece is selected to provide the voice output signal.

25. The wearable two-way communication audio device according to claim 18, wherein, In the absence of wind, the second earpiece is selected by default to provide the voice output signal.

Images