Integrated low latency automixer with voice and noise activity detection

Abstract

A system and method are disclosed for providing voice and noise activity detection integrated with an audio automixer that can remove irregular non-voice or non-human noise while maximizing signal-to-noise ratio and minimizing audio latency.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 62 / 855,491, filed May 31, 2019, which is incorporated herein by reference in its entirety.

[0002] This application relates generally to systems and methods for providing low latency voice and noise activity detection integrated with an audio automixer, and more particularly, to systems and methods for providing voice and noise activity detection integrated with an audio automixer that can remove errant non-voice or non-human noise while maximizing signal-to-noise ratio and minimizing audio latency. [Background technology]

[0003] Meeting and presentation environments, such as conference rooms, meeting settings, and the like, may involve the use of multiple microphones or microphone array lobes to capture sound from various audio sources. The audio sources may include, for example, a human speaker. The captured sound may be disseminated to a local audience within the environment via amplified speakers (for sound reinforcement) and / or to others remote from the environment (e.g., via telecasting and / or webcasting). Each microphone or array lobe may form a channel. The captured sound may be input as multi-channel audio and provided as a single mixed audio channel.

[0004] Typically, the captured sound may also include random non-speech or non-human noises in the environment, such as sudden, impulsive, or recurring sounds like shuffling paper, opening bags and containers, chewing, and typing. To minimize the random noises contained in the captured sound, a voice activity detection (VAD) algorithm and / or an automixer may be applied to microphone or array lobe channels. The automixer can automatically reduce the strength of the audio input signal of a particular microphone when not capturing human speech or voice to reduce the impact of background, static, or stationary noise. VAD is a technology used in speech processing that can detect the presence or absence of human speech or voice. Additionally, noise reduction techniques can reduce certain background, static, or stationary noises, such as the noise of fans and HVAC systems. However, such noise reduction techniques are not ideal for reducing or eliminating random noises.

[0005] While current systems combine automixing with VAD, such combinations typically fail to substantially remove random noise, especially when low audio latency is required for real-time communication or in-room public address. Because automixers typically rely on relatively simple channel selection rules, such as first arrival time or maximum amplitude at a given moment, removing random noise can degrade the performance of a typical automixer. Current systems that integrate automixing and VAD may be suboptimal due to long latency and / or front-end clipping (FEC) of speech or audio. For example, aligning the VAD detection delay with the onset of speech to minimize FEC on syllables or words within speech or audio may add additional audio latency to the channel, resulting in unacceptable delays in the audio stream. Alternatively, FEC may be tolerated by determining not to add audio latency to align the VAD detection delay with the audio stream, but this may result in imperfect speech or speech in the audio stream. These situations may result in reduced user satisfaction. Additionally, many current systems that use VAD may only utilize a single audio channel and do not need to consider the spatial relationship between speech / voice and noise occurring in a particular environment for effective operation.

[0006] Furthermore, in automixing applications (using individual microphone units or audio lobes steered from a microphone array), speech and random noise may occur in the same environment and be present in all microphones and / or lobes due to the imperfect acoustic polar patterns of the microphones and / or lobes. This can create problems with VAD detection (both individual and aggregated channel approaches), proper automixer channel selection (attempting to avoid random noise while selecting channels containing speech), and suppression of random noise in lobes gated on because they contain speech / voice.

[0007] Thus, an opportunity exists for systems and methods that address these concerns. More particularly, an opportunity exists for systems and methods that can provide speech and noise activity detection coupled with an audio automixer that can remove irregular non-speech or non-human noise while maximizing signal-to-noise ratio, increasing intelligibility, minimizing audio latency, and increasing user satisfaction. By combining automixing principles with more advanced voice activity detection techniques, microphone / lobe selection can be enhanced to maximize speech-to-irregular noise ratio. Summary of the Invention

[0008] The present invention aims to solve the above problems by providing a system and method that is designed to, among other things: (1) utilize a modified voice activity detector that is adapted to function as a noise activity detector to detect whether voice or random noise is present in a channel; (2) perform additional channel gating based on metrics and decisions from the voice activity detector that may affect and / or override the channel gating performed by the automixer; (3) reduce or eliminate the amount of front-end clipping of captured voice / speech; and (4) minimize the impact of front-end noise leakage from random noise that may initially be contained in a particular gated-on channel.

[0009] In one embodiment, the method includes determining whether non-speech audio is present in an audio signal of a channel that was initially gated on by a mixer, the mixer generating a mixed audio signal based at least on the audio signal of the channel that was initially gated on; and if non-speech audio is determined to be present in the audio signal of the channel that was initially gated on, overriding the mixer by gating off the channel that was initially gated on to cause the mixer to generate a mixed audio signal that is free of the audio signal of the channel that was initially gated on.

[0010] In another embodiment, the system includes an activity detector configured to determine whether non-speech audio is present in the audio signal of the channel initially gated on by the mixer, the mixer configured to generate a mixed audio signal based at least on the audio signal of the channel initially gated on. The system also includes a channel gating module in communication with the activity detector, the channel gating module configured to override the mixer to gate off the channel initially gated on and generate a mixed audio signal without the audio signal of the channel initially gated on if the activity detector determines that non-speech audio is present in the audio signal of the channel initially gated on.

[0011] These and other embodiments, as well as various permutations and aspects, will become apparent and more fully understood from the following detailed description and accompanying drawings that set forth illustrative embodiments that illustrate various ways in which the principles of the present invention may be employed. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of a system including a mixer and a voice activity detector for channel gating, according to some embodiments. [Figure 2] 2 is a flowchart illustrating operations for gating a channel from a microphone using the system of FIG. 1 according to some embodiments. [Figure 3] 2 is a diagram of an exemplary gate control state machine used in the mixer of the system of FIG. 1 in accordance with some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0013] The following description describes, illustrates, and illustrates one or more specific embodiments of the present invention in accordance with the principles of the present invention. This description is not intended to limit the present invention to the embodiments described herein, but rather to explain and teach the principles of the present invention in a manner that enables those skilled in the art to understand and, under that understanding, apply them to practice not only the embodiments described herein, but also other embodiments that may come to mind in accordance with these principles. The scope of the present invention is intended to encompass all such embodiments that may fall within the scope of the appended claims, either literally or under the doctrine of equivalents.

[0014] It should be noted that in the present description and drawings, equivalent or substantially similar elements may be labeled with the same reference numerals. However, these elements may be labeled with different numerals, for example, when such labeling would make the description clearer. Furthermore, the drawings described herein are not necessarily drawn to scale, and in some cases, proportions may be exaggerated to more clearly depict particular features. Such labeling and drawing practices do not necessarily suggest an underlying essential purpose. As stated above, the specification is intended to be taken as a whole and interpreted in accordance with the principles of the present invention as taught herein and as understood by those skilled in the art.

[0015] The systems and methods described herein can generate a mixed audio signal from an automixer that reduces and minimizes the effects of random non-speech or non-human noise detected in the environment. These systems and methods can utilize an automixer in conjunction with a voice activity detector (or random noise activity detector), each making independent channel gating decisions. The automixer can gate on or off specific channels based on channel selection rules, and the voice / random noise activity detector can override the automixer's channel gating decisions depending on whether voice or random noise is detected in the channels gated on by the automixer. Metrics from the voice / random noise activity detector, such as a confidence score, can also influence the channel gating decisions and / or the relative mix selected for each channel in the automixer. To support low-latency audio output, some random noise may leak into the audio mix before the voice / random noise activity detector can override the audio mixer. These systems and methods can enable this operation while minimizing the impact of this channel gating on noise onset energy and subjective sound quality. This allows minimizing the energy from random noise leaking into the channel while maintaining low latency.

[0016] Figure 1 is a schematic diagram of a system 100 that can be utilized to remove random noise, including a microphone 102, a mixer 104, and a voice activity detector 108. Figure 2 is a flowchart of a process 200 for removing random noise using the system 100 of Figure 1. The system 100 and process 200 may result in an output mixed audio signal that has an optimal signal-to-noise ratio and contains desirable speech while minimizing the inclusion or influence of random noise.

[0017] Environments such as conference rooms may utilize system 100 to facilitate communication with people in remote locations, for example. The type of microphones 102 and their placement in a particular environment may depend on the location of the audio source, physical space requirements, aesthetics, room layout, and / or other considerations. For example, in some environments, microphones may be placed on a table or podium near the audio source. In other environments, microphones may be mounted overhead, for example, to capture sound from throughout the room. Communication system 100 may operate in conjunction with any type and number of microphones 102. The various components included in communication system 100 may be implemented using software executable by one or more servers or computers, such as, for example, a computing device having a processor and memory, a graphics processing unit (GPU), and / or with hardware (e.g., discrete logic circuits, application-specific integrated circuits (ASICs), programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.).

[0018] In general, a computer program product according to embodiments includes a computer-usable storage medium (e.g., standard random access memory (RAM), an optical disk, a universal serial bus (USB) drive, etc.) having computer-readable program code embodied therein, the computer-readable program code adapted to be executed by a processor (e.g., in conjunction with an operating system) to implement the methods described below. In this regard, the program code may be implemented in any desired language, and may be implemented as machine language (e.g., via C, C++, Java, Actionscript, Objective-C, Javascript, CSS, XML, and / or others), assembly code, bytecode, interpreted executable source code, etc.

[0019] 1 , system 100 may include microphones 102, mixers 104, premixers 106, voice activity detectors 108, and channel gating modules 110. Each microphone 102 may detect sounds in an environment, convert the sounds into audio signals, and form channels. In an embodiment, some or all of the audio signals from microphones 102 may be processed by a beamformer (not shown) to generate one or more beamformed audio signals, as known in the art. Thus, although these systems and methods are described herein as using audio signals from microphones 102, it is contemplated that these systems and methods may utilize any type of acoustic source, such as beamformed audio signals generated by a beamformer.

[0020] The audio signals from each microphone 102 may be received by a mixer 104, a premixer 106, and a voice activity detector 108, such as in step 202 of a process 200 shown in Figure 2. The mixer 104 may ultimately generate and output a mixed audio signal that may match a desired audio mix, such that audio signals from particular microphones are emphasized and audio signals from other microphones are deemphasized or suppressed. Exemplary embodiments of an audio mixer are disclosed in commonly assigned patents, U.S. Pat. Nos. 4,658,425 and 5,297,210, each of which is incorporated by reference in its entirety.

[0021] The mixed audio signal from the mixer 104 may include contributions from one or more channels, i.e., audio signals from microphones 102 gated on using the system 100. The mixer 104 and channel gating module 110 may provide the captured audio without suppression (or, in certain embodiments, with minimal suppression) by gating on one or more channels in response to determining that the captured audio includes human speech and / or according to certain channel selection rules. The mixer 104 and channel gating module 110 may also reduce the intensity of certain captured audio by gating off one or more channels in response to determining that the captured audio in a channel is background noise, static noise, or stationary noise. At step 204, a channel gating decision may be made by the mixer 104 and channel gating module 110. The mixer 104 and channel gating module 110 may make a channel gating decision for each of a plurality of channels corresponding to a plurality of microphones or array lobes 102. Process 200 may continue at step 206.

[0022] If, in step 206, a channel was determined to be gated off in step 204, process 200 may proceed to step 218, where mixer 104 may output a mixed audio signal that does not include the gated-off channel. However, if, in step 206, a channel was determined to be gated on in step 204, process 200 may continue to step 208, where, in particular embodiments, a non-speech de-emphasis filter may be applied that functions as a bandwidth-limiting filter (e.g., a low-pass filter, a band-pass filter, a linear predictive coding (LPC), etc.) to subjectively minimize front-end noise leakage, as described in further detail below.

[0023] In step 210, the audio signal from the microphone 102 may also be received by a voice activity detector (VAD) 108. The VAD 108 may execute an algorithm to determine whether speech is present in a particular channel or, conversely, whether noise is present in a particular channel in step 210. For example, if the VAD 108 finds speech present (or no noise is found) in a particular channel, the VAD 108 may consider the channel to contain speech or to be “not noise.” Similarly, if the VAD 108 finds no speech present (or finds noise) in a particular channel, the VAD 108 may consider the channel to contain noise or to be “not speech.” In embodiments, the VAD 108 may be implemented by analyzing the spectral variance of the audio signal, using linear predictive coding (LPC), applying machine learning or deep learning techniques to detect speech, and / or using well-known techniques such as ITU G.729 VAD, the ETSI standard for VAD calculation included in the GSM standard, or long-term pitch prediction.

[0024] By identifying whether a particular channel contains random noise (i.e., "not speech"), system 100 can override the decisions made by mixer 104 and channel gating module 110 to gate on the channel and subsequently gate off such channel so that the random noise is not ultimately included in the mixed audio signal output from mixer 104. Specifically, if, at step 212, it was determined at step 210 that random noise was present in the channel, process 200 may continue to step 220. At step 220, the decisions by mixer 104 and channel gating module 110 to gate on the channel may be overridden due to the detection of random noise, and the channel may be gated off. Process 200 may continue to step 218, where mixer 104 may output a mixed audio signal that does not include a contribution from the now-gated-off channel. In an embodiment, the confidence score from the VAD 108 may be used to determine whether a decision by the mixer 104 to gate on a channel can be overridden to gate off a channel, and / or the confidence score may be used to influence the relative mix selected for each channel in the automixer.

[0025] On the other hand, if it was determined in step 210 that speech is present in the channel (i.e., "noise is present") in step 212, process 200 may continue to step 214. In step 214, the filter applied in step 208 may be removed, as described in more detail below. In step 216, the channel may remain gated on by mixer 104, and in step 218, mixer 104 may output a mixed audio signal including this channel.

[0026] In an embodiment, steps 210 and 212 by the VAD 108 to identify whether speech or noise is present in a channel may be performed in parallel or immediately after the mixer 104 and channel gating module 110 make their channel gating decisions in steps 204 and 206. For example, the VAD 108 may collect and buffer audio data from the input audio signal for a predetermined period of time to obtain sufficient information to determine whether a channel contains speech or noise. Thus, in the period between the mixer 104's decision and the VAD 108's decision (regarding whether to override or not override the mixer 104 and channel gating module 110's decision), random noise may temporarily affect the mixed audio signal. This short-term random noise effect may be referred to as front-end noise leakage (FENL). The occurrence of FENL in a mixed audio signal may be considered more desirable and less noticeable to a listener of the mixed audio signal compared to front-end clipping. The subjective impact of allowing FENL can be minimized by controlling the amplitude and frequency content of the FENL period and the selected length of time for which FENL is allowed.

[0027] In an embodiment, the mixer 104 may include a gating state machine that controls the final application of channel gating based on the decisions of the mixer 104, the channel gating module 110, and the VAD 108. This state machine may include (1) an FEC period controlled by algorithmic design outside the design of the mixer 104 and the channel gating module 110 that delays the gate-on time; (2) a specific time during the FENL period during which the mixer 104 and the channel gating module 110 have full control over the channel gating; and / or (3) a final period during which the gating instructions from the mixer 104 and the channel gating module 110 may be logically ANDed with the gating instructions from the VAD 108. When the gating instructions of the mixer 104 and the channel gating module 110 return to gating off the channel, the gating state machine may be returned to its starting state. A diagram of the gating state machine is shown in FIG. 3.

[0028] The impact of FENL on the mixed audio signal may be minimized using various techniques, detailed below, by minimizing the energy and spectral impact of random noise that may temporarily leak into a particular channel. Minimizing the impact of FENL on the mixed audio signal may reduce the impact on speech and voice in the mixed audio signal during periods when FENL may occur. Such FENL minimization techniques may, in some embodiments, be implemented in the premixer 106.

[0029] The premixer 106, in some embodiments, may receive state information from the voice activity detector 108. The state information may include a combination of an automixer gating flag, a VAD / NAD indicator, and a FENL period. The premixer 106 may use the state information to determine amplitude attenuation and frequency filtering to apply over time. The mixer 104 may receive processed audio signals from the premixer 106. The number of processed audio signals from the premixer 106 to the mixer 104 may be the same as the number of microphones 102 in some embodiments or may be less than the number of microphones 102 in other embodiments.

[0030] One technique may involve applying an attenuated gate-on amplitude until the VAD 108 can affirmatively confirm the mixer 104's decision to gate on the channel. Attenuating a channel during the FENL period can reduce the impact of random noise while having a relatively small impact on the intelligibility of speech in the mixed audio signal. This technique may be implemented in the premixer 106 by applying a simple attenuation to the channel that the automixer most recently gated on within the FENL period window in step 209 and removing the attenuation in step 215. The FENL period window ends after the expiration of a timer corresponding to the length of time that noise is allowed to leak through without appreciably affecting the subjective quality of the speech.

[0031] Another technique may include reducing the audio bandwidth during the FENL period. Reducing the audio bandwidth in this scenario can significantly reduce the impact of having full-band FENL for a certain period (e.g., a few milliseconds) while preserving the frequencies most important to speech or voice intelligibility in the mixed audio signal during the FENL period. This technique may be implemented in the premixer 106 by applying a non-speech suppression filter in step 208 as described above and de-applying the non-speech suppression filter in step 214. After the mixer 104 determines whether to gate a channel on or off (e.g., in steps 204 and 206), but before the VAD 108 determines whether speech or noise is present in the channel, a low-pass filter, for example, may be applied in step 208. If the VAD 108 determines that speech is present in the channel (e.g., in steps 210 and 212), the non-speech suppression filter may be de-applied in step 214. In an embodiment, the non-speech suppression filter in the premixer 106 may be a static, second-order Butterworth filter cross-faded with the unprocessed audio signal from the microphone 102. In other embodiments, the non-speech suppression filter in the premixer 106 may be implemented as two serial, first-order low-pass filters, with the pole locations of the filters moved over time to adaptively limit the low and high bandwidths independently over time to apply more or less filtering. The adaptive control of these filters may correspond to a FENL timer parameter or a VAD reliability metric. In other embodiments, the non-speech suppression filter in the premixer 106 may be implemented as a more complex bandwidth-limiting filter that uses linear predictive coding to preserve the formant structure of speech.

[0032] Other techniques may include modifying the crest factor of the audio to minimize the perception of noise. Many types of random noise may have a higher crest factor than human speech. A sustained high crest factor may be perceived by humans as noisy. Compressing the crest factor of audio in the FENL region to a value below that of human speech can maintain the intelligibility of human speech while reducing the perceived noisiness of random noise. In some embodiments, signals with instantaneous time-domain crest factors above the target can be dynamically compressed to maintain the desired crest factor. In other embodiments, the compression can be modified to be a limiter to better ensure that the resulting audio has the desired crest factor.

[0033] Further techniques may include introducing a predetermined amount of FEC that can psychoacoustically minimize the subjective impact of sharp, momentary random noises (e.g., a pen click, a book dropped on a table, etc.) while not significantly impacting the subjective quality of speech (which typically has a non-instantaneous onset). Introducing FEC in this situation can be further refined to mimic the inverse envelope of momentary random noise, thereby significantly reducing the perception of noise without completely eliminating the onset of speech that would otherwise occur under static attenuation during the FENL period. This is accomplished in step 209 and can be deactivated in step 215 by applying a time-varying attenuation rather than a static one. Using one or more of these techniques can unnoticeably minimize the impact of random noise leaking into the mixed audio signal until the VAD 108 can determine whether speech or noise is present in the channel. This can therefore benefit speech intelligibility without adding audio path latency.

[0034] The above-described FENL minimization techniques can be enhanced by using adaptive techniques that can automatically modify their operation to better suit the environment in which the system 100 is operating. Such adaptive techniques may control parameters such as the time parameters of the gating state machines described above, as well as the inverse FEC envelope shape, the bandwidth reduction value, the amount of attenuation during the FENL period, the time-dependent ingress / egress behavior of the FENL minimization, and / or the time-dependent ballistics of the mixer 104 to gate off channels that the VAD 108 has identified as containing random noise.

[0035] In an embodiment, system 100 may collect statistics for each channel (corresponding to each of multiple microphones or array lobes 102) to identify whether a particular channel contains, on average, voice / speech or noise. For example, in a particular environment, one channel may be directed toward a door and another channel may be directed toward a chairman's position. In this environment, over time, system 100 may determine that the channel directed toward the door contains almost exclusively random noise, while the channel directed toward the chairman's position contains almost exclusively voice. In response, system 100 may adjust the channel directed toward the door to apply a longer, more restrictive FEC, use more aggressive FENL minimization parameters, and / or have the gating state machine prioritize VAD 108 for gating decisions. Conversely, the system 100 may adjust the channel directed toward the chairperson's position to eliminate FEC, reduce the use of FENL minimization techniques, and / or have the gate control state machine provide gating control to the mixer 104 for a longer period of time (which may allow the VAD 108 to be more confident in its decisions regarding noise before overriding and gating off the channel).

[0036] Another technique may include the system 100 allowing adaptation to train only when the VAD 108 reaches a high threshold level of confidence for a particular channel. This may mitigate false positives and / or false negatives in the adaptation operation applied to the FENL minimization technique. A further technique may include the system 100 sampling and analyzing audio envelope data of channels gated on to audio periods that are later tagged as noise by the VAD 108 in order to update the inverse FEC envelope shape described above.

[0037] In embodiments, adaptive behavior may also be applied to the process of gating off channels. For example, during normal speech, system 100 may apply a slow ramp-out to gate off channels to minimize the perception of a rising or falling or varying audio noise floor. As another example, in the presence of noise, system 100 may apply a fast ramp to gate off channels to maximize the effectiveness of gating off channels in response to decisions by VAD 108. In embodiments, system 100 may combine information from mixer 104 and VAD 108 to identify reasons for gating off channels. This information may be used to dynamically change the rate at which channels are gated off. Also, a non-uniform slope of the ramp may be used to perceptually optimize both random noise and speech conditions.

[0038] The system 100 may include additional techniques to address the situation where many or all channels may contain both speech and random noise due to imperfect audio selectivity between the microphones or lobes 102. In this situation, simply gating off the particular channel containing the most random noise may not completely eliminate the random noise from the mixed audio signal. Thus, some of the random noise may still be present in the gated-on channel containing speech. One technique for addressing this situation may include using a noise leakage filter in the premixer 106. The noise leakage filter may be applied during a portion of the time after the VAD 108 determines that speech is present in a particular channel. If another channel is determined to contain random noise (i.e., if the mixer 104's decision to gate on that other channel is overridden by the VAD 108), the noise leakage filter may be applied to the channel containing speech to reduce high-frequency leakage of noise into the channel containing speech. In other words, the noise leakage filter may be applied when there is at least one channel identified as containing random noise and other channels identified as not containing random noise (i.e., containing speech). In an embodiment, the noise leakage filter in the premixer 106 may be a static second-order Butterworth filter crossfaded with the raw audio signal from the microphone 102. In other embodiments, the noise leakage filter in the premixer 106 may be implemented as two serial first-order low-pass filters, with the filter pole locations moved over time to adaptively limit the low and high bandwidths independently over time to apply more or less filtering. The adaptive control of these filters may correspond to the number of other channels identified as noise or the VAD reliability metric. In other embodiments, the noise leakage filter in the premixer 106 may be implemented as a more complex bandwidth-limiting filter that uses linear predictive coding to preserve the formant structure of speech.

[0039] For example, typically when a particular channel is gated off by mixer 104, mixer 104 may attenuate the audio signal of that channel (e.g., by applying a −15 dB attenuation) to preserve room presence, have a consistent noise floor as various channels are gated on and off, and reduce the impact of FEC on channels that are subsequently gated on. Using the noise leakage filter described above, system 100 may reduce the bandwidth of the gated-on channel so that frequencies for speech intelligibility are preserved while random noise frequencies are removed. This may reduce random noise leaking into the gated-on channel.

[0040] In particular embodiments, to further reduce the effects of random noise, if one or more channels are identified by VAD 108 as containing random noise, system 100 may apply additional attenuation (i.e., a change from -15 dB to -25 dB) to all gated-off channels and reduce the bandwidth of those channels.

[0041] It should be noted that standard static noise reduction techniques may be utilized in the system 100. In an embodiment, the VAD 108 may utilize an audio signal from the microphone 102 that is not noise reduced. It may be optimal for the VAD 108 to use the audio signal that is not noise reduced, allowing the VAD 108 to make decisions based on the original noise floor of the audio signal.

[0042] In this application, the use of disjunctions is intended to include conjunctions. The use of definite or indefinite articles is not intended to indicate cardinality. In particular, reference to an object preceded by "the" or preceded by "a" and "an" is also intended to indicate one of a plurality of possible such objects. Furthermore, the conjunction "or" may be used to convey simultaneous features rather than mutually exclusive alternatives. In other words, the conjunction "or" should be understood to include "and / or." The terms "includes," "including," and "include" are inclusive and have the same scope as "comprises," "comprising," and "comprise," respectively.

[0043] The process descriptions or blocks in the figures should be understood as representing modules, segments, or portions of code that contain one or more executable instructions for implementing a particular logical function or step within the process, and alternative implementations are within the scope of embodiments of the invention in which functions may be performed in an order different from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as will be understood by those skilled in the art.

[0044] This disclosure is intended to describe how to make and use various embodiments in accordance with the present technology, but is not intended to limit its true intended fair scope and spirit. The foregoing description is not intended to be exhaustive or to be limited to the precise forms disclosed. Modifications or variations are possible in light of the above teachings. The embodiments have been chosen and described to provide the best illustration of the principles of the described technology and its practical application, and to enable those skilled in the art to utilize the technology of the various embodiments with various modifications as suited to the particular uses contemplated. All such modifications and variations are within the scope of the embodiments, as determined by the appended claims, as may be amended during the pendency of this patent application, and all equivalents thereof, when interpreted in accordance with the extent to which they are fairly, legally, and equitably entitled.

Claims

1. determining whether non-speech audio is present in an audio signal of a channel gated on by a mixer, the mixer generating a mixed audio signal based at least on the audio signal of the gated on channel; if it is determined that the non-speech audio is present in the audio signal of the gated-on channel, overriding the mixer by gating off the gated-on channel to cause the mixer to generate the mixed audio signal without the audio signal of the gated-on channel; A method comprising:

2. 10. The method of claim 1, further comprising: minimizing front-end noise leakage in the audio signal of the gated-on channel during a period between (1) the mixer determining to gate on the gated-on channel and (2) determining whether the non-speech audio is present in the audio signal of the gated-on channel.

3. The method of claim 1 , further comprising applying a non-speech suppression filter to the audio signal of the gated-on channel.

4. determining whether speech audio is present in the audio signal of the gated-on channel; disabling the non-speech suppression filter from the audio signal of the gated-on channel if it is determined that the speech audio is present in the audio signal of the gated-on channel; The method of claim 3 further comprising:

5. 4. The method of claim 3, further comprising disabling the non-speech suppression filter from the audio signal of the gated-on channel after a period of time has elapsed between (1) the mixer determining to gate on the gated-on channel and (2) determining whether the non-speech audio is present in the audio signal of the gated-on channel.

6. The method of claim 1 , further comprising attenuating the audio signal of the gated-on channel.

7. determining whether speech audio is present in the audio signal of the gated-on channel; removing the attenuation from the audio signal of the gated-on channel if it is determined that the speech audio is present in the audio signal of the gated-on channel; The method of claim 6 further comprising:

8. 7. The method of claim 6, further comprising removing the attenuation from the audio signal of the gated-on channel after a period of time has elapsed between (1) the mixer determining to gate on the gated-on channel and (2) determining whether the non-speech audio is present in the audio signal of the gated-on channel.

9. The method of claim 1 , further comprising applying a time-varying attenuation to the audio signal of the gated-on channel.

10. determining whether speech audio is present in the audio signal of the gated-on channel; removing the time-varying attenuation from the audio signal of the gated-on channel if it is determined that the speech audio is present in the audio signal of the gated-on channel; 10. The method of claim 9, further comprising:

11. 10. The method of claim 9, further comprising removing the time-varying attenuation from the audio signal of the gated-on channel after a period of time has elapsed between (1) the mixer determining to gate on the gated-on channel and (2) determining whether the non-speech audio is present in the audio signal of the gated-on channel.

12. 10. The method of claim 1, further comprising applying one or more of a crest factor compressor or a crest factor limiter to the audio signal of the gated-on channel.

13. determining whether speech audio is present in the audio signal of the gated-on channel; disabling the one or more of the crest factor compressor or the crest factor limiter from the audio signal of the gated-on channel if it is determined that the speech audio is present in the audio signal of the gated-on channel; The method of claim 12 further comprising:

14. 13. The method of claim 12, further comprising disabling the one or more of the crest factor compressor or the crest factor limiter from the audio signal of the gated-on channel after a period of time has elapsed between (1) the mixer determining to gate on the gated-on channel and (2) determining whether the non-speech audio is present in the audio signal of the gated-on channel.

15. 10. The method of claim 1, further comprising: applying additional attenuation to the gated-on channel after it is gated off if the non-speech audio is determined to be present in the audio signal of the gated-on channel.

16. 3. The method of claim 2, further comprising: modifying a parameter associated with minimizing the front-end noise leakage based on whether the gated-on channel previously contained the non-speech audio or speech audio.

17. 10. The method of claim 1, wherein overriding the mixer comprises overriding the mixer by controlling a rate at which the gated-on channels are gated off.

18. determining whether speech audio is present in the audio signal of the gated-on channel; determining whether non-speech audio is present in a second audio signal in a second channel gated on by the mixer; applying a noise leakage filter to the audio signal of the gated-on channel when it is determined that the speech audio is present in the audio signal of the gated-on channel and the non-speech audio is present in the second audio signal of the gated-on second channel; The method of claim 1 further comprising:

19. 10. The method of claim 1, further comprising: determining, by the mixer, to gate on the gated-on channel based on one or more of: (1) a channel selection rule; or (2) whether the audio signal of the gated-on channel includes speech audio.

20. an activity detector configured to determine whether non-speech audio is present in audio signals of channels gated on by a mixer, the mixer configured to generate a mixed audio signal based at least on the audio signals of the gated on channels; and a channel gating module in communication with the activity detector; and wherein the channel gating module, when the activity detector determines that the non-speech audio is present in the audio signal of the gated-on channel, causes the mixer to: gating off the gated-on channel; generating the mixed audio signal without the audio signal of the gated-on channel; The system is configured to override the mixer so that:

21. 21. The system of claim 20, further comprising a premixer in communication with the mixer, the premixer configured to minimize front-end noise leakage in the audio signal of the gated-on channel during a period between (1) the mixer determining to gate on the gated-on channel and (2) the activity detector determining whether the non-speech audio is present in the audio signal of the gated-on channel.

Images