Context-aware hardware-based voice activity detection

Abstract

Certain aspects of the present disclosure provide a method for performing voice activity detection, comprising: receiving audio data from an audio source of an electronic device; generating a plurality of model input features based on the received audio data using a hardware-based feature generator; providing the plurality of model input features to a hardware-based voice activity detection model; receiving an output value from the hardware-based voice activity detection model; and determining a presence of voice activity in the audio data based on the output value.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit and priority of U.S. Provisional Patent Application No. 16 / 888,522, filed May 29, 2020, the entire contents of which are incorporated herein by reference.

[0003] introduction

[0004] Various aspects of this disclosure relate to power-efficient speech activity detection, and more particularly to systems and methods for performing context-aware hardware-based speech activity detection.

[0005] Voice activity detection is a function implemented by electronic devices to activate the device or some of its functions. For example, it allows people to give commands to electronic devices (such as smart speakers, mobile devices, wearable devices, etc.) without directly interacting with the device physically. Voice activity detection is often implemented as an "always-on" function in electronic devices to maximize its utility. Unfortunately, always-on functions require always-on software and / or hardware resources, which in turn leads to always-on power consumption. Mobile electronic devices, Internet of Things (IoT) devices, etc., are particularly sensitive to such always-on power requirements because they shorten battery life and consume other limited system resources, such as processing power.

[0006] Typically, due to performance limitations in conventional implementations, voice activity detection has been implemented as either a high-accuracy, high-power function or a low-accuracy, low-power function. For example, due to stringent power, performance, and area considerations for mobile applications, voice activity detection is often implemented as a low-accuracy, low-power function in mobile devices. Unfortunately, such low-accuracy implementations often frustrate users who desire better device performance.

[0007] Accordingly, what is needed are systems and methods for performing high-accuracy, low-power speech activity detection.

[0008] Brief Overview

[0009] Some embodiments provide a method for performing speech activity detection, the method comprising: receiving audio data from an audio source of an electronic device; generating a plurality of model input features based on the received audio data using a hardware-based feature generator; providing the plurality of model input features to a hardware-based speech activity detection model; receiving an output value from the hardware-based speech activity detection model; and determining the presence of speech activity in the audio data based on the output value.

[0010] An additional embodiment provides a processing system configured to perform speech activity detection, the processing system comprising: a memory including computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to: receive audio data from an audio source; generate a plurality of model input features based on the received audio data using a hardware-based feature generator; provide the plurality of model input features to a hardware-based speech activity detection model; receive output values ​​from the hardware-based speech activity detection model; and determine the presence of speech activity in the audio data based on the output values.

[0011] Additional embodiments provide a non-transient computer-readable medium including computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform a method for performing speech activity detection, the method comprising: receiving audio data from an audio source of an electronic device; generating a plurality of model input features based on the received audio data using a hardware-based feature generator; providing the plurality of model input features to a hardware-based speech activity detection model; receiving an output value from the hardware-based speech activity detection model; and determining the presence of speech activity in the audio data based on the output value.

[0012] An additional embodiment provides a processing apparatus comprising: means for receiving audio data from an audio source of an electronic device; means for generating a plurality of model input features based on the received audio data using a hardware-based feature generator; means for providing the plurality of model input features to a hardware-based speech activity detection model; means for receiving an output value from the hardware-based speech activity detection model; and means for determining the presence of speech activity in the audio data based on the output value.

[0013] The following description and related figures illustrate certain illustrative features of one or more embodiments in detail. Brief description of the attached diagram

[0015] The accompanying drawings illustrate certain aspects of one or more embodiments and are therefore not intended to limit the scope of this disclosure.

[0016] Figure 1 A context-aware, hardware-based example speech activity detection system is described.

[0017] Figure 2 An example embodiment of the feature generator is described.

[0018] Figure 3 A high-level block diagram of a multi-channel minimum statistics (MCMS) speech activity detection (VAD) circuit is depicted.

[0019] Figure 4 An example inference signal flow for a speech activity detection model is depicted.

[0020] Figure 5 An example sampling method for context detection in audio data is described.

[0021] Figure 6 An example method for performing hardware-based speech activity detection is described.

[0022] Figure 7 An example processing system for performing hardware-based speech activity detection is described.

[0023] To facilitate understanding, the same reference numerals are used where possible to designate common elements in the various figures. Elements and features contemplated in one embodiment may be beneficially incorporated into other embodiments without further elaboration.

[0024] Detailed description

[0025] Various aspects of this disclosure provide apparatus, methods, processing systems, and computer-readable media for performing speech activity detection with higher accuracy and lower power consumption than conventional methods.

[0026] Voice activity detection generally refers to the differentiation of human speech by electronic devices in order to perform certain functions. For example, many consumer electronics devices use voice activity detection (such as recognition of specific keywords) to "wake up" the device, query the device, and enable the device to perform various other functions. Voice activity detection can also be used for more complex functionalities, such as "far-field" voice detection (e.g., from mobile devices placed across a room), user authentication (e.g., via voice signature), voice detection during other audio outputs (e.g., detecting voice commands while music is playing on the device or detecting interrupt commands while a smart assistant is speaking), and voice interaction in complex noisy environments (such as inside a moving vehicle). These are just a few examples, and many other examples are possible.

[0027] Like many other processing tasks on electronic devices, voice activity detection requires power and dedicated hardware and / or software to operate. Furthermore, voice activity detection is generally implemented as an "always-on" function to maximize its utility for users of electronic devices with voice-activated functionality. For devices that are always plugged in, the power usage of always-on voice activity detection functionality is primarily an efficiency consideration; however, for power-sensitive devices with always-on voice activity functionality (e.g., mobile electronic devices, IoT devices, etc.), the considerations are more complex. For example, power usage from always-on functionality may limit the runtime of such devices and reduce capacity available for other system processing needs. Additionally, the hardware associated with voice activity detection may consume valuable and limited space on the chips within mobile electronic devices, space that cannot subsequently be used for other functionalities.

[0028] There are generally two types of speech activity detection implementations: hardware-based speech activity detection and software-based speech activity detection.

[0029] Hardware-based speech activity detection typically uses hardware-implemented circuitry to analyze incoming audio signals to detect the presence of speech, such as by using circuitry systems for signal-to-noise ratio (SNR) detection. Unfortunately, conventional hardware-based speech activity detection exhibits poor accuracy for non-constant audio signals (e.g., high false positive and false negative rates). Therefore, such implementations tend to be highly sensitive to the type of incoming audio (e.g., noise) and thus perform irregularly across different noisy environments (e.g., in a quiet room, on a busy street, or in moving vehicles, to name a few). Furthermore, even when the incoming audio signal has a high SNR, conventional hardware-based systems still suffer from poor accuracy (e.g., in terms of false positive and false negative rates) due to highly non-constant noisy environments.

[0030] Software-based speech activity detection typically uses software-implemented models to analyze incoming audio signals to detect the presence of speech. For example, various software-based preprocessing steps generate input data for machine learning models (such as deep neural network models), which are then used to detect speech in the audio signal through inference. While software-based speech activity detection systems achieve higher accuracy than conventional hardware-based systems, largely due to their ability to run state-of-the-art, retrainable detection models, they require more complex and power-hungry processing hardware, such as dedicated digital signal processors (DSPs), machine learning accelerators, and artificial intelligence processing cores. Even using standard processing units to run software-based models results in significantly higher power consumption than conventional hardware-based speech activity detection systems. Therefore, high-power software-based speech activity detection systems are generally not ideal for power-sensitive devices such as mobile electronic devices and other edge processing devices.

[0031] The embodiments described herein overcome the shortcomings of conventional solutions by utilizing a hardware-based speech activity detection design that includes an efficient feature compression front-end that supplies an efficient hardware-based machine learning model back-end.

[0032] Specifically, the embodiments described herein use Fast Fourier Transform (FFT) to generate compact multi-band feature vectors, which in turn enables multi-band signal-to-noise ratio (SNR) determination based on multi-band minimum statistics (e.g., noise floor determination). Compared to conventional software-based methods with much larger outputs (e.g., 512 bands), using compact FFT outputs (e.g., 16-band (or point or channel) outputs) achieves a smaller model size and a smaller data size, which reduces physical (hardware) model implementation space, memory, and power requirements.

[0033] Furthermore, the embodiments described herein employ a hardware-based machine learning model backend, which enables more robust speech activity detection than conventional hardware-based SNR threshold detection schemes.

[0034] Furthermore, the embodiments described herein implement context awareness to further improve speech detection accuracy. In some embodiments, context awareness allows for the identification of specific background noise types, thus allowing type-specific model parameters to be loaded into a hardware-based machine learning model backend to improve speech activity detection accuracy.

[0035] Thus, the embodiments described herein achieve the best of two conventional solution types: software-implementation level accuracy at hardware-implementation level power efficiency.

[0036] Context-aware hardware-based example speech activity detection system

[0037] Figure 1 A context-aware, hardware-based example speech activity detection system 100 is described.

[0038] System 100 receives audio signals from an audio source in an electronic system (e.g., receiving PCM audio data from an analog microphone, receiving PDM high-definition audio from a digital microphone, etc.). For example, the audio signal may be generated by one or more microphones of an electronic device (such as a mobile electronic device, a smart home device, an Internet of Things (IoT) device, or other edge processing device).

[0039] The received audio signal is processed by feature generator 104. Feature generator 104 can be, for example, a hardware-implemented Fourier transform, such as a Fast Fourier Transform (FFT) function or circuit. Fourier transform is generally a function used to deconstruct a time-domain representation of a signal (such as the received audio signal) into a frequency-domain representation. The frequency-domain representation may include voltage or power present at varying frequencies in the received audio signal. (Regarding...) Figure 2 An example embodiment of feature generator 104 is described in more detail. It is worth noting that other or additional forms of feature generation may be used in other embodiments.

[0040] The features generated by feature generator 104 from the audio signal from audio source 102 are then processed by context detector 106. In this example, context may refer to a predefined or pre-characterized audio environment, such as a quiet room, a busy street, an office environment, aircraft and cars, a shopping mall, a concert, etc. In some embodiments, context is determined based on features generated by feature generator 104 (e.g., by maximum likelihood estimation using changes in the sum and difference of FFT features or by simple threshold comparison). The following is about... Figure 6 An example method for context detection is described in more detail.

[0041] The determined context is used by the speech activity detector 108 to load speech activity models that vary depending on the context, such as those that may be stored in a speech activity model library 110, which in some embodiments may be stored in the memory of an electronic device.

[0042] Context-dependent speech activity models generally improve the ability of speech activity detector 108 to accurately identify speech in the received audio signal. It is worth noting that in some embodiments, context detector 106 may not exist, or context detector 106 may be unsure of the context. In such cases, speech activity detector 108 may use a basic or default speech activity detection model instead of a context-dependent model.

[0043] The speech activity detector 108 uses the speech activity detection model 112 to determine whether the received audio signal contains speech activity. As mentioned above, speech activity may include the presence of human speech in the audio signal, and may further include one or more keywords, sentences parsed by a natural language engine, etc. In some cases, the speech activity detector 108 may use a multi-level speech activity detection model 112.

[0044] Once the voice activity detector 108 determines voice activity in the received audio signal, it generates a signal for another system function 114. For example, keywords in the detected voice activity may cause an application to launch, another part of the electronic device (e.g., the screen, other processor, or other sensors) to wake up, or a query to run locally or at a remote data service. In some embodiments, system function 114 may simply receive an indication that voice activity has been detected, while in other embodiments, system function 114 may receive additional information, such as information specific to the detected voice activity, such as one or more detected keywords in the voice activity. It is worth noting that additional functions (not shown) may exist between the voice activity detector 108 and system function 114, such as additional levels of voice activity detection or analysis.

[0045] Figure 2 An example embodiment of feature generator 200 is depicted, which can be implemented as follows: Figure 1 Feature generator 104.

[0046] Feature generator 200 receives audio signals at signal preprocessor 202. As mentioned above, the audio signals may come from an audio source of an electronic device, such as a microphone.

[0047] The signal preprocessor 202 can perform various preprocessing steps on the received audio signal. For example, the signal preprocessor 202 can split the audio signal into parallel audio signals and delay one of these signals by a predetermined amount of time to prepare these audio signals for input to the FFT circuit.

[0048] As another example, the signal preprocessor 202 can execute window functions, such as Hamming, Hann, Blackman-Harris, Kaiser-Bessel, or other sinusoidal window functions, which can improve the performance of further processing stages (such as the signal domain transformer 204). Generally, the windowing (or windowing) function can be used to reduce the magnitude of discontinuities at the boundaries of each finite sequence of the received audio signal data to improve further processing.

[0049] As another example, the signal preprocessor 202 can convert audio signal data from parallel to serial, or vice versa, for further processing.

[0050] The preprocessed audio signal is then provided to audio data buffer 212, which can store audio data pending voice activity detection and subsequently share data with additional processing levels, such as information about voice activity. Figure 3-5 Further description.

[0051] The preprocessed audio signal is also provided to a signal domain converter 204, which is generally configured to transform the preprocessed audio signal from a first domain to a second domain, such as from the time domain to the frequency domain.

[0052] In some embodiments, the signal domain transformer 204 implements a Fourier transform, such as a Fast Fourier Transform (FFT). For example, in some embodiments, the FFT may be a 16-band (or slot, channel, or point) FFT, which generates a compact feature set that can be efficiently processed by the model. Advantageously, compared to conventional single-channel processing (such as conventional hardware SNR threshold detection), the Fourier transform provides fine-grained spectral domain information about the incoming audio signal.

[0053] The result of the signal domain converter 204 is a set of audio characteristics, such as a set of voltages, power, or energy for each frequency band in the transformed data.

[0054] The set of audio features is then provided to a signal feature filter 206, which is configured to reduce the size of the feature set in the audio feature data or to compress the feature set. In some embodiments, the signal feature filter 206 may be configured to discard certain features from the audio feature set, such as symmetrical or redundant features from multiple frequency bands of a multi-band fast Fourier transform. Discarding this data reduces the overall size of the data stream for further processing and may be referred to as compressing the data stream.

[0055] For example, in some embodiments, since the audio signal is a real number, the 16-band FFT may include eight symmetrical or redundant frequency bands after the power is squared. Thus, the signal characteristic filter 206 can filter out redundant or symmetrical frequency band information.

[0056] The filtered audio feature set is then provided to the noise background analyzer 208, which can be configured to determine the minimum statistics for each feature in the audio feature set.

[0057] In some embodiments, the noise floor analyzer 208 is configured as a buffer with slots for a predetermined number of individual features, which can determine minimum observations (e.g., minimum statistics) over a predetermined number of observations or frames of audio data based on the predetermined number of individual features. These minimum values ​​of each feature in the filtered audio feature set may include a noise floor feature set, which can be used in subsequent processing to determine the signal-to-noise ratio.

[0058] The filtered audio feature set and the noise floor feature set are then provided to a signal-to-noise ratio (SNR) converter 210, which is configured to determine the SNR based on these feature sets. In some embodiments, this conversion is performed using a logarithmic function (such as a base-10 logarithmic function). The logarithmic function can advantageously compress the feature range to make further processing more efficient. Furthermore, the logarithmic function can advantageously reduce or avoid computationally costly division operations by converting division into computationally less costly subtraction (e.g., where the logarithms of the numerator and denominator are subtracted from each other to obtain a value equivalent to a division operation).

[0059] Figure 3 A high-level block diagram is depicted of a multi-channel minimum statistics (MCMS) voice activity detection (VAD) circuit 300 that can be implemented as hardware in an electronic device. The VAD circuit 300 can be as follows: Figure 1 An example of a voice activity detector 108.

[0060] The audio signal 302 may be, for example, 16kHz PCM data. The audio signal data is initially split, with one branch entering a delay buffer 304. In some embodiments, the delay buffer 304 is configured to generate a 50% delay in the audio signal data. For example, if the audio signal has 16 samples, a 50% delay will shift the signal by 8 samples, such as... Figure 5 As shown in the example.

[0061] In one embodiment, the 16kHz PCM incoming signal is grouped into a 16×1 vector with a step size of 8 (i.e., 50% overlap), or in other words, 16 PCM samples after windowing, overlap, and summation.

[0062] The original and delayed branches of the audio signal 302 are then processed by window function 306 and subsequently passed to converters 308A and 308B, respectively. In some embodiments, the window function is a sinusoidal window function, such as a Hamming or Hann window function. Furthermore, in this embodiment, converters 308A and 308B are serial-to-parallel (S / P) converters, which convert serial data streams into parallel data streams.

[0063] The windowed and transformed data stream (e.g., a vector) is then provided to multiplexer 310, which combines the data streams for input to a Fast Fourier Transform (FFT) 312. Generally, FFT 312 is configured to generate power data for multiple individual frequency bands; in other words, a hardware-based FFT circuit.

[0064] In one embodiment, FFT 312 is a 16-band FFT engine with 50% overlap due to delay buffer 304. In this embodiment, FFT 312 decompresses the input signal into multiple frequency bands, each with a defined power value. Notably, since the input audio signal in this example is always a real number, the absolute value of the FFT output from FFT 312 is symmetrical. Thus, in this example, repeating frequency bands are filtered out, and only 8 of the 16 frequency bands (i.e., 314A-314N, where N=8) are retained for subsequent processing.

[0065] In the example where the input audio signal 302 is 16kHz PCM data with 50% overlap via delay buffer 304 and the FFT 312 is a 16-band transform, the output sampling rate at each FFT output port is 16kHz / 16-band FFT / 0.5 overlap = 2kHz.

[0066] Individual band power 314A-N is then provided to single band minimum statistics (SBMS) blocks 316A-N, which are configured to determine the signal power level (e.g., 318A) and noise floor power level (318B) for each individual power band 314A-N.

[0067] In one embodiment, the input to the SBMS 316A-316N is the amplitude-squared output of the individual band power 314A-N determined by the FFT 312. The amplitude-squared FFT output allows for easy determination of the repetition band value. In an example where the FFT 312 is configured to output eight individual bands 314A-N, there are eight corresponding SBMS blocks (316A-N). In one embodiment, the sampling rate of the output signal of each SBMS block is 100Hz or 10ms per sample.

[0068] Each SBMS block 316A-N can be configured to use, for example, integration and dumping circuitry to calculate the signal power. In such an example, the input signal from the FFT 312 is first squared in magnitude and then accumulated for C cycles, where C gives the desired frame rate. Thus, for a 2kHz input rate and a 10ms frame rate, C is 20.

[0069] Each SBMS block 316A-N can also be configured to output a base-10 logarithmic signal power and a base-10 logarithmic noise power, the difference of which gives the individual band logarithmic signal-to-noise ratio (SNR) 320A-320N. These SNR values ​​are then fed into model 322 to ultimately determine the speech activity in the audio signal 302.

[0070] In other embodiments, other measurements may be used as inputs to model 322. For example, in some cases, only a signal component (e.g., 318A) may be provided to model 322, and this signal component may be compared with a baseline noise floor or a constant.

[0071] In one embodiment, model 322 is a multi-column support vector machine (SVM) model, and the speech activity detection decision is made for each individual FFT band. In other embodiments, model 322 may be a single-column SVM for lower power applications.

[0072] It is worth noting that in this embodiment, model 322 is implemented in hardware. The parameters of model 322 can be stored in memory (not depicted) and loaded. As described above... Figure 1 As shown, model 322 can load different model parameters (e.g., weights and biases) based on different contexts determined for audio signal 302.

[0073] Figure 4 Describing models for speech activity detection (such as...) Figure 3 Example of inferring signal flow 400 in model 322).

[0074] Initially, each vector has M eigenvectors with signal-to-noise ratios (SNR) or logarithmic SNRs (such as those derived from...). Figure 3 The feature vectors output by the SBMC316A-N are provided to a serial-to-parallel data converter 404. The converter 404 takes a multidimensional vector input and outputs a set of feature frames (including a target feature frame 406B, F past feature frames 406A, and F future feature frames 406C) to an input feature buffer 422. Here, "past" and "future" are relative terms based on the target feature frame. Thus, in this example, there are a total of 2F+1 frames in the input feature space, and each frame (e.g., 406) has M observations.

[0075] Feature frames 406 are provided to a multi-column SVM model 408 of size (2F+1)*M×H, where H is the number of outputs of the multi-column SVM model 408 (e.g., the number of nodes in the output layer). It is worth noting that in other examples, the SVM model 408 may be replaced by other types of machine learning models (such as neural network models). In this example, the multi-column SVM model 408 is implemented in hardware. In some embodiments, the weights and biases of model 408 may be static, while in other embodiments, the weights and biases may be dynamically loaded, for example, from memory, as described above. Figure 1 As described.

[0076] In the depicted example, the multi-column SVM model 408 outputs a feature map 410 of size 1×H. The feature map 410 is then subjected to a 1×H bias set by the bias operator 412 before being processed by the activation function 414. In some embodiments, the activation function 412 is a non-linear activation function, such as ReLU.

[0077] The activation output is then passed to an H×1 single-column SVM model 416, which in this example outputs a 1×1 scalar value. This scalar value is then compared to a threshold to determine whether any speech activity is detected in the characterized audio signal 402. For example, if the scalar value is greater than or equal to the threshold, speech activity is detected; if the scalar value is less than the threshold, no speech activity is detected, and vice versa.

[0078] Therefore, the signal flow 400 can be described as a multilayer perceptron model, which in this example includes a two-layer SVM model.

[0079] In one example, the feature generation block (as described above) outputs an 8×1 SNR vector every 10ms frames. These frames are then fed to a frame tap delay line to stack 21 frames (10 past frames, one target frame, and 10 future frames), which are then fed into a 168×20 SVM matrix (e.g., 21 frame SNR * 8 features = 168), which functions like a polyphase interpolator. Generally, a polyphase interpolator acquires the signal stream and performs upsampling (interpolation) using a polyphase filter bank (e.g., a filter bank that splits the input signal into a given number of equidistant subbands). Framing the features advantageously includes more audio context (history) information from the input data stream and increases the feature size.

[0080] An SVM matrix is ​​applied to the input data to generate a 1×20 vector per frame. This 1×20 vector is then added to a fixed 1×20 bias vector. Subsequently, a non-linear activation function (e.g., ReLU) is used to create activations and add non-linearity to the detection system. Finally, speech activity detection is performed by applying a 20×1 single-column SVM model to the activations to generate a scalar output that can be compared to a threshold.

[0081] Thus, in this example, a total of 21 frames of data are used for single speech activity detection, including 10 past feature frames (e.g., 406A), 10 future feature frames (e.g., 406C), and a single target feature frame for speech activity detection (e.g., 406B). In this way, the multi-column SVM model 408 processes the signals before and after the target frame (decision point) and makes a joint decision effectively based on these frames.

[0082] Example Context Detection Method

[0083] A beneficial aspect of context awareness is its ability to classify or modify the classification of audio scenes and events. As mentioned above, when the context is determined, a set of context-dependent model parameters are loaded into the model, such as parameters related to... Figure 1 The speech activity models discussed in (112), 3(322) and 4(400).

[0084] In one embodiment, the context detection method buffers N audio samples and then performs a domain transform (e.g., FFT) on the buffered samples to obtain the signal power spectrum. The method then steps through N / 2 samples and repeats the domain transform, as follows: Figure 5 As described in [the text]. Specifically, Figure 5 The text describes a first choice 502 with N audio samples, a step size 504 with N / 2 audio samples, and a second choice 506 with N audio samples, half of which overlaps with the first step.

[0085] In some embodiments, additional filtering may be applied to the spectral coefficients. In some embodiments, N may be relatively small, such as 16, 8, or even 4.

[0086] In some embodiments, each domain transform of the N samples is performed using an FFT, and the spectral coefficients are obtained by squared the absolute values ​​of the coefficients according to the following formula:

[0087]

[0088] Spectral coefficient S t,m This can then be filtered. For example, in some embodiments, a coefficient α = 1 – N / 2048 can be used, which corresponds to a filter lifetime decay of approximately 1 / 20 of a second when using input audio data at 16k samples per second. This can be expressed as:

[0089] γ t,m =αγ t-1,m +(1-α)S t,m ,

[0090] Where γ t,m These are the filtered spectral coefficients.

[0091] The filtered result can then be converted into decibels using the following formula:

[0092] Γ t,m =10log 10 γ t,m

[0093] Where Γ t,m It is the decibel value of the filtered spectral coefficients.

[0094] The total energy E can then be determined using the following formula. tIt is the sum of the filtered spectral coefficients:

[0095] E t =10log 10 ∑γ t,m

[0096] Finally, it can be seen from Γ t,m Subtract E from the middle t (decibels) to produce:

[0097] △ t,m =Γ t,m -E t ,

[0098] Where E t It is the smoothed total energy, and △ t,m These are the smoothed and normalized spectral coefficients.

[0099] To determine whether the audio scene is different or constant, E can be used. t and △ t,m Compare with a threshold. Generally, most of the spectral energy will fall into the first spectral band, and this hypothesis is used to avoid calculating E. t It is possible to subtract the first band from other bands instead.

[0100] If any of the following are true, the method will trigger the detected context change; in other words, it is context-aware:

[0101]

[0102]

[0103]

[0104]

[0105] In the previous expression, the T value was the threshold, and currently the upper and lower thresholds are set to the same value. However, it might be worthwhile to retain the flexibility of keeping the T value m as a separate threshold. A reference value is also required whenever a detected context change occurs. and ), and these reference values ​​are set to E. t and △ t,m The current value. In other words, whenever a new context is detected, the coefficient becomes a new reference for subsequent context changes.

[0106] In some embodiments, the energy threshold is 15 dB and the Δ threshold is 10 dB.

[0107] Therefore, the aforementioned context detection method is designed to be triggered whenever there is a change in the total energy (upward or downward) in the audio or whenever there is a change in the mode of the audio.

[0108] Example methods for performing hardware-based speech activity detection

[0109] Figure 6 An example method 600 for performing hardware-based speech activity detection is described.

[0110] Method 600 begins in step 602 with receiving audio data from an audio source of the electronic device. In some embodiments, the audio source includes one or more microphones of the electronic device.

[0111] Method 600 then proceeds to step 604, where a hardware-based feature generator is used to generate multiple model input features based on the received audio data.

[0112] In some embodiments, the hardware-based feature generator includes a hardware-implemented Fast Fourier Transform circuit, such as those described above. Figure 2 As described in (204) and 3(312). In some embodiments, the feature generator may be... Figure 7 The feature generator circuit 726 in the middle is used to implement this.

[0113] In some embodiments, using the hardware-based feature generator to generate the plurality of model input features includes one or more of the following: preprocessing received audio data to generate domain-transformed input data; using a domain-transformed circuit to generate a plurality of frequency band outputs based on the domain-transformed input data; selecting a subset of frequency band outputs from the plurality of frequency band outputs from the domain-transformed circuit; and determining the signal-to-noise ratio (SNR) of each frequency band output in the subset of frequency band outputs, wherein each SNR of each frequency band output is a model input feature, such as those described above. Figure 2 and 3 As described. In some embodiments, the subset of frequency band outputs includes eight frequency band outputs.

[0114] In some embodiments, preprocessing the received audio data includes one or more of the following: splitting the received audio data into a first audio data stream and a second audio data stream; applying a delay function to the second audio data stream to generate a delayed second audio data stream; applying a window function to the first audio data stream and the delayed second audio data stream; and applying a serial-to-parallel conversion to the first audio data stream and the delayed second audio data stream.

[0115] In some embodiments, determining the signal-to-noise ratio (SNR) of each frequency band output in a subset of frequency band outputs includes one or more of the following: determining the noise floor of each frequency band output in the subset of frequency band outputs; applying a logarithmic function to the noise floor of each frequency band output in the subset of frequency band outputs; determining the signal power level of each frequency band output in the subset of frequency band outputs; and applying a logarithmic function to the signal power level of each frequency band output in the subset of frequency band outputs, wherein the SNR of each frequency band output in the subset of frequency band outputs includes a logarithmic SNR, such as the above regarding... Figure 3 As described (e.g., SNR 320A-N).

[0116] Method 600 then proceeds to step 606, where the plurality of model input features are provided to a hardware-based speech activity detection model. The speech activity detection model can then process the input features based on model parameters (e.g., context-dependent model parameters) to generate model output values.

[0117] Method 600 then proceeds to step 608, where output values ​​are received from the hardware-based speech activity detection model, such as those mentioned above. Figure 3 The described 324 or Figure 4 The output of box 416 in the middle.

[0118] Method 600 then proceeds to step 610, where the presence of speech activity in the audio data is determined based on the output value. In some embodiments, determining speech activity may be implemented by a speech activity detector circuit, such as... Figure 7 730 in the middle.

[0119] In some embodiments, the hardware-based speech activity detection model includes a hardware-implemented SVM model, such as those described above. Figure 1-4 As described above. In some embodiments, the hardware-implemented SVM model includes: a first multi-column SVM circuit; and a second single-column SVM circuit configured to generate the output value, such as, for example, the above regarding Figure 4 As described.

[0120] In some embodiments, method 600 further includes: loading a plurality of model parameters of the SVM model used for the hardware implementation into the memory of the electronic device (e.g., Figure 7 In the memory (724).

[0121] In some embodiments, method 600 further includes disabling speech activity detection for a time interval after determining the presence of speech activity in the audio data to avoid redundant detection. In some embodiments, this time interval may be referred to as a "residual time interval" and may be, for example, by... Figure 7 The legacy circuit 732 is used to implement this.

[0122] In some embodiments, method 600 further includes: determining the context of the audio data; loading context-dependent model parameters into the memory of the electronic device; and determining the presence of speech activity in the audio data based on the context-dependent model parameters (e.g., wherein the hardware-based speech activity detection model uses the context-dependent model parameters to generate model output values). In some embodiments, context detection may be performed, for example, by... Figure 7 The context detector circuit 728 is used to implement this.

[0123] Example processing system for performing hardware-based speech activity detection

[0124] Figure 7 Describes the methods for performing hardware-based speech activity detection (such as, for example, in this paper regarding...). Figure 6 Example processing system 700 (described).

[0125] Electronic device 700 includes a central processing unit (CPU) 702, which in some examples may be a multi-core CPU. Instructions executed at CPU 702 may be loaded, for example, from memory associated with CPU 702 (such as, for example, memory 724).

[0126] The electronic device 700 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 704, a digital signal processor (DSP) 706, a neural processing unit (NPU) 708, a multimedia processing unit 710, and a wireless connectivity component 712.

[0127] An NPU (such as the NPU 708) is generally a dedicated circuit configured to implement all the necessary control and arithmetic logic for executing machine learning algorithms, such as those for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU is sometimes alternatively referred to as a neural signal processor (NSP), tensor processing unit (TPU), neural network processor (NNP), intelligent processing unit (IPU), vision processing unit (VPU), or graphics processing unit.

[0128] NPUs (such as the NPU 708) are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip (such as a system-on-a-chip (SoC)), while in other examples, multiple NPUs may be part of a dedicated neural network accelerator.

[0129] An NPU can be optimized for either training or inference, or in some cases, configured to balance performance between training and inference. For an NPU capable of performing both training and inference, these two tasks may typically still be performed independently.

[0130] NPUs designed to accelerate training are typically configured to speed up the optimization of new models. This involves a highly computationally intensive process: taking an existing dataset (often labeled or sublabeled), iterating over the dataset, and then tuning model parameters (such as weights and biases) to improve model performance. Generally, optimization based on incorrect predictions involves backtracking through the layers of the model and determining gradients to reduce prediction errors.

[0131] NPUs designed to accelerate inference are typically configured to operate on the full model. Such NPUs can thus be configured to take new data segments as input and process those segments quickly using an already trained model to generate model outputs (e.g., inference).

[0132] In one implementation, the NPU 708 is part of one or more of the CPU 702, GPU 704, and / or DSP 706.

[0133] In some examples, the wireless connectivity component 712 may include sub-components for, for example, third-generation (3G) connectivity, fourth-generation (4G) connectivity (e.g., 4G LTE), fifth-generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. The wireless connectivity processing component 712 is further connected to one or more antennas 714.

[0134] The electronic device 700 may also include one or more sensor processing units 716 associated with any type of sensor, one or more image signal processors (ISP) 718 associated with any type of image sensor, and / or may include a navigation processor 720 that may include satellite-based positioning system components (e.g., GPS or GLONASS) and inertial positioning system components.

[0135] Electronic device 700 may also include one or more input and / or output devices 722, such as screens, touch-sensitive surfaces (including touch-sensitive displays), physical buttons, speakers, microphones, etc.

[0136] In some examples, one or more processors of the electronic device 700 may be based on the ARM or RISC-V instruction set.

[0137] The electronic device 700 also includes a memory 724, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 724 includes computer-executable components that can be executed by one or more of the aforementioned processors of the electronic device 700.

[0138] Specifically, in this example, memory 724 includes a determining component 724A, a receiving component 724B, a processing component 724C, an output component 724D, and VAD model parameters 724E. The depicted components, as well as other components not depicted, can be configured to perform aspects of the methods described herein.

[0139] Electronic device 700 further includes feature generator circuitry 726, such as those described above. Figure 1 As described in (104), 2(200) and 3(326).

[0140] Electronic device 700 further includes context detector circuitry 728, such as those described above. Figure 1 (106) describes it.

[0141] Electronic device 700 further includes a voice activity detector circuit 730, such as those described above. Figure 1 The terms described in (108), 3(300), and 4(400) are as follows.

[0142] Electronic device 700 further includes a feature legacy circuit 732, such as those described above. Figure 6 As described.

[0143] Generally, electronic device 700 and / or its components may be configured to perform the methods described herein.

[0144] Example Terms

[0145] Examples of implementations are described in the following numbered clauses.

[0146] Clause 1: A method for performing speech activity detection, comprising: receiving audio data from an audio source of an electronic device; generating a plurality of model input features based on the received audio data using a hardware-based feature generator; and determining the presence of speech activity in the audio data based on an output value generated by a hardware-based speech activity detection model based on the model input features.

[0147] Clause 2: The method of Clause 1, wherein using the hardware-based feature generator to generate the plurality of model input features includes: preprocessing the received audio data to generate domain-transformed input data; using a domain-transformed circuit to generate a plurality of frequency band outputs based on the domain-transformed input data; selecting a subset of frequency band outputs from the plurality of frequency band outputs from the domain-transformed circuit; and determining the signal-to-noise ratio (SNR) of each frequency band output in the subset of frequency band outputs, wherein each SNR of each frequency band output is a model input feature among the plurality of model input features.

[0148] Clause 3: The method of Clause 2, wherein the hardware-based feature generator includes a hardware-implemented Fast Fourier Transform circuit.

[0149] Clause 4: The method of any of Clauses 2-3, wherein determining the signal-to-noise ratio (SNR) of each frequency band output in the frequency band output subset comprises: determining the noise floor of each frequency band output in the frequency band output subset; applying a logarithmic function to the noise floor of each frequency band output in the frequency band output subset; determining the signal power level of each frequency band output in the frequency band output subset; and applying a logarithmic function to the signal power level of each frequency band output in the frequency band output subset, wherein the SNR of each frequency band output in the frequency band output subset includes a logarithmic SNR.

[0150] Clause 5: The method of any of Clauses 2-4, wherein preprocessing the received audio data comprises: splitting the received audio data into a first audio data stream and a second audio data stream; applying a delay function to the second audio data stream to generate a delayed second audio data stream; applying a window function to the first audio data stream and the delayed second audio data stream; and applying a serial-to-parallel conversion to the first audio data stream and the delayed second audio data stream.

[0151] Clause 6: The method of any of Clauses 1-5, wherein the hardware-based speech activity detection model includes a hardware-implemented SVM model.

[0152] Clause 7: The method of Clause 6, wherein the hardware-implemented SVM model comprises: a first multi-column SVM circuit; and a second single-column SVM circuit configured to generate the output value.

[0153] Clause 8: The method of Clause 6 further includes loading multiple model parameters of the SVM model used for the hardware implementation into the memory of the electronic device.

[0154] Clause 9: The method of any of Clauses 2-8, wherein the subset of frequency band outputs comprises eight frequency band outputs.

[0155] Clause 10: The method of any of Clauses 1-9, wherein the audio source includes one or more microphones of the electronic device.

[0156] Clause 11: A processing system comprising: a memory including computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform a method pursuant to any of Clauses 1-10.

[0157] Clause 12: A processing system comprising means for performing a method according to any one of Clauses 1-10.

[0158] Clause 13: A non-transient computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform a method according to any one of Clauses 1-10.

[0159] Clause 14: A computer program product implemented on a computer-readable storage medium, the computer-readable storage medium including code for performing a method according to any one of Clauses 1-10.

[0160] Additional considerations

[0161] The foregoing description is provided to enable any person skilled in the art to practice the various embodiments described herein. The examples discussed herein are not intended to limit the scope, applicability, or embodiments set forth in the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. For example, changes may be made to the function and arrangement of the elements discussed without departing from the scope of this disclosure. Various procedures or components may be appropriately omitted, substituted, or added to various examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Moreover, features described with reference to some examples may be combined in others. For example, any number of aspects set forth herein may be used to implement an apparatus or practice a method. Furthermore, the scope of this disclosure is intended to cover such apparatuses or methods practiced using other structures, functionalities, or structures and functionalities that supplement or differ from the aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure disclosed herein may be implemented by one or more elements of the claims.

[0162] As used herein, the term “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” should not be construed as superior to or better than the others.

[0163] As used herein, the phrase “at least one of” a list of items refers to any combination of those items, including a single member. As an example, “at least one of a, b, or c” is intended to cover: a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0164] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, computation, processing, derivation, research, searching (e.g., looking in a table, database, or other data structure), ascertaining, and the like. Furthermore, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), and the like. Moreover, "determine" can include parsing, selecting, choosing, building, and the like.

[0165] The methods disclosed herein include one or more steps or actions for implementing the method. These method steps and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Furthermore, the various operations of the above methods can be performed by any suitable means capable of performing the corresponding functions. These means may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Generally, where operations illustrated in the drawings are present, these operations may have corresponding paired means with similar numbers plus functional components.

[0166] The following claims are not intended to be limited to the embodiments shown herein, but should be granted the full scope consistent with the language of the claims. Within the claims, references to singular elements are not intended to mean “one and only one” (unless specifically stated so), but rather “one or more.” Unless specifically stated otherwise, the term “some / a” refers to one or more. No element of the claims should be interpreted in accordance with the provisions of 35 U.S.SC §112(f) unless the element is expressly stated using the phrase “means for…” or, in the case of a method claim, the element is stated using the phrase “steps for…”. Elements of all aspects described throughout this disclosure that are now or hereafter known to a person skilled in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be donated to the public, whether or not such disclosure is expressly stated in the claims.

Claims

1. A processing system configured to perform voice activity detection, comprising: The memory includes computer-executable instructions; as well as One or more processors, the one or more processors being configured to execute the computer-executable instructions and enable the processing system to: Receive audio data from the audio source; A hardware-based feature generator is used to generate multiple model input features based on the received audio data, wherein, in order to use the hardware-based feature generator to generate the multiple model input features, the one or more processors are configured to make the processing system: The received audio data is preprocessed to generate domain-transformed input data; Multiple frequency band outputs are generated based on the domain transform input data using a domain transform circuit; The current context is determined based on spectral coefficients associated with the received audio data and spectral coefficients associated with a previous context in which the previous audio data was captured, wherein, in order to determine the current context, the one or more processors are configured to cause the processing system to: When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is within a threshold distance, it is determined that the current context has not changed relative to the previous context, or When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is outside the threshold distance, it is determined that the current context is different from the previous context; By filtering the symmetrical frequency band outputs among the plurality of frequency band outputs, a subset of frequency band outputs is selected from the plurality of frequency band outputs from the domain transformation circuit, wherein the subset of frequency band outputs includes fewer frequency band outputs than the plurality of frequency band outputs; as well as Determine the signal-to-noise ratio (SNR) of each frequency band output in the frequency band output subset, wherein each SNR of each frequency band output in the frequency band output subset is a model input feature among the plurality of model input features; as well as The presence of speech activity in the audio data is determined based on the output value generated by a hardware-based speech activity detection model based on the model's input features, wherein the hardware-based speech activity detection model includes a model that includes context-dependent model parameters associated with the determined current context of the audio data.

2. The processing system of claim 1, wherein the hardware-based feature generator includes a hardware-implemented Fast Fourier Transform circuit.

3. The processing system of claim 1, wherein, in order to determine the signal-to-noise ratio of each frequency band output in the subset of frequency band outputs, the one or more processors are further configured to cause the processing system to: Determine the noise floor of each frequency band output in the frequency band output subset; The logarithmic function is applied to the noise floor of each frequency band output in the subset of frequency band outputs; Determine the signal power level of each frequency band output in the frequency band output subset; as well as The logarithmic function is applied to the signal power level of each frequency band output in the subset of frequency band outputs. The signal-to-noise ratio of each frequency band output in the aforementioned frequency band output subset includes the logarithmic signal-to-noise ratio.

4. The processing system of claim 1, wherein, for preprocessing the received audio data, the one or more processors are further configured to cause the processing system to: The received audio data is split into a first audio data stream and a second audio data stream; The delay function is applied to the second audio data stream to generate a delayed second audio data stream; Apply the window function to the first audio data stream and the delayed second audio data stream; as well as The serial-to-parallel conversion is applied to the first audio data stream and the delayed second audio data stream.

5. The processing system of claim 1, wherein the hardware-based speech activity detection model includes a hardware-implemented support vector machine (SVM) model.

6. The processing system of claim 5, wherein the hardware-implemented SVM model comprises: First multi-column SVM circuit; as well as A second single-column SVM circuit is configured to generate the output value.

7. The processing system of claim 5, wherein the one or more processors are further configured to cause the processing system to: load a plurality of model parameters of the SVM model for the hardware implementation into the memory.

8. The processing system of claim 1, wherein the frequency band output subset comprises eight frequency band outputs.

9. The processing system of claim 1, wherein the audio source includes one or more microphones of the processing system.

10. A method for performing speech activity detection, comprising: Receive audio data from the audio source of the electronic device; A hardware-based feature generator is used to generate multiple model input features based on the received audio data, wherein generating the multiple model input features includes: The received audio data is preprocessed to generate domain-transformed input data; Multiple frequency band outputs are generated based on the domain transform input data using a domain transform circuit; The current context is determined based on spectral coefficients associated with the received audio data and spectral coefficients associated with a previous context in which the previous audio data was captured, wherein determining the current context includes: When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is within a threshold distance, it is determined that the current context has not changed relative to the previous context, or When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is outside the threshold distance, it is determined that the current context is different from the previous context; By filtering the symmetrical frequency band outputs among the plurality of frequency band outputs, a subset of frequency band outputs is selected from the plurality of frequency band outputs from the domain transform circuit, wherein the subset of frequency band outputs includes fewer frequency band outputs than the plurality of frequency band outputs; and Determine the signal-to-noise ratio (SNR) of each frequency band output in the frequency band output subset, wherein each SNR of each frequency band output in the frequency band output subset is a model input feature among the plurality of model input features; and The presence of speech activity in the audio data is determined based on the output value generated by a hardware-based speech activity detection model based on the model's input features, wherein the hardware-based speech activity detection model includes a model that includes context-dependent model parameters associated with the determined current context of the audio data.

11. The method of claim 10, wherein the plurality of model input features include smoothed energy measurements and a set of smoothed spectral coefficients.

12. The method of claim 10, wherein the hardware-based feature generator comprises a hardware-implemented Fast Fourier Transform circuit.

13. The method of claim 10, wherein determining the signal-to-noise ratio of each frequency band output in the frequency band output subset comprises: Determine the noise floor of each frequency band output in the frequency band output subset; The logarithmic function is applied to the noise floor of each frequency band output in the subset of frequency band outputs; Determine the signal power level of each frequency band output in the frequency band output subset; as well as The logarithmic function is applied to the signal power level of each frequency band output in the subset of frequency band outputs. The signal-to-noise ratio of each frequency band output in the aforementioned frequency band output subset includes the logarithmic signal-to-noise ratio.

14. The method of claim 10, wherein preprocessing the received audio data comprises: The received audio data is split into a first audio data stream and a second audio data stream; The delay function is applied to the second audio data stream to generate a delayed second audio data stream; Apply the window function to the first audio data stream and the delayed second audio data stream; as well as The serial-to-parallel conversion is applied to the first audio data stream and the delayed second audio data stream.

15. The method of claim 10, wherein the hardware-based speech activity detection model comprises a hardware-implemented support vector machine (SVM) model.

16. The method of claim 15, wherein the hardware-implemented SVM model comprises: First multi-column SVM circuit; as well as A second single-column SVM circuit is configured to generate the output value.

17. The method of claim 15, further comprising: Multiple model parameters of the SVM model used for the hardware implementation are loaded into the memory of the electronic device.

18. The method of claim 10, wherein the subset of frequency band outputs comprises eight frequency band outputs.

19. The method of claim 10, wherein the audio source comprises one or more microphones of the electronic device.

20. A non-transient computer-readable medium comprising computer-executable instructions, which, when executed by one or more processors of a processing system, cause the processing system to perform a method for performing speech activity detection, the method comprising: Receive audio data from the audio source of the electronic device; A hardware-based feature generator is used to generate multiple model input features based on the received audio data, wherein generating the multiple model input features includes: The received audio data is preprocessed to generate domain-transformed input data; Multiple frequency band outputs are generated based on the domain transform input data using a domain transform circuit; The current context is determined based on spectral coefficients associated with the received audio data and spectral coefficients associated with a previous context in which the previous audio data was captured, wherein determining the current context includes: When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is within a threshold distance, it is determined that the current context has not changed relative to the previous context, or When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is outside the threshold distance, it is determined that the current context is different from the previous context; By filtering the symmetrical frequency band outputs among the plurality of frequency band outputs, a subset of frequency band outputs is selected from the plurality of frequency band outputs from the domain transform circuit, wherein the subset of frequency band outputs includes fewer frequency band outputs than the plurality of frequency band outputs; and Determine the signal-to-noise ratio (SNR) of each frequency band output in the frequency band output subset, wherein each SNR of each frequency band output in the frequency band output subset is a model input feature among the plurality of model input features; and The presence of speech activity in the audio data is determined based on the output value generated by a hardware-based speech activity detection model based on the model's input features, wherein the hardware-based speech activity detection model includes a model that includes context-dependent model parameters associated with the determined current context of the audio data.

21. The non-transient computer-readable medium of claim 20, wherein the hardware-based feature generator comprises a hardware-implemented Fast Fourier Transform circuit.

22. The non-transient computer-readable medium of claim 20, wherein determining the signal-to-noise ratio of each frequency band output in the subset of frequency band outputs comprises: Determine the noise floor of each frequency band output in the frequency band output subset; The logarithmic function is applied to the noise floor of each frequency band output in the subset of frequency band outputs; Determine the signal power level of each frequency band output in the frequency band output subset; as well as The logarithmic function is applied to the signal power level of each frequency band output in the subset of frequency band outputs. The signal-to-noise ratio of each frequency band output in the aforementioned frequency band output subset includes the logarithmic signal-to-noise ratio.

23. The non-transient computer-readable medium of claim 20, wherein preprocessing the received audio data includes: The received audio data is split into a first audio data stream and a second audio data stream; The delay function is applied to the second audio data stream to generate a delayed second audio data stream; Apply the window function to the first audio data stream and the delayed second audio data stream; as well as The serial-to-parallel conversion is applied to the first audio data stream and the delayed second audio data stream.

24. The non-transient computer-readable medium of claim 20, wherein the hardware-based speech activity detection model comprises a hardware-implemented support vector machine (SVM) model.

25. The non-transient computer-readable medium of claim 24, wherein the hardware-implemented SVM model comprises: First multi-column SVM circuit; as well as A second single-column SVM circuit is configured to generate the output value.

26. The non-transient computer-readable medium of claim 24, wherein the method further comprises: Multiple model parameters of the SVM model used for the hardware implementation are loaded into the memory of the electronic device.

27. The non-transient computer-readable medium of claim 20, wherein the subset of frequency band outputs comprises eight frequency band outputs.

28. A processing apparatus, comprising: A means for receiving audio data from an audio source of an electronic device; A means for generating multiple model input features based on received audio data using a hardware-based feature generator, wherein the means for generating the multiple model input features includes: A means for preprocessing received audio data to generate domain-transformed input data; A device for generating multiple frequency band outputs based on the domain transform input data using a domain transform circuit; A means for determining a current context based on spectral coefficients associated with received audio data and spectral coefficients associated with a previous context of the audio environment in which previous audio data was captured, wherein the means for determining the current context is configured to: When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is within a threshold distance, it is determined that the current context has not changed relative to the previous context, or When the distance between the spectral coefficients associated with the received audio data and the spectral coefficients associated with the previous context is outside the threshold distance, it is determined that the current context is different from the previous context; A means for selecting a subset of frequency band outputs from a plurality of frequency band outputs from a domain transform circuit by filtering a symmetrical frequency band output among the plurality of frequency band outputs, wherein the subset of frequency band outputs includes fewer frequency band outputs than the plurality of frequency band outputs; and A means for determining the signal-to-noise ratio of each frequency band output in the frequency band output subset, wherein each signal-to-noise ratio of each frequency band output in the frequency band output subset is a model input feature among the plurality of model input features; A means for providing the plurality of model input features to a hardware-based speech activity detection model, wherein the hardware-based speech activity detection model includes a model that includes context-dependent model parameters associated with a determined current context related to the audio data. A means for receiving output values ​​from the hardware-based speech activity detection model; and A means for determining the presence of speech activity in the audio data based on the output value.

Images