Robust multi-channel voice activity detection for speech and audio processing in communication devices

Abstract

This disclosure describes devices and methods for robust multi-channel voice activity detection for speech and audio processing in communication devices. A hearable device (300) processes voice activity detection using multiple inputs. The inputs can be sensor (210) inputs, feedback cancelation signals (904), or a combination of both. The inputs are processed through a machine-learned model (208), which generates an output (706). The output (706) is used to detect a voice (406, 410). Voice (406, 410) detection can cause the hearable device (300) to perform a number of actions.

Description

ROBUST MULTI-CHANNEL VOICE ACTIVITY DETECTION FOR SPEECH AND AUDIOPROCESSING IN COMMUNICATION DEVICESBACKGROUND

[0001] Modem communication devices often require voice activity detection (“voice detection”) with a high degree of accuracy to perform various tasks. For example, a user wearing noise-canceling true-wireless earbuds, which are streaming audio to the user, may wish to begin a conversation and, as a result, wish for the true- wireless earbuds to attenuate the audio playback, cancel active noise cancelation (ANC), activate ambient sound mode, or any combination of these actions. To accomplish this, the true-wireless earbuds may be configured to use voice detection to detect that the user’s voice is present and subsequently perform one or more of the actions or a different action or take no action.

[0002] In such a scenario, the voice detection accuracy may be critical. An overly conservative voice detection may not detect the user’s voice when the user wishes it to, such as during a phone conversation, during in-person conversation, or when the user may be reciting a hot-word or device command. By contrast, an overly aggressive voice detection may detect one or more false positives where the user is not actively vocalizing. Either case may result in an undesirable user experience.

[0003] The current invention is aimed at reducing the number of false positives while still providing a high voice-detection fidelity in detecting the user’s voice. By doing so, the user experience with voice detection may be significantly enhanced.SUMM RY

[0004] This disclosure describes devices and methods for robust multi-channel voice activity detection for speech and audio processing in communication devices, allowing for enhancing voice detection fidelity without experiencing an increase in false positives. In some aspects, an electronic device is described (e.g., a pair of true-wireless earbuds) comprising one or more first sensors, a memory configured to store computer-executable instructions, and one or more processors in communication with the memory and the one or more first sensors. The one or more first sensors may be configured to detect audio external to a user and to output, based on the detected audio, an electronic audio signal. The one or more processors may be configured to receive the electronic audio signal, generate a feedback cancelation signal, generate, based on the electronic audio signal, an audio data, generate, using a machine-learned model, an output based on the feedback cancelation signal and the audio data, and detect, based on the output, a voice in the electronic audio signal.

[0005] The one or more processors may be further configured to perform an action respondent to detecting the voice. The action may comprise one or more of a hot-word detection, a command, an activation or cancelation of an active noise cancelation, and an automatic control of an audio output. The one or more processors may be further configured to receive a transmission signal, and the feedback cancelation signal may be based at least in part on the transmission signal. Additionally or alternately, the feedback cancelation signal may be based at least in part on the electronic audio signal.

[0006] The electronic device may further comprise one or more second sensors configured to output a second signal. The one or more processors may be further configured to receive the second signal and to further base the audio data on the second signal. The machine-learned model may be trained by a machine-learning algorithm, wherein the machine-learning algorithm may comprise at least one of a convolutional neural-network (CNN) and a long short-term memory' (LSTM) algorithm. The one or more processors may be further configured to, respondent to detecting the voice, determine whether there is more than one voice was detected.

[0007] In some aspects, an electronic device is described (e.g., a pair of true-wireless earbuds) comprising one or more first sensors, one or more second sensors, a memory configured to store computer-executable instructions, and one or more processors in communication with the memory, the one or more first sensors, and the one or more second sensors. The one or more first sensors may be configured to detect audio external to a user and to output, based on the detected audio, an electronic audio signal. The one or more second sensors may be configured to detect a motion of the user’s body and to output, based on the motion, a user body signal. The one or more processors may be configured to receive the electronic audio signal, receive the user body signal, generate, based on the electronic audio signal and the user body signal, an audio data, generate, using a machine-learned model, an output based on the audio data, and detect, based on the output, a voice in the electronic audio signal. The one or more first sensors may comprise a microphone and the one or more second sensors may comprise an inertial measurement unit (IMU) sensor.|0008| Additionally, the one or more processors may be further configured to receive a transmission signal. The one or more processors may, additionally or alternately, be further configured to generate a feedback cancelation signal. The feedback cancelation signal may be based on one or more of the transmission signal, the electronic audio signal, and the user body signal. The output may be further based on the feedback cancelation signal.

[0009] This document also describes a method for detecting a voice by an electronic device. The method may comprise receiving, by the electronic device, an electronic audio signal generated based on audio external to a user, generating, by the electronic device, a feedback cancelation signal, generating, by the electronic device and based on the electronic audio signal,an audio data, generating, by the electronic device and using a machine-learned model, an output based on the feedback cancelation signal and the audio data, and detecting, by the electronic device and based on the output, the voice in the electronic audio signal.

[0010] The method may further comprise receiving, by the electronic device, a second signal generated based on movement of a body of the user, and the audio data may further be based on the second signal. The method may further comprise performing an action respondent to detecting the voice. The action may comprise one or more of a hot-word detection, a command, an activation or cancelation of an active noise cancelation, and an automatic control of an audio output.

[0011] Additionally, the feedback cancelation signal may be generated based at least in part on the electronic audio signal. The method may further comprise receiving a transmission signal, and the feedback cancelation signal may be based in whole or in part on the transmission signal. Additionally, the machine-learned model may be trained by a machine-learning algorithm. The machine-learning algorithm may be comprised of at least one of a CNN and an LSTM algorithm.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The details of one or more aspects of robust multi-channel voice activity detection for speech and audio processing in communication devices are described in this document with reference to the following drawings. The same numbers are used throughout the drawings to reference like features and components:FIG. 1 illustrates an example environment in which techniques for robust multi-channel voice activity detection for speech and audio processing in communication devices can be implemented.FIG. 2 illustrates an example of the user equipment set forth in FIG. 1.FIG. 3 illustrates further details of the user equipment set forth in FIG. 2.FIG. 4 illustrates an example of a voice detection technique employed by the user equipment set forth in FIG. 3.FIG. 5 illustrates another example of a voice detection technique employed by the user equipment set forth in FIG. 3FIG. 6 illustrates a spectrogram, as outlined in this disclosure.FIG. 7 illustrates concepts of a machine-learned (ML) model.FIG. 8 illustrates a specific instance of an ML model using a convolutional neural network for training.FIG. 9 illustrates an example method for robust multi-channel voice activity detection for speech and audio processing in communication devices.FIG. 10 illustrates another example method for robust multi-channel voice activity detection for speech and audio processing in communication devices.FIG. 11 illustrates another example method for robust multi-channel voice activity' detection for speech and audio processing in communication devices.DETAILED DESCRIPTION

[0013] This disclosure describes devices and methods for robust multi-channel voice activity detection for speech and audio processing in communication devices. Voice activity detection (“voice detection”) may be used in a variety of applications, including but not limited to issuing commands to an electronic device, activating a transparency mode on a hearable device, such as true-wireless earbuds, controlling active noise cancelation (ANC) on a hearable device, and other applications. Modem hearables in particular may leverage voice detection to enhance a user experience and provide a wearer with greater functionality.

[0014] The utility of voice detection may be undermined, however, by a voice detection that is either too conservative or too aggressive. An overly conservative voice detection may miss voice activity, for instance classifying a user’s voice command as part of a background noise. This may result in the user’s command not registering with a device performing the voice detection function, thus not executing the user’s command. Additionally or alternately, the user may start a conversation with another person while concurrently, by way of example, listening to music on a pair of true-wireless earbuds. The goal of voice detection in this scenario may be to attenuate or mute the audio volume, cancel ANC, and activate the true-wireless earbuds’ external microphones in order to activate a transparency mode, the transparency mode allowing for external audio captured by the external microphones to be sent to the user in order to mimic how the user might hear external audio while not wearing the true-wireless earbuds. An overly conservative voice detection on the true- wireless earbuds may not register the user’s voice, thus not performing the actions aforementioned and hindering the user’s ability to converse.100151 An overly aggressive voice detection may likewise provide a poor user experience. For instance, an overly aggressive voice detection on a pair of smart glasses may classify a portion of background noise as the user’s voice while the user is walking and listening to a podcast on speakers of the smart glasses. The smart glasses may perform one or more actions in response to errantly classifying the background noise as the user’s voice, such as canceling ANC, activating a transparency mode, and attenuating or muting audio from the podcast. This may result in a negative listening experience for the user.

[0016] To address the challenge of voice detection fidelity, in particular for existing hearables, techniques are described herein that implement a multi-channel voice detection thatmay leverage a machine-learned (ML) model in order to accurately classify the user’s voice. Existing voice detection algorithms may rely solely on an external microphone of a hearable device to provide an audio signal that may contain the user’s voice. By adding additional signals from additional sensors or sources and providing these multiple inputs to the ML model for classification, it may be possible to avoid overly conservative or overly aggressive voice detection.

[0017] FIG. 1 is an illustration of an example environment 100 in which multi-channel voice detection may be employed. In the example environment 100, a person 102 is wearing a hearable device 104. The hearable device 104 is here shown as a pair of true- wireless earbuds, but this is shown for example purposes and should not be taken as a limiting case. The hearable device 104 may be smart glasses, hearing aids, a smart helmet, over-the-ear headphones, or any number of other hearables known to a practitioner skilled in the art. The person 102 may be listening to audio on the hearable device 104. The hearable device 104 may be performing other actions, such as ANC.

[0018] The hearable device 104 may have several sensors, such as an external microphone 106 and an inertial measurement unit (IMU) sensor 108. The IMU sensor 108 may be comprised of multiple accelerometers, which may be able to detect a change in angle relative to a known, previous, or arbitrary axis of the hearable device 104. The person 102 may begin speaking, producing a voice 110. The person 102 may be conversing with a person 112. The person 112 may produce a voice 114. There may also be background noise 116 present in the environment 100. The external microphone 106 may sense the audio sources 110, 114, and 116. The IMU sensor 108 may sense movement of the person 102 when the person 102 produces the voice 110, such as based on movement of a jaw of the person 102 or another body movement of the person 102 related to the production of the voice 110.

[0019] The sensors 106 and 108 may produce signals based on inputs they receive. For example, the external microphone 106 may produce an audio signal (not pictured) based on the voice 110, voice 114, and background noise 116. The IMU sensori 08 may produce a body signal based on the movement of the person 102 related to the production of the voice 110. Other sensors not pictured may be present in the hearable device 104. These other sensors may include, by way of example only, accelerometers, infrared sensors, and additional microphones and IMU sensors.

[0020] FIG. 2 illustrates an example user device 200. The user device 200 is illustrated with various non-limiting example devices, including irue-wireless earbuds 200-2, headphones 200-4, smart glasses 200-6, a VR headset 200-8, a smart helmet 200-10, and hearing aids 200-12. Other devices may also be used.

[0021] The user device 200 includes one or more computer processors 202 and at least one computer-readable memory 204. The memory 204 may include instructions 206, an MLmodel 208, or other components not shown. The user device 200 further includes one or more sensors 210. The sensors 210 may comprise any one or more of one or more microphones 212, one or more accelerometers 214, one or more IMU sensors 216, and additional sensors 218. The user device 200 may also include a transceiver module 220 for wireless communications with other electronic devices (not pictured), where the transceiver module 220 may comprise at least one of one or more antennas 222 and radios 224. For example, the transceiver module 220 may communicate data over a local-area network (LAN), a wireless local-area network (WLAN), a personal-area network (PAN), a wide-area network (WAN), an intranet, the Internet, a peer-to- peer network, a point-to-point network, a mesh network, Bluetooth™, and the like.

[0022] Additional components not pictured in FIG. 2 may also be included in the user device 200, such as speakers, infrared sensors, capacitive input sensors, and other features known to a person of ordinary skill in the art. The features outlined in FIG. 2 for the user device 200 are meant to be illustrative and not limiting.

[0023] FIG. 3 illustrates an example hearable device 300 and components of it, the hearable device 300 capable of implementing voice detection as outlined in this disclosure. Although the hearable device 300 is shown as one of a pair of in-ear earbuds of the true-wireless variety, this is meant to be illustrative and not limiting. The hearable device 300 may be, for example, any one of the example devices illustrated in FIG. 2 or another device known to a person of ordinary skill in the art. The hearable device 300 may have one or more microphones, such as microphones 302, 304, and 306. The microphones may be external, such as 302 and 304, or internal, such as 306. An external microphone 302, 304 may be defined as a microphone that is not configured to be on an interior portion of a user body. An internal microphone 306 may be defined as a microphone that is configured to be on an interior portion of a user body. An example of an internal microphone 306 may be an in-ear microphone, such as a microphone found in an earbud, which is configured to reside in the ear canal of the user when the earbud is worn as designed. The hearable device 300 may include additional components, such as an IMU sensor 308, accelerometers 310, a capacitive touch sensor 312, and a speaker 314. Positions of each of the components may be different than illustrated, depending on the configuration desired. Additional components may also be included, as well as fewer components than those pictured.

[0024] The microphones 302, 304, and 306 may be configured to create one or more audio signals based on audio sensed by the microphones 302, 304, and 306. The IMU sensor 308 may be configured to create a body signal based on movement sensed. The audio signals and the body signal may be used as inputs for an ML model 208. An output of the ML model 208 may be a prediction of whether a user voice 110 is detected in the input signals. Additionally or alternately, the ML model 208 may take as an input a feedback cancelation signal, which will be detailed in alater section of this disclosure. Other sensors of the hearable device 300 may also be used to provide the ML model 208 inputs. For example, the accelerometers 310 may produce an accelerometer signal based on a user movement. As another example, an IR sensor may produce an IR signal based on IR sensed from a user.

[0025] Other input types may be envisioned by a person of ordinary skill in the art. Some or all of the sensor outputs may be used as inputs for the ML model 208, creating a multi-channel input. This multi-channel input may be more sensitive to the user voice 110 than simply using input from one or more microphones (such as 302 and 304), which may result in a more granular voice detection with fewer false positives and fewer missed detections.EXAMPLE DEVICES

[0026] FIG. 4 illustrates an example of a hearable device such as that exemplified in FIG. 3, which implements a voice detection as outlined in this disclosure. An environment 400 may have a person 402, who may be wearing a hearable device 404. The hearable device 404 may be materially similar to the hearable device 300 in FIG. 3. The person 402 may be vocalizing a voice 406. There may be a second person 408 in the environment 400. The person 408 may be vocalizing a voice 410. The environment 400 may also have background noise 412 stemming from one or more sources in the environment 400.

[0027] The hearable device 404 may have one or more of sound waves from 406, 410, and 412 incident upon it. The hearable device 404 may have input sensors, such as microphones 302, 304, and 306. These microphones 302, 304, and 306 may produce audio signals based on sound from 406, 410, 412, and possibly other sources not pictured. The audio signals may contain the voice 406. The hearable device 404 may also create one or more body signals based on movement of a jaw or other body part of the person 402, the movement performed in order to produce the voice 406. The hearable device 404 may additionally create a feedback cancelation signal. The feedback cancelation signal may be, by way of example, part of an ANC technique, which may be used to cancel or otherwise attenuate sound from external sources, such as 410 and 412, from being perceived by the person 402. The feedback cancelation signal may be based on one or more of the audio signals.

[0028] The hearable device 404 may use any one or more of the audio signals, the body signal, and the feedback cancelation signal as inputs to an ML model 208. The ML model 208 may be configured such that it generates an output, the output comprising a prediction of whether the voice 406 is present in the inputs. This prediction may constitute a voice detection. When using more than one input, the voice detection may represent a multi-channel voice detection. Byusing more than a single source for input into the ML model 208, the voice detection may be more accurate than single-source voice detection techniques.

[0029] In one embodiment, the hearable device 404 may use only the audio signals and the body signal as inputs. In this case, the audio signals may be based on the environmental or background noise 412, the voice 406, the voice 410, or any combination of one or more of these or other audio sources The body signals may be based on input from sensors such as an IMU sensor 308, accelerometers 310, or additional sensors that may be configured to sense changes in relative angle, direction, velocity, acceleration, or any other type of movement. There may be multiple sensors of the same kind, such as multiple IMU sensors. In this embodiment, the one or more audio signals and the one or more body signals are used as inputs for the ML model 208. The ML model 208 provides an output, which may be used to determine if the voice 406 was detected. The ML model 208 may also provide, as part of the output, whether more than one voice was detected, such as the voice 410 or additional voices.

[0030] Upon detecting one or more voices, the hearable device 404 may take one or more actions. Actions may be one or more of a hot-word detection, a command, an activation or cancelation of an active noise cancelation, and an automatic control of an audio output. By way of example, suppose the person 402 is issuing a command for the hearable device 404 to pause playback of audio. Upon determining that the voice 406 was present, the hearable device 404 may determine that a command was spoken and that the command was to pause the audio playback. The hearable device 404 may then pause the playback of audio.

[0031] The action taken may also be an automatic action, which may not be related to a direct command from the person 402. For example, the hearable device 404 may be configured such that, respondent to voice detection determining that the voice 406 is present, it may automatically switch from an ANC mode to a transparency mode. A transparency mode may be a mode such that the hearable device 404 cancels any ANC and outputs audio to a speaker 314 based on sounds sensed by external microphones 302 and 304. This may allow the person 402 to experience sound similar to what they might experience without wearing the hearable device 404. This may be desired if the person 402 is intent upon, for example, having a conversation.

[0032] The action taken may also be dependent on the number of voices detected. For example, detection of the voice 406 and another voice, such as the voice 410, may cause the hearable device 404 to perform one or more of attenuating or muting audio playback, pausing audio playback, canceling ANC, and putting the hearable device 404 into a transparency mode. By contrast, the hearable device 404 may be configured to not automatically take one or more of these actions should only the voice 406 be detected by the voice detection. By way of example, the hearable device 404 may take actions respondent to detection of the voice 406 such as one ormore of transcription, hot-word detection, command execution, and using content of the voice 406 as an input. Some, all, or none of these techniques may be used in the hearable device 404. The aforementioned actions are meant to be illustrative and should not be seen as limiting. A person of ordinary skill in the art may readily envision several activities, which may be listed or not listed in this disclosure, that may be beneficial to a user should there be one, multiple, or no voices found by the voice detection.

[0033] FIG. 5 illustrates an example of a hearable device such as that exemplified in FIG. 3, which implements a voice detection as outlined in this disclosure. An environment 500 may have a person 502, who may be wearing a hearable device 504. The hearable device 504 may be materially similar to the hearable device 300 in FIG. 3. The person 502 may be vocalizing a voice 506. There may be a second person 508 who may be speaking with the person 502 remotely, such as in a telephone conversation. The person 508 may be vocalizing a voice 510, which may be transmitted to the hearable device 504 over a network (not pictured). The environment 500 may also have background noise 512 stemming from one or more sources in the environment 500.

[0034] The hearable device 504 may have one or more of sound waves from 506 and 512 incident upon it. The hearable device 504 may have input sensors, such as microphones 302, 304, and 306. These microphones 302, 304, and 306 may produce audio signals based on the sound from 506 and 512 and possibly other sources not pictured. The audio signals may contain the voice 506. The hearable device 504 may also create a body signal based on movement of a jaw or other body part of the person 502, the movement done in order to produce the voice 506. The hearable device 504 may additionally create a feedback cancelation signal. The feedback cancelation signal may be, by way of example, part of an ANC technique to cancel sound from external sources, such as 512, from being perceived by the person 502. The feedback cancelation signal may be based on one or more of the audio signals. Additionally or alternately, the feedback cancelation signal may be based on a transmitted and / or received audio signal from the voice 510. For example, the feedback cancelation signal may be part of an echo cancelation technique employed by the hearable device 504 in order to give a better call experience to the person 502.

[0035] The hearable device 504 may use any one or more of the audio signals, the body signal, and the feedback cancelation signal as inputs to an ML model 208. The ML model 208 may be configured such that it generates an output, the output comprising a prediction of whether the voice 506 is present in the inputs. This prediction may constitute a voice detection. When using more than one input, the voice detection may represent a multi-channel voice detection. By using more than a single source for input into the ML model 208, the voice detection may be more accurate than single-source voice detection techniques.

[0036] The hearable device 504 may take one or more actions respondent to detecting the voice 506. These actions may be materially similar to those outlined for the hearable device 404 in FIG. 4.SPECTROGRAM

[0037] Signals used as inputs for the ML model 208 may undergo processing prior to being sent to the ML model 208. One example of such processing may be the use of a spectrogram. A spectrogram may be a representation of audio or other frequency-based signals with respect to a second coordinate, which may be time. For example, a spectrogram may be a mapping of frequency of inputs vs time, with the mapping representing relative intensity of the frequencies at the times.

[0038] FIG. 6 illustrates a spectrogram 600. An x-axis 602 may be time and ay-axis 604 may be a frequency of inputs over the time. A density 606 of the plot represents the intensity of the frequencies at the times. In this way, FIG. 6 is a three-dimensional spectrogram. Spectrograms of other dimensionality may also be made, such as two-dimensional, four-dimensional, or any other dimensionality. The spectrogram 600 may be represented as a spectrogram data, for example by use of a matrix, vector, tensor, or other mathematical object of the same dimensionality as the spectrogram 600. Spectrogram data may be used as an input for an ML model 208.

[0039] The spectrogram 600 has features, which may represent a physical phenomenon present in captured audio or other frequency-based signals. For example, a feature 608 may represent a voice such as 110, 114, 406, 410, 506, or 510. A feature 610 may represent a background noise; by way of example, the feature 610 may represent moving traffic in an environment, such as the environment 400. Many additional features may be present, or more or fewer features than what has been outlined.

[0040] Feature generation in the spectrogram 600 may be done by using signals from sensors 210. For example, one or more external microphones 212 may sense audio from a voice, such as a voice 406, and other audio, such as background noise 412. One or more IMU sensors 216 may sense motion from a jaw used to create the voice 406. The sensors 212 and 216 may output signals based on inputs sensed. These signals may represent frequencies and intensities of the frequencies over a time period.

[0041] By way of example only, the one or more microphones 212 may have their output represented on the spectrogram 600. The portion of their output corresponding to the voice 406 may be the feature 608 in the frequency range of 10.0-16.0 kHz. The one or more IMU sensors 216 may have their output also represented on the spectrogram 600. The portion of their outputcorresponding to the voice 406 may be the feature 608 in the frequency range of 6.8-10.0 kHz. Other sensors may also have input corresponding to the feature 608. Additional features of the spectrogram 600, such as the feature 610, may be due to audio captured that is not from the voice 406. In this way, multiple signals may be used to identify a voice, such as the voice 406, distinct from other captured audio.

[0042] Additional signals may be present in a spectrogram feature, such as the feature 608. For example, a feedback cancelation signal may have a frequency component and a time component, which may be incorporated into the spectrogram 600 and contribute to features useful for classification, such as voice detection. The feedback cancelation signal may be based on environmental noise, conversation, audio not captured by a hearable device 300, such as an echo cancelation line used in applications such as phone conversations, or any combination of these or another source known to a person of ordinary skill in the art.

[0043] An echo cancelation line may be an audio data, which may include audio from both a transmitted audio signal and from a microphone on the user device. By way of example, an echo cancelation line may be used in a phone call where the user performing the phone call may be wearing earbuds with one or more internal and / or external microphones. A microphone audio signal may be generated by the one or more microphones on the earbuds and may include a voice of the user. The transmitted audio signal may be the audio from an other person on the phone call. There may be an echo effect if the voice of the user is transmitted through a speaker of the earbud. Cancelation of the echo effect may include processing the microphone audio signal and the transmitted audio signal such that the user does not hear the voice of the user in the earbud speaker. Other techniques and methods to cancel the echo effect may additionally or alternately employed, including alternate configurations of an echo cancelation line, by a person of ordinary skill in the art.MACHINE LEARNING|0044| An ML model, as discussed in this disclosure, refers to a computer model that has been trained using one or more machine-learning techniques. In general, this training may be done by providing training inputs to one or more training models, which in turn may provide an output. The output may be in the form of a prediction, a confidence score, or other probabilitybased metrics.

[0045] FIG. 7 illustrates a generic ML model training setup. An ML model generator 700 may comprise training elements in the form of inputs 702, training models 704, and outputs 706 and may be used to generate an ML model 708. The inputs 702 may be training data. The training data 702 may be processed by the training models 704. Examples of the training models 704include multi-layer perceptron (MLP) models, convolutional neural networks (CNN), long shortterm memory' (LSTM) algorithms, generative adversarial networks (GAN), K-means clustering, Gaussian mixture models (GMM), or any other of a number of machine-learning training techniques known to a person of ordinary skill in the art. The training models 704 may comprise a single type of model or multiple types, as well as various combinations of types including combinations of single types and of multiple types.

[0046] The training models 704 may use supervised learning methods, where the training data may be labeled and the outputs 706 may be graded based on their fidelity to a “truth” output. The training models 704 may use an unsupervised learning method, where there may not be labels on the training data and the training models 704 may classify correlations without reference to a “truth” value. The training models 704 may combine supervised and unsupervised techniques.

[0047] The input 702 may be data from one or more sensors 210. For example, the input 702 may be audio data from one or more microphones 212, body data from one or more IMU sensors 216, movement data from one or more accelerometers 214, other data, or any combination of these data. Additionally or alternately, the input 702 may be spectrogram data, such as spectrogram data generated from a spectrogram 600. The spectrogram 600 may be constructed using data from the sensors 210.

[0048] The input 702 may also be from training data used to generate the ML model 708. The ML model 708 may be generated once the training of the ML model generator 700 is complete. The ML model 708 may be trained on a same device where the ML model 708 is stored or on at least one other device. The ML model 708 may be updated with additional training after it has been initially trained.

[0049] FIG. 8 illustrates an example of how the ML model 708 may be trained in the context of this disclosure. FIG. 8 is meant not to be limiting but to show one possible example out of many, which a person skilled in the art may be able to employ. A trainer 800 may have inputs 802. The inputs 802 may be spectrogram data, audio data from audio including a voice, external noise, conversation, other audio sources, feedback cancelation signals, or any combination of these elements or other elements. In this example, the inputs 802 are first processed by a CNN 804. The CNN 804 may extract features from the inputs 802. The CNN 804 may be a disconnected neural network, meaning all members of the neural network, such as perceptrons, may not be in a causal relationship with one another. It should be mentioned that other feature extraction techniques besides CNNs may be equally employed in lieu of the CNN 804. An MLP neural network 806 may receive the extracted feature outputs from the CNN 804 as inputs. The MLP 806 may be a fully connected network, which may comprise all perceptrons in a causal relationship with one another. The MLP 806 may have as an output a detectionprobability prediction 808. The prediction 808 may be, for example, the probability that a user’s voice was detected in the inputs 802, the probability that multiple voices were detected in the inputs 802, a binary classification or other dimensional classification for voice detection, or some other prediction related to voice detection in the inputs 802.

[0050] An ML model trained by a trainer such as the ML trainer 800 may exhibit similar architecture. For example, suppose the ML trainer 800 uses a matrix containing spectrogram data as the inputs 802 and provides a prediction of voice detection as a number between 0 and 1 as the prediction 808. An ML model trained by the ML trainer 800 may take a matrix of spectrogram data as its input and output a probability, between 0 and 1, of a voice having been detected. Such an ML model may have one or more feature extraction layers, such as the CNN 804, and one or more fully connected layers, such as the MLP 806.

[0051] It will be understood by a person of ordinary skill in the art that the aforementioned are merely examples of the types of machine-learning training that may be done to produce an ML model for voice detection. Many other combinations and techniques may be employed, which do not alter the scope of the inventive concept of multi-channel voice detection using an ML model. Examples such as employing a CNN, use of spectrogram data, having a fully connected portion of the machine-learning training, etc. are not meant to limit the scope of the invention, but rather to serve as examples of how an ML model for voice detection using multi-channel signals may be accomplished.

[0052] By way of example, training for the ML model may equivalently be accomplished by incorporating, as either a part of or the complete machine-learning trainer 800, an LSTM. The LSTM algorithm is a type of recurrent neural network (RNN), which may process data in a time- indexed fashion. The LSTM may solve a so-called “vanishing gradient problem,” in which gradients used in fitting may tend to zero and, thus, may not yield useful fit parameters for a given model (e.g., weights and biases). During the training phase, the LSTM may allow for persistent gradients used to fit when the gradients may otherwise go to zero (e.g., in a traditional RNN). For the purpose of the present disclosure, an example of implementing the LSTM in the machinelearning trainer 800 may be to replace the MLP 806 with the LSTM or to include the LSTM before or after the MLP 806. A person of ordinary skill in the art will understand that other configurations using the LSTM may also be equivalently used, and that the example given here is meant to be illustrative and not limiting.EXAMPLE METHODS

[0053] Example methods 900-1100 are described with reference to FIGs. 9 to 11 in accordance with one or more aspects for robust multi-channel voice activity detection for speechand audio processing in communication devices. The order in which the method blocks are described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement a method or an alternate method. Generally, any of the components, modules, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Some operations of the example methods may be described in the general context of executable instructions stored on computer-readable storage memory that is local and / or remote to a computer processing system, and implementations can include software applications, programs, functions, and the tike. Alternatively or in addition, any of the functionalities described herein can be performed, at least in part, by one or more hardware logic components, such as, and without limitation, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SoCs), complex programmable logic devices (CPLDs), and the like.

[0054] FIG. 9 illustrates an example method 900 of multi-channel voice detection using an ML model. Operation 902 comprises receiving, by an electronic device, an electronic audio signal based on audio external to a user. This audio external to the user may comprise a voice of the user, background noise, a voice or voices not of the user, other audio, or a combination of these sources. The received audio signal of operation 902 may be a combination of signals from various sensors of varying types.

[0055] Operation 904 comprises generating, by the electronic device and based on the electronic audio signal, a feedback cancelation signal. The feedback cancelation signal may be further based on echo cancelation for a phone call or any other type of input or signal used for feedback cancelation known to a person of ordinary skill in the art.

[0056] Operation 906 comprises generating, by the electronic device and based on the electronic audio signal, an audio data. The audio data may be, for example, a spectrogram, part of a spectrogram, or a data representation of a spectrogram or part of a spectrogram. The audio data may be a machine-language representation of the audio signal. The audio data may further be based on the feedback cancelation signal.

[0057] Operation 908 comprises generating, by the electronic device and using a machine- learned model, an output based on the feedback cancelation signal and the audio data. The feedback cancelation signal and the audio data may be used as inputs for the ML model. The feedback cancelation signal and the audio data may be pre-processed prior to being used as inputs for the ML model, such as using the audio data and the feedback cancelation signal to generate a spectrogram, where the spectrogram may be represented by spectrogram data and the spectrogram data may be used as the input for the ML model. The output may be a prediction of whether avoice was found in the audio signal, the feedback cancelation signal, the audio data, or any combination of these.

[0058] Operation 910 comprises detecting, by the electronic device and based on the output, a voice in the electronic audio signal. The detection may be, for example, based on the output exceeding a threshold value. It may be based on the output having a binary classification or other dimensional classification of different types of audio events, where at least one of the types may be a voice event. Detecting a voice may also comprise determining whether multiple voices have been detected.

[0059] FIG. 10 illustrates another example method 1000 of multi-channel voice detection using an ML model. Operation 1002 comprises receiving, by an electronic device, an electronic audio signal based on audio external to a user. This audio external to the user may comprise a voice of the user, background noise, a voice or voices not of the user, other audio, or a combination of these or other audio sources. The received audio signal of the operation 1002 may be a combination of signals from various sensors, which may be of varying types.

[0060] Operation 1004 comprises receiving, by the electronic device, a user body signal based on a motion of a body of the user. By way of example, the motion of the body of the user may be a movement of the user’s jaw in order to vocalize a voice. The user body signal may be based on input from a motion type sensor, such as, but not limited to, an IMU sensor, an accelerometer, a gyroscope, or any other sensor configured to detect user body motion.

[0061] Operation 1006 comprises generating, by the electronic device and based on the electronic audio signal and the user body signal, an audio data. The audio data may be, for example, a spectrogram, part of a spectrogram, or a data representation of a spectrogram or part of a spectrogram. The audio data may be a machine-language representation of the audio signal and the user body signal. Both the audio signal and the user body signal may contain frequency information, which may allow both signals to be represented by the same audio data. The frequencies in such a case may be in different ranges of frequency for the audio signal and the user body signal, in a same range of frequency, or in different but overlapping ranges of frequency.

[0062] Operation 1008 comprises generating, by the electronic device and using an ML model, an output based on the audio data. The audio data may be used as inputs for the ML model. The audio data may be pre-processed prior to being used as inputs for the ML model, such as using the audio signal and the user body signal to generate a spectrogram, where the spectrogram may be represented by spectrogram data and the spectrogram data may be used as the input for the ML model. The output may be a prediction of whether a voice was found in the audio signal, the user body signal, or both.

[0063] Operation 1010 comprises detecting, by the electronic device and based on the output, whether a voice is in the audio data. The detection may be, for example, based on the output exceeding a threshold value. It may be based on the output having a binary classification or other dimensional classification of different types of audio events, where at least one of the types may be a voice event. Detecting a voice may also comprise determining whether multiple voices have been detected.

[0064] FIG. 11 illustrates another example method 1100 of multi-channel voice detection using an ML model. Operation 1102 comprises receiving, by an electronic device, an electronic audio signal based on audio external to a user. This audio external to the user may comprise a voice of the user, background noise, a voice or voices not of the user, other audio, or a combination of these sources. The received audio signal of operation 1102 may be a combination of signals from various sensors of varying types.

[0065] Operation 1104 comprises receiving, by the electronic device, a user body signal based on a motion of a body of the user. By way of example, the motion of the body of the user may be movement of the user’s jaw in order to vocalize a voice. The user body signal may be based on input from a motion type sensor, such as, but not limited to, an IMU sensor, an accelerometer, a gyroscope, or any other sensor configured to detect user body motion.

[0066] Operation 1106 comprises receiving, by the electronic device, a transmission signal. The transmission signal may be, for example, an audio signal received by the electronic device while the electronic device may be performing a voice call. The transmission signal may be received over a wireless communication network.

[0067] Operation 1108 comprises generating, by the electronic device, a feedback cancelation signal. The feedback cancelation signal may be, for example, based on the audio signal in order to process ANC, based on the transmission signal in order to process echo cancelation, based on the user body signal, or based on a combination of these signals or on other signals for purposes known to a person of ordinary skill in the art.|0068| Operation 1110 comprises generating, by the electronic device and based on one or more of the electronic audio signal, the user body signal, and the feedback cancelation signal, an audio data. For example, the audio data may be in the form of spectrogram data generated from a spectrogram including frequency and time correspondence information from the audio signal, the user body signal, the feedback cancelation signal, or any combination of these or other signals.

[0069] Operation 1112 comprises generating, by the electronic device and using an ML model, an output based on the audio data. The audio data may be used as inputs for the ML model. The audio data may be pre-processed prior to being used as inputs for the ML model, such asusing one or more of the audio signal, the user body signal, and the feedback cancelation signal to generate a spectrogram, where the spectrogram may be represented by spectrogram data and the spectrogram data may be used as the input for the ML model. The output may be a prediction of whether a voice was found in one or more of the audio signal, the user body signal, and the feedback cancelation signal.

[0070] Operation 1114 comprises detecting, by the electronic device and based on the output, a voice in the electronic audio signal. The detection may be, for example, based on the output exceeding a threshold value. It may be based on the output having a binary classification or other dimensional classification of different types of audio events, where at least one of the types may be a voice event. Detecting the voice may also comprise determining whether multiple voices have been detected.

[0071] Operation 1116 comprises performing, respondent to the voice being detected in the audio data, an action. The action may comprise one or more of a hot-word detection, a command, an activation or cancelation of an active noise cancelation, and an automatic control of an audio output. By way of example, a user may vocalize a command to pause an audio output on the electronic device. The voice detection outlined in FIG. 11 may detect a voice, and, respondent to detecting the voice, the electronic device may process the command and pause the audio output. The action may be to automatically pause the audio output upon detecting the voice without processing a user command.

[0072] The steps for each of the methods outlined in FIG. 9, FIG. 10, and FIG. 11 are not meant to be limiting, but illustrative. It should be obvious to a person of ordinary skill in the art that the steps may be taken in different orders or using fewer or more steps than those outlined.

[0073] Throughout this disclosure, examples are described where a computing system (e.g., the hearable device 104, a client device, a server device, a computer, or another type of computing system) may analyze information associated with a user (e.g., the user voice 110 of FIG. 1). Further to the descriptions above, a user may be provided with controls allowing the user to make an election as to both if and when systems, programs, and / or features described herein may enable collection of information (e.g., information about a user’s social network, social actions, social activities, profession, preferences, or current location), as well as if the user is sent content or communications from a server. The computing system can be configured to only use the information after the computing system receives explicit permission from the user of the computing system to use the data. For example, in situations where the hearable device 104 may be used to determine if a user voice 110 is present, individual users may be provided with an opportunity to provide input to control whether programs or features of the hearable device 104 can collect and make use of the data. Further, individual users may have constant control overwhat programs can or cannot do with the information. In addition, information collected may be pre-treated in one or more ways before it is transferred, stored, or otherwise used so that personally identifiable information is removed. For example, a user’s identity may be treated so that no personally identifiable information can be determined for the user (for example, voice data used to train the ML model generator 700 from FIG. 7), such that the identity of the user cannot be determined. Thus, the user may have control over whether information is collected about the user and the user’s device and how such information, if collected, may be used by the computing device and / or a remote computing system.ADDITIONAL EXAMPLES

[0074] The following are additional examples of the described devices and methods for robust multi-channel voice activity detection for speech and audio processing in communication devices.

[0075] Example 1. An electronic device comprising one or more first sensors configured to detect audio external to a user; and output, based on the detected audio, an electronic audio signal; a memory configured to store computer-executable instructions; and one or more processors in communication with the memory and the one or more first sensors, the one or more processors configured to: receive the electronic audio signal output from the one or more first sensors; generate a feedback cancelation signal; generate, based on the electronic audio signal, an audio data; generate, using a machine-learned model, an output based on the feedback cancelation signal and the audio data; and detect, based on the output, a voice in the electronic audio signal.

[0076] Example 2. The electronic device of example 1, further comprising one or more second sensors configured to output a second signal; wherein the one or more processors are further configured to receive the second signal from the one or more second sensors; and wherein the audio data is further based on the second signal from the one or more second sensors.

[0077] Example 3. The electronic device of any one of the previous examples, wherein the feedback cancelation signal is based at least in part on the electronic audio signal.

[0078] Example 4. The electronic device of any one of the previous examples, wherein the one or more processors are further configured to, respondent to detecting the voice, perform an action.

[0079] Example 5. The electronic device of example 4, wherein the action comprises one or more of a hot-word detection, a command, an activation or cancelation of an active noise cancelation, and an automatic control of an audio output.

[0080] Example 6. The electronic device of any one of the previous examples, wherein: the one or more processors are further configured to receive a transmission signal; and the feedback cancelation signal is based at least in part on the transmission signal.

[0081] Example 7. The electronic device of any one of the previous examples, wherein the machine-learned model is trained by a machine-learning algonthm, the machine-learning algorithm comprising at least one of a convolutional neural network (CNN) and a long short-term memory (LSTM) algorithm.

[0082] Example 8. The electronic device of any one of the previous examples, wherein the one or more processors are further configured to, respondent to detecting the voice, determine whether there is more than one voice detected.

[0083] Example 9. An electronic device comprising: one or more first sensors configured to: detect audio external to a user; and output, based on the detected audio, an electronic audio signal; one or more second sensors configured to: detect a motion of the user’s body; and output a user body signal based on the motion of the user’s body; a memory configured to store computerexecutable instructions; and one or more processors in communication with the memory, the one or more first sensors, and the one or more second sensors, the one or more processors configured to: receive the electronic audio signal from the one or more first sensors; receive the user body signal from the one or more second sensors; generate, based on the electronic audio signal and the user body signal, an audio data; generate, using a machine-learned model, an output, the output based on the audio data; and detect, based on the output, a voice in the electronic audio signal.

[0084] Example 10. The electronic device of example 9, wherein the one or more first sensors comprise a microphone and the one or more second sensors comprise an inertial measurement unit sensor.

[0085] Example 11. The electronic device of any one of examples 9 and 10, wherein: the one or more processors are further configured to generate, based on the electronic audio signal, a feedback cancelation signal; and the generating of the output is further based on the feedback cancelation signal.

[0086] Example 12. The electronic device of any one of examples 9-11, wherein the one or more processors are further configured to, respondent to detecting the voice, determine whether there is more than one voice detected.

[0087] Example 13. The electronic device of any one of examples 9-12, wherein the machine-learned model is trained by a machine-learning algorithm, the machine-learning algorithm comprising at least one of a CNN and an LSTM algorithm.

[0088] Example 14. A method for detecting a voice by an electronic device, the method comprising: receiving, by the electronic device, an electronic audio signal generated based onaudio external to a user; generating, by the electronic device, a feedback cancelation signal; generating, by the electronic device and based on the electronic audio signal, an audio data; generating, by the electronic device and using a machine-learned model, an output based on the feedback cancelation signal and the audio data; and detecting, by the electronic device and based on the output, the voice in the electronic audio signal.

[0089] Example 15. The method of example 14, further comprising receiving, by the electronic device, a second signal generated based on a movement of a body of the user, wherein the audio data is further based on the second signal.

[0090] Example 16. The method of any one of examples 14-15, further comprising: constructing a spectrogram based at least in part on the audio data; and generating an input data based on the spectrogram, wherein the output being based on the feedback cancelation signal and the audio data comprises the output being based at least in part on the input data.

[0091] Example 17. The method of example 16, wherein the spectrogram is further based on the feedback cancelation signal.

[0092] Example 18. The method of any one of examples 14-17, further comprising performing an action respondent to detecting the voice.

[0093] Example 19. The method of example 18, wherein the action comprises one or more of a hot-word detection, a command, an activation or cancelation of an active noise cancelation, and an automatic control of an audio output.

[0094] Example 20. The method of any one of examples 14-19, wherein the machine- learned model is trained by a machine-learning algorithm, the machine-learning algorithm comprising at least one of a CNN and an LSTM algorithm.

[0095] Example 21. The method of any one of examples 14-20, wherein the one or more processors are further configured to, respondent to detecting the voice, determine whether there is more than one voice is detected.

[0096] Example 22. The method of any one of examples 14-21, further comprising receiving a transmission signal, wherein the feedback cancelation signal is based at least in part on the transmission signal.

[0097] Example 23. The method of any one of examples 14-21, wherein the feedback cancelation signal is based at least in part on the electronic audio signal.

[0098] Example 24. The method of any one of examples 14-23, wherein the audio signal is generated by one or more sensors.

[0099] Example 25. The method of example 24, wherein the one or more sensors comprise one or more of a microphone, an inertial measurement unit, an optical sensor, and an accelerometer.

[0100] Example 26. The method of any one of examples 14-25, wherein the electronic device is a pair of wireless earbuds.

[0101] Example 27. A method for detecting a voice by an electronic device, the method comprising: receiving, by the electronic device, an electronic audio signal generated based on audio external to a user; receiving, by the electronic device, a user body signal generated based on a movement of the user; generating, by the electronic device and based on the electronic audio signal and the user body signal, an audio data; generating, by the electronic device and using a machine-learned model, an output based on the audio data; and detecting, by the electronic device and based on the output, the voice in the audio data.

[0102] Example 28. The method of example 27, further comprising, respondent to detecting the voice, determining whether there is more than one voice.

[0103] Example 29. The method of any one of examples 27-28, wherein the machine- learned model is trained by a machine-learning algorithm, the machine-learning algorithm comprising at least one of a CNN and an LSTM algorithm.

[0104] Example 30. The method of any one of examples 27-29, further comprising: receiving a transmission signal; and generating, based on the transmission signal, a feedback cancelation signal, wherein the generating of the output is further based on the feedback cancelation signal.

[0105] Example 31. The method of any one of examples 27-29, further comprising generating, based on the electronic audio signal, a feedback cancelation signal, wherein the generating of the output is further based on the feedback cancelation signal.

[0106] Example 32. The method of any one of examples 27-31, further comprising: constructing a spectrogram based at least in part on the audio data; and generating an input data based on the spectrogram, wherein the output being based on the audio data comprises the output being based at least in part on the input data.

[0107] Example 33. The method of example 32, wherein the spectrogram is further based on the feedback cancelation signal.

[0108] Example 34. The method of any one of examples 27-33, further comprising performing an action respondent to detecting the voice.

[0109] Example 35. The method of example 34, wherein the action comprises one or more of a hot-word detection, a command, an activation or cancelation of an active noise cancelation, and an automatic control of an audio output.

[0110] Example 36. The method of any one of examples 27-35, wherein the electronic audio signal is generated by one or more sensors.

[0111] Example 37. The method of example 36, wherein the one or more sensors comprise one or more of a microphone, an inertial measurement unit, an optical sensor, and an accelerometer.

[0112] Example 38. The method of any one of examples 27-37, wherein the electronic device is a pair of wireless earbuds.

[0113] Example 39. A method for detecting a voice by an electronic device, the method comprising receiving, by the electronic device, an electronic audio signal generated based on audio external to a user; generating, by the electronic device and based on the electronic audio signal, a feedback cancelation signal; generating, by the electronic device and based on the electronic audio signal, an audio data; generating, by the electronic device and using a machine- learned model, an output based on the feedback cancelation signal and the audio data; and detecting, by the electronic device and based on the output, the voice in the electronic audio signal.

[0114] Example 40. The method of example 39, further comprising receiving, by the electronic device, a second signal generated based on a movement of a body of the user, and wherein generating the audio data is further based on the second signal.

[0115] Example 41. The method of any one of examples 39 or 40, further comprising constructing a spectrogram based at least in part on the audio data; and generating an input data based on the spectrogram, and wherein generating the output is further based on the input data.

[0116] Example 42. The method of example 41, wherein constructing the spectrogram is further based on the feedback cancelation signal.

[0117] Example 43. The method of any one of examples 39-42, further comprising performing an action responsive to detecting the voice.

[0118] Example 44. The method of example 43, wherein the action comprises one or more of detecting a hot-word in the electronic audio signal, executing a command, activating or canceling an active noise cancelation, and controlling an audio output.

[0119] Example 45. The method of any one of examples 39-44, wherein the machine- learned model is trained by a machine-learning algorithm, the machine-learning algorithm comprising at least one of a convolutional neural network (CNN) and a long short-term memory' (LSTM) algorithm.

[0120] Example 46. The method of any one of examples 39-45, further comprising determining, responsive to detecting the voice, that a second voice was detected in the electronic audio signal.

[0121] Example 47. The method of any one of examples 39-46, further comprising receiving a transmission signal, and wherein generating the feedback cancelation signal is based at least in part on the transmission signal.

[0122] Example 48. The method of any one of examples 39-47, further comprising receiving, by the electronic device, a user body signal generated based on a movement of the user, and wherein generating the feedback cancelation signal is further based on the user body signal.

[0123] Example 49. The method of any one of examples 39-48, wherein the audio signal is generated by one or more sensors.

[0124] Example 50. The method of example 49, wherein the one or more sensors comprise one or more of a microphone, an inertial measurement unit, an optical sensor, and an accelerometer.

[0125] Example 51. A non-transitory computer-readable medium storing instructions that, when accessed by one or more processors, cause the one or more processors to execute any one of the methods of examples 14-50.

[0126] Example 52. An electronic device comprising: one or more sensors; one or more processors; and a memory configured to store computer-executable instructions that, when accessed by the one or more processors, cause the one or more processors to execute any one of the methods of examples 14-50.CONCLUSION

[0127] While the present subj ect matter has been described in detail with respect to various specific example implementations thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such implementations. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations, and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one implementation can be used with another implementation to yield a still further implementation. Thus, it is intended that the present disclosure cover such alterations, variations, and equivalents.

Claims

CLAIMSWhat is claimed is:

1. A method for detecting a voice by an electronic device, the method comprising: receiving, by the electronic device, an electronic audio signal generated based on audio external to a user; generating, by the electronic device and based on the electronic audio signal, a feedback cancelation signal; generating, by the electronic device and based on the electronic audio signal, an audio data; generating, by the electronic device and using a machine-learned model, an output based on the feedback cancelation signal and the audio data; and detecting, by the electronic device and based on the output, the voice in the electronic audio signal.

2. The method of claim 1, further comprising receiving, by the electronic device, a second signal generated based on a movement of a body of the user, and wherein generating the audio data is further based on the second signal.

3. The method of any one of the previous claims, further comprising: constructing a spectrogram based at least in part on the audio data; and generating an input data based on the spectrogram, and wherein generating the output is further based on the input data.

4. The method of claim 3, wherein constructing the spectrogram is further based on the feedback cancelation signal.

5. The method of any one of the previous claims, further comprising performing an action responsive to detecting the voice.

6. The method of claim 5, wherein the action comprises one or more of detecting a hot-word in the electronic audio signal, executing a command, activating or canceling an active noise cancelation, and controlling an audio output.

7. The method of any one of the previous claims, wherein the machine-learned model is trained by a machine-learning algorithm, the machine-learning algorithm comprising at least one of a convolutional neural network (CNN) and a long short-term memory (LSTM) algorithm.

8. The method of any one of the previous claims, further comprising determining, responsive to detecting the voice, that a second voice was detected in the electronic audio signal.

9. The method of any one of the previous claims, further comprising receiving a transmission signal, and wherein generating the feedback cancelation signal is based at least in part on the transmission signal.

10. The method of any one of the previous claims, further comprising receiving, by the electronic device, a user body signal generated based on a movement of the user, and wherein generating the feedback cancelation signal is further based on the user body signal.

11. The method of any one of the previous claims, wherein the audio signal is generated by one or more sensors.

12. The method of claim 11 , wherein the one or more sensors comprise one or more of a microphone, an inertial measurement unit, an optical sensor, and an accelerometer.

13. The method of any one of the previous claims, wherein the electronic device is a pair of wireless earbuds.

14. A non-transitory computer-readable medium storing instructions that, when accessed by one or more processors, cause the one or more processors to execute any one of the methods of claims 1-13.

15. An electronic device comprising: one or more sensors; one or more processors; and a memory' configured to store computer-executable instructions that, when accessed by the one or more processors, cause the one or more processors to execute any one of the methods of claims 1-13.

Images