Self voice audio processing for hearing loss

Abstract

Aspects of the subject technology relate to a device comprising a microphone and a processor. The processor receives an audio signal corresponding to the microphone. The processor detects one or more of ambient sound or speech of a user of the device in the received audio signal. The processor applies a first gain to the ambient sound when the ambient sound is detected in the received audio signal, and applies a second gain different from the first gain to the speech of the user of the device when the speech of the user of the device is detected in the received audio signal.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 530,051, filed on July 31, 2023, entitled “OWN VOICE AUDIO PROCESSING FOR HEARING LOSS,” the disclosure of which is expressly incorporated herein by reference in its entirety. Technical Field

[0003] The present specification relates generally to audio processing and, more particularly, but not exclusively, to own speech audio processing for hearing loss. Background Art

[0004] Active headphones with audio processing can analyze and optimize audio signals in real time to create a realistic and immersive sound field. In addition, these headphones can integrate active noise reduction technology, using microphones to capture ambient noise and process and / or filter audio signals containing unwanted sounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Certain features of the subject technology are set forth in the appended claims. However, for purposes of illustration, several aspects of the subject technology are set forth in the following figures.

[0006] Figure 1 A high-level block diagram illustrating an example of self-speech transmission through air conduction paths and bone conduction paths in accordance with various aspects of the subject technology.

[0007] Figure 2 An example environment is conceptually illustrated in which an electronic device (or system) that performs own speech audio processing for hearing loss can be implemented in accordance with various aspects of the subject technology.

[0008] Figure 3 A high-level block diagram illustrating an example of an electronic device with native speech audio processing in accordance with various aspects of the subject technology.

[0009] Figure 4 Illustrate various aspects of the subject technology Figure 2 A high-level block diagram of the audio processing circuit.

[0010] Figure 5A Graphs illustrating example loudness growth functions for own speech and ambient sounds for the hearing impaired in accordance with various aspects of the subject technology.

[0011] Figure 5BA diagram illustrating example loudness compensation for own speech and ambient sounds for the hearing impaired in accordance with various aspects of the subject technology.

[0012] Fig. 6A A graph illustrating examples of dominant ambient sounds coexisting with the sound of one's own speech at frequency bins in accordance with various aspects of the subject technology.

[0013] Figure 6B A graph illustrating example loudness compensation for frequency bins coexisting with own speech and ambient sounds in accordance with various aspects of the subject technology.

[0014] Figure 7 A flow chart illustrating an example process for own speech audio processing in accordance with various aspects of the subject technology.

[0015] Figure 8

[0013] A wireless communication device is illustrated within which some aspects of the subject technology are implemented. DETAILED DESCRIPTION

[0016] The detailed description set forth below is intended to be a description of various configurations of the subject technology and is not intended to represent the only configuration by which the subject technology can be practiced. The accompanying drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details intended to provide a thorough understanding of the subject technology. However, it will be clear and obvious to those skilled in the art that the subject technology is not limited to the specific details set forth herein and can be practiced without these specific details. In some cases, well-known structures and components are shown in block diagram form to avoid blurring the concept of the subject technology.

[0017] Assuming, for example, that a user's own voice can be perceived through both a bone conduction path and an air conduction path, the user's own voice can be sent and perceived differently from ambient sound. However, loudness compensation (or amplification) for hearing-impaired users is primarily derived from ambient sound sensing. For hearing-impaired users, applying this same loudness compensation to the user's own voice may be perceived as unnatural. The subject technology attempts to make the hearing-impaired user's own voice more natural. By providing hearing-impaired users with better perception of their own voice, hearing-impaired users can, for example, achieve better control over their own voice level.

[0018] The subject technology can use own voice activity detection to distinguish own voice from environmental sounds (e.g., ambient noise, music, speech, etc.). The own voice activity detection can receive sensor signals from one or more in-ear sensors (e.g., error-microphone, accelerometer) and external ear sensors (e.g., reference-microphone, voice-microphone). The subject technology can classify dominant signals for one or more sub-bands or frequency intervals with coexisting own voice and environmental sounds. The technology can apply different loudness compensation (or amplification) between own voice and environmental sounds according to different psychoacoustic loudness growth functions. As a type of noise suppression before loudness compensation, when own voice and environmental sounds coexist in a frequency interval, the technology can also attenuate the frequency interval dominated by environmental sounds.

[0019] Figure 1 A high-level block diagram illustrating an example of self-speech transmission through air conduction paths and bone conduction paths in accordance with various aspects of the subject technology. Figure 1 As illustrated in FIG. 1 , the physical transmission path of the sound of the own voice includes transmission through an air conduction path and a bone conduction path. For example, the air conduction path passes through the vocal cords 102, the vocal cavity 104, the mouth 106, the outer ear 110, the middle ear 112, the inner ear 114, and reaches the auditory nerve 116. In another example, the bone conduction path passes through the vocal cords 102, the vocal cavity 104, the skull 108, the outer ear 110, the middle ear 112, the inner ear 114, and reaches the auditory nerve 116.

[0020] The transmission of one's own speech through the air conduction path involves several anatomical structures. The process begins with the vocal cords 102 located in the larynx (voice box). The vocal cords 102 vibrate to produce sound waves. These sound waves then travel through the vocal cavity 104 and reach the mouth 106. From the mouth 106, the sound waves propagate through the surrounding air and enter the outer ear 110, specifically the auricle (the visible part of the ear).

[0021] The sound waves then travel through the ear canal and reach the middle ear 112. In the middle ear 112, the sound waves encounter the tympanic membrane (eardrum), which vibrates in response to the sound waves. The vibrations are sent to three small bones called ossicles in the middle ear 112: the malleus (hammer), the incus (anvil), and the stapes (stirrup). The ossicles amplify the vibrations and send them to the oval window, which is a membrane-covered opening to the inner ear 114.

[0022] Once the sound waves reach the inner ear 114, they stimulate the fluid-filled cochlea (a spiral-shaped structure). The movement of fluid in the cochlea causes the hair cells lining its walls to bend, generating electrical signals. These electrical signals are then picked up by the auditory nerve 116 that connects the cochlea to the brain. The auditory nerve 116 carries the signals to the auditory cortex of the brain, where they are interpreted as sound.

[0023] Similar to the air conduction path, the bone conduction path also involves the vocal cords 102 and the vocal cavity 104. However, instead of the sound waves traveling through the air, they are conducted through the bones of the skull. The vibrations generated by the vocal cords 102 and the resonances in the vocal cavity 104 are sent directly to the skull 108.

[0024] The vibrations are sent from the skull 108 to the outer ear 110, specifically to the pinna where they are collected. The vibrations then travel through the ear canal and reach the middle ear 112 where they cause the eardrum and ossicles to vibrate in a manner similar to that in the air conduction path. The ossicles amplify the vibrations, which are then sent to the oval window.

[0025] Once the vibrations reach the inner ear 114, they continue to stimulate the cochlea in the same way as in the air conduction path. The fluid in the cochlea moves, causing the hair cells to bend and generate electrical signals. These signals are picked up by the auditory nerve 116 and sent to the brain for interpretation.

[0026] In one or more embodiments, the sound of one's own voice is transmitted and perceived differently from the ambient sound. For example, the physical transmission path of the ambient sound includes transmission at least partially through an air conduction path without passing through a bone conduction path. The ambient sound from a sound source in the surrounding environment can be received at the outer ear 110, and along the air conduction path, the sound waves continue to propagate through the middle ear 112, the inner ear 114 and stimulate the auditory nerve 116, and finally reach the brain for sound perception.

[0027] In the field of hearing loss compensation, amplification of ambient sounds is of major concern for several reasons. First, the technical challenges associated with reliably detecting one's own voice make it difficult to distinguish the signal captured by the microphone as the user's own voice or ambient sounds. This challenge is exacerbated by the lack of in-ear sensors (such as accelerometers) in most conventional hearing aids. The lack of these sensors has hampered the development of reliable algorithms for distinguishing the user's voice from ambient sounds.

[0028] Second, while international and U.S. standards (e.g., ISO 532; ANSI S3.4) provide loudness models for ambient sounds, including extensions for hearing loss, there is no readily available loudness growth function for a user's own speech, whether in normal or impaired hearing. As a result, hearing aid manufacturers lack guidance on setting appropriate gain levels to restore the loudness of a user's own speech to a normal range. For example, when the ear canal is blocked by an earpiece, there is an occlusion effect that boosts sound frequencies below 1 kHz by up to 30 dB and causes a lack of clarity in one's own speech. In another example, headphones with fed-mix active noise reduction (FBANC) can attenuate the occlusion effect by up to about 20 dB, where 20 dB is the approximate limit of FBANC stability. In another example, a hearing aid without FBANC may traditionally use an ear mold with a vent or an ear plug with a hole to unclog the ear canal and reduce the occlusion effect.

[0029] Furthermore, empirical evidence contradicts the notion that hearing aid users will adapt to their amplified own voice. In fact, users often express dissatisfaction with the perception that their own voice is unnatural. This unnatural perception, in turn, affects the user's ability to control their own voice level during a conversation. For example, if the amplification is set too high for the user's own voice, they may need to change their pronunciation, resulting in reduced audibility for the listener. Conversely, speaking to the listener at a normal level causes the user to perceive their own voice as too loud due to the high amplification of the hearing aid.

[0030] The above reasons emphasize the primary emphasis on amplifying all sounds, including both ambient sounds and one's own voice, at the same level in hearing loss compensation, as opposed to applying different gains to ambient sounds and the user's own voice, because challenges associated with detecting one's own voice, the lack of in-ear sensors, and the lack of reliable algorithms have hindered effective compensation for the user's voice. Because the user's own voice is perceived as different from ambient sounds due to the different physical transmission paths between the ambient sounds and the user's own voice, amplifying all sounds to the same level using loudness compensation traditionally derived from ambient sound sensing results in an unnatural perception of the user's own voice. Addressing these limitations may help develop improved hearing aid systems that provide accurate and natural amplification for the user's own voice. This will be discussed below. FIG. 5A to FIG. 5B The effects of the two conduction paths are described in more detail in .

[0031] Figure 2An example environment 200 is conceptually illustrated in which an electronic device (or system) that performs own speech audio processing for hearing loss can be implemented in accordance with various aspects of the subject technology. Environment 200 includes electronic device 210 and ambient sound 220. In one or more specific implementations, ambient sound 220 represents various audio signals that can be present in environment 200, including speech, noise, and other acoustic stimuli.

[0032] The electronic device 210 includes an audio processing circuit 230. The electronic device 210 acts as a platform for implementing the audio processing circuit 230 and transmitting the optimized audio signal to the user. It may include hardware and software circuits to perform real-time processing, including signal acquisition, processing algorithms, and audio output capabilities. In some specific implementations, some functionality of the audio processing circuit 230 may be performed by a processor of a host device (such as a smart phone or smart watch). In one or more specific implementations, the electronic device 210 may be a headset, earphones, earbuds, or audio output device that is wirelessly or via a wired link coupled to a handheld device (such as a smart phone, smart watch, or tablet) or a computing device (such as a laptop or desktop computer).

[0033] The audio processing circuit 230 helps enhance the user's perception of his own voice and ambient sounds. It includes two main elements: a loudness compensation circuit 240 and an own voice activity detector (OVAD) 250 with multiple sensors.

[0034] The loudness compensation circuit 240 includes an own voice gain circuit 242 and an ambient sound gain circuit 244. The own voice gain circuit 242 performs operations related to amplifying and adjusting the user's own voice signal, which will be referred to below. Figures 3 to 7 Described in more detail. Own speech gain circuit 242 may account for the specific hearing loss characteristics of an individual user and facilitate clear and comfortable perception of amplified own speech.

[0035] The ambient sound gain circuit 244 adjusts the amplification level of the ambient sound captured by the electronic device 210. The ambient sound gain circuit 244 may employ algorithms and signal processing techniques to enhance the audibility of the ambient sound and optimize its clarity.

[0036] The own voice activity detector (OVAD) 250 with multi-sensor helps to accurately identify the user's own voice activity. The OVAD 250 with multi-sensor can integrate multiple different sensors (such as in-ear sensors and external ear sensors) to capture and / or analyze audio signals. The multi-sensor approach helps to distinguish the user's voice from other environmental sounds and provides contextual information for accurate processing and adjustment.

[0037] Figure 3A high-level block diagram of an example of an electronic device 210 with native speech audio processing according to various aspects of the subject technology is illustrated. The electronic device 210 incorporates multiple sensors to enhance its functionality and improve the user's listening experience. The electronic device 210 may be composed of in-ear sensors and out-ear sensors, each serving a specific purpose. In one or more specific implementations, the in-ear sensors and out-ear sensors are as described in reference Figure 2 The OVAD 250 with multiple sensors is described or includes at least a portion thereof. The in-ear sensor includes an error microphone 304 and an accelerometer 302, while the outer ear sensor consists of a reference microphone 306 and a speech microphone 308. In one or more specific implementations, the electronic device 210 includes a speaker 310.

[0038] An error microphone 304 located in the ear canal captures sound signals generated by a speaker 310 of the electronic device 210. The error microphone 304 provides feedback about the output sound, allowing the electronic device 210 to adjust its settings and facilitate accurate sound reproduction. An accelerometer 302, also located in the ear, measures motion and vibration, providing contextual information about the user's movements and activities, such as the movement of the user's mouth when speaking.

[0039] An outer ear sensor including a reference microphone 306 and a speech microphone 308 is placed outside the ear. The reference microphone 306 picks up ambient sounds and background noise in the environment, while the speech microphone 308 is specifically used to capture the user's own voice.

[0040] Own voice activity detector (e.g. Figure 4 406) helps to distinguish the user's own voice from the ambient sound, and the reference Figure 4 This is described in more detail. It combines signals from one or more of the sensors to accomplish this task. The inputs of the error microphone 304, accelerometer 302, reference microphone 306, and / or voice microphone 308 are collectively analyzed to determine the presence of a user's voice.

[0041] The signals of the error microphone 304 and the accelerometer 302 provide information about the generated sound and the user's movement pattern, respectively. By comparing these signals with the inputs of the reference microphone 306 and the voice microphone 308, the self voice activity detector can distinguish the user's voice from the ambient sound.

[0042] Reference microphone 306 captures ambient noise, thereby helping to identify and suppress unwanted sounds in the user's own voice. In addition, voice microphone 308 is specifically used to capture the user's voice, thereby serving as the primary source for detecting the presence of own voice activity.

[0043] Using advanced algorithms and signal processing techniques, the own voice activity detector analyzes and combines the inputs from all sensors. It applies pattern recognition, spectrum analysis and / or other algorithms to distinguish the unique characteristics of the user's own voice from the ambient sound. For example, the own voice activity detector can execute the OVAD algorithm. In one or more specific implementations, the OVAD algorithm can combine one or more of the sensor signals to distinguish the user's own voice from the ambient sound. In one or more specific implementations, the input signal can be divided into the following types of classifications by the OVAD algorithm: a classification indicating that the input signal only includes one's own voice, a classification indicating that the input signal only includes ambient sound, and a classification indicating that the input signal includes a combination of one's own voice and ambient sound.

[0044] By accurately detecting own voice activity, the electronic device 210 can optimize the processing and amplification settings specific to the user's voice. This enables improved audibility and clarity, thereby minimizing interference from ambient sounds and maximizing the user's ability to perceive and understand their own speech.

[0045] Figure 4 Illustrate various aspects of the subject technology Figure 2 2 is a high-level block diagram of another example of an audio processing circuit 230 of FIG. The audio processing circuit 230 includes a loudness compensation circuit 240 and an OVAD with multiple sensors 250. The OVAD with multiple sensors 250 includes a microphone 308, an accelerometer 302, and an own voice activity detector 406. The loudness compensation circuit 240 includes a noise suppression circuit 408, an own voice adjustment circuit 410, a hearing loss compensation circuit 412, and a gain combiner 414. The audio processing circuit 230 is also coupled to a speaker 310.

[0046] The own voice audio processing system includes several circuits designed to enhance the quality and intelligibility of the user's own voice.The own voice audio processing system includes a microphone 308 to capture the user's voice and an accelerometer 302 to measure motion and vibration.

[0047] The signal of microphone 308 is separated and transmitted to two processing circuits: noise suppression circuit 408 and own voice activity detector 406. In one or more implementations, microphone 308 may include at least a portion of reference microphone 306. In one or more implementations, audio processing circuit 230 includes reference microphone 306 separate from microphone 308 and coupled to one or more of own voice activity detector 406, noise suppression circuit 408, hearing loss compensation circuit 412, or gain combiner 414.

[0048] The noise suppression circuit 408 is intended to reduce background noise and other unwanted sounds in the input of the microphone 308. At the same time, the own voice activity detector 406 analyzes the audio signal to detect and distinguish the user's own voice activity.

[0049] The data from the accelerometer 302 is also fed into the own voice activity detector 406, providing additional contextual information that helps accurately identify the user's own voice. By incorporating motion and vibration data, the own voice audio processing system can better distinguish the user's voice from other environmental sounds.

[0050] The own voice activity detector 406 can accurately detect and determine the occurrence of the user's own voice. This is achieved by calculating metrics such as magnitude squared coherence, normalized cross coherence, and / or other suitable metrics.

[0051] Magnitude squared coherence is a metric used to quantify the similarity or coherence between two signals. In the context of own voice activity detector 406, it measures the similarity between a captured audio signal and a reference signal representing a user's own voice. By analyzing the magnitude squared coherence, own voice activity detector 406 can identify whether the captured signal corresponds to the user's own voice activity.

[0052] Normalized cross-correlation is another metric that can be used in own voice activity detector 406. It measures the similarity between two signals by calculating the degree of coherence between them. In this case, own voice activity detector 406 calculates the normalized cross-correlation between the captured audio signal and a reference signal representing the user's own voice. By analyzing the resulting value, own voice activity detector 406 determines whether the captured signal contains the user's own voice.

[0053] These metrics are calculated using signal processing techniques within own voice activity detector 406. Own voice activity detector 406 can continuously process the audio input, thereby capturing and analyzing the characteristics of the signal in real time. By comparing the calculated metrics with predefined thresholds, own voice activity detector 406 can classify whether the audio input corresponds to the user's own voice or other environmental sounds.

[0054] The use of metrics such as magnitude squared coherence or normalized cross coherence provides a reliable means of determining the occurrence of a user's own voice. These calculations enable the own voice activity detector 406 to make accurate, informed decisions in the audio processing system, thereby facilitating the appropriate handling of the user's own voice for optimal performance and user experience.

[0055] The noise suppression circuit 408 is a component of a self-speech audio processing system whose purpose is to reduce the presence of background noise in a speech signal. In one or more specific implementations, the noise suppression circuit 408 is implemented at least in part by hardware, firmware and / or software. One aspect of its operation relates to estimating bin-by-bin suppression gain for speech signals. The bin-by-bin suppression gain estimation is based on analyzing the spectral characteristics of the speech signal in each frequency bin. The circuit performs spectral analysis of the incoming speech signal, thereby dividing it into independent frequency bins.

[0056] For each frequency interval, the self-speech adjustment circuit 410 calculates a suppression gain value. The gain value represents a reference Figure 6B The amount by which the ambient sound in this particular frequency interval should be attenuated is illustrated. The gain estimation takes into account the power or amplitude of the ambient sound relative to the desired own speech signal.

[0057] To determine the suppression gain, the own voice adjustment circuit 410 may rely on statistical analysis and adaptive algorithms, utilizing techniques such as spectral subtraction, Wiener filtering, and / or other advanced algorithms that take into account both the properties of the ambient sound and the characteristics of the desired own voice signal.

[0058] The estimation of the bin-by-bin suppression gain in the noise suppression circuit 408 can help achieve noise reduction while preserving the integrity and naturalness of the speech signal. By adaptively adjusting the suppression gain in real time, the noise suppression circuit 408 facilitates optimal noise suppression performance across different acoustic environments and noise types.

[0059] The output from the noise suppression circuit 408 is then passed to the own voice adjustment circuit 410. The own voice adjustment circuit 410 processes the signal from the microphone 308 to refine and optimize the user's voice rendering based on the own voice activity detected from the own voice activity detector 406. It adjusts parameters such as frequency response, gain, or dynamic range to improve the overall clarity and quality of the own voice signal.

[0060] The own voice adjustment circuit 410 plays a key role in the audio processing system by fine-tuning the presentation of the user's own voice. It does this by combining the output of the own voice activity detector 406 with the output of the noise suppression circuit 408 to estimate the interval-by-interval own voice presence probability (OVPP). In one or more specific implementations, the OVPP indicates the likelihood of the presence of own voice in a frequency interval.

[0061] In one or more specific implementations, the self-voice adjustment circuit 410 may determine a dominant signal between the ambient sound and the voice of the user of the electronic device 210 for one or more frequency intervals associated with the received audio signal based on at least partial overlap in frequency between the ambient sound and the voice of the user of the electronic device 210.

[0062] The OVPP values ​​generated by the own voice adjustment circuit 410 help determine the dominant signal between the user's own voice and the ambient sound within a given frequency interval. The hearing loss compensation circuit 412 uses this information to accurately distinguish and prioritize the relevant signals. When processing the audio signal, the hearing loss compensation circuit 412 analyzes the OVPP values ​​on a bin-by-bin basis. Each frequency bin is evaluated to assess the likelihood that the user's own voice is present compared to the ambient sound.

[0063] Based on the OVPP value of a particular frequency interval, the own voice adjustment circuit 410 determines the dominant signal. If the OVPP value indicates a high probability of the user's own voice, the own voice adjustment circuit 410 identifies the own voice as the dominant signal in the interval. Conversely, if the OVPP value implies a low probability of the user's own voice, the own voice adjustment circuit 410 identifies the ambient sound as the dominant signal.

[0064] By making this distinction, the own voice adjustment circuit 410 can adapt its processing techniques accordingly. When the user's own voice is identified as the dominant signal, the own voice adjustment circuit 410 helps the own voice be retained and appropriately amplified to maintain audibility and intelligibility. Conversely, when ambient sounds are identified as the dominant signal, the own voice adjustment circuit 410 focuses on suppressing or reducing the amplification of those sounds to enhance speech clarity and reduce background noise interference.

[0065] By applying the estimated bin-by-bin suppression gain to the speech signal, the noise suppression circuit 408 selectively reduces the amplitude or power of the noise in each frequency bin having a dominant signal characterized as ambient sound. In one or more specific implementations, when the dominant signal is characterized as own speech, the noise suppression circuit 408 may refrain from attenuating the speech signal in the corresponding frequency bin. This process helps enhance the clarity and intelligibility of the speech signal by minimizing interference caused by background noise.

[0066] The own voice activity detector 406 provides information about the presence of the user's own voice within the audio input. It analyzes the characteristics of the signal to identify and classify segments containing the user's own voice. The output of the own voice activity detector 406 indicates the presence or absence of the user's own voice at different time intervals. In one or more specific implementations, the own voice activity detector 406 is at least partially implemented by hardware, firmware, or software. In some specific implementations, some functionality of the own voice activity detector 406 can be performed by a processor of a host device (such as a smart phone or smart watch).

[0067] On the other hand, the noise suppression circuit 408 focuses on reducing the impact of background noise in the audio signal. It selectively attenuates noise to enhance the intelligibility and clarity of the desired speech signal. For example, when the dominant signal is characterized as ambient sound, the noise suppression circuit 408 selectively attenuates the gain of the speech signal. The output of the noise suppression circuit 408 represents the processed speech signal with reduced noise.

[0068] The own voice adjustment circuit 410 combines the outputs of the own voice activity detector 406 and the noise suppression circuit 408 to estimate the interval-by-interval OVPP. It analyzes the information provided by the own voice activity detector and the presence or absence of the noise-suppressed speech signal. In one or more specific implementations, the own voice adjustment circuit 410 is at least partially implemented by hardware, firmware, or software. In some specific implementations, some functionality of the own voice adjustment circuit 410 can be performed by a processor of a host device (such as a smart phone or smart watch).

[0069] By examining the detection time intervals of the own voice and comparing it to the noise suppressed speech signal, the own voice adjustment circuit assigns a probability value to each frequency bin. The probability represents the likelihood that the user's own voice is present in that particular bin.

[0070] The estimation of the probability of the presence of one's own voice in each interval enables precise control and adjustment of the user's own voice in the final audio output. It allows selective modification of the user's voice presentation, thereby ensuring appropriate amplification or attenuation according to the estimated probability of presence.

[0071] The interval-by-interval OVPP from the own voice adjustment circuit 410 provides valuable information that can be utilized in the own voice processing model to adjust for hearing loss compensation. This enables the user's own voice to be processed differently from ambient sounds, resulting in an optimized listening experience. In one or more specific implementations, the own voice processing model may represent a trained machine learning model, which will be referred to below in reference Figure 8 Describe some aspects of the model.

[0072] To determine the OVPP value, the own voice adjustment circuit 410 analyzes the characteristics of the audio signal in each frequency interval over time. It takes into account both the user's own voice and the ambient sound present in that interval. By examining the time characteristics of the signal, the own voice adjustment circuit 410 can identify instances of overlap.

[0073] In the event of overlap, the self-speech adjustment circuit 410 adopts a decision process that prioritizes the user's own speech as the dominant signal within the frequency interval. This is based on the understanding that the user's own speech is usually the main speech signal of concern and the ambient sound is considered background noise or interference.

[0074] By assigning a higher probability to the user's own speech during the overlapping time intervals, the own speech adjustment circuit 410 indicates its dominance within the frequency interval. This identification enables subsequent processing stages to treat the user's own speech differently from ambient sound.

[0075] The method allows accurate presentation of and differentiation between a user's own voice and ambient speech. This helps the user's speech be prioritized and receive processing tailored to their specific listening needs, while ambient sounds are attenuated or suppressed to enhance speech intelligibility.

[0076] In general, by considering the dominance of the user's own voice within the frequency interval, the own voice adjustment circuit 410 generates an OVPP value that reflects the presence and significance of the user's speech. This information can be used by subsequent processing stages to distinguish and optimize the user's own voice and environmental sounds, thereby providing an improved and customized listening experience.

[0077] The own speech processing model takes into account the OVPP values ​​estimated for each frequency bin. It uses this information to apply customized processing techniques specifically tailored to the user's own speech. By distinguishing the user's own speech from environmental sounds, the model can address the unique characteristics and challenges associated with applying hearing loss compensation to the user's speech.

[0078] Based on the OVPP value, the own speech processing model can selectively enhance or modify certain aspects of the user's own speech presentation. This may involve adjusting gain, frequency response, dynamic range, or other parameters to compensate for the user's specific hearing loss profile. These modifications are intended to optimize the audibility, clarity, and overall perception of the user's own speech.

[0079] Meanwhile, the own voice processing model can apply different processing techniques to the ambient sound circuit. By identifying the absence of the user's own voice in a specific frequency interval, the model can focus on suppressing or reducing the amplification of background noise or other non-essential sounds. This selective processing helps enhance the perception of the user's own voice by minimizing interference from ambient sounds.

[0080] The integration of interval-by-interval OVPP values ​​within the own speech processing model allows for personalized and adaptive adjustments in hearing loss compensation. By processing the user's own speech differently from ambient sounds, the model optimizes the presentation of the user's speech, thereby enhancing audibility and intelligibility. This customized approach contributes to a more effective and natural listening experience for individuals with hearing loss.

[0081] At the same time, the output of the own voice adjustment circuit 410 and the output from the microphone 308 are directed to the hearing loss compensation circuit 412, which applies a specific algorithm to compensate for any hearing impairment that the user may have. The circuit modifies the audio signal to enhance audibility and facilitate optimal perception for individuals with different degrees of hearing loss. In one or more specific implementations, the hearing loss compensation circuit 412 applies a general gain to account for the user's hearing loss; however, the own voice adjustment circuit 410 has previously applied or adjusted the gain of the user's own voice so that it is amplified differently from ambient sounds by the hearing loss compensation circuit 412. In one or more specific implementations, the hearing loss compensation circuit 412 is at least partially implemented by hardware, firmware, or software. In some specific implementations, some functionality of the hearing loss compensation circuit 412 may be performed by a processor of a host device (such as a smart phone or smart watch).

[0082] The output of microphone 308 is combined with the outputs from hearing loss compensation circuit 412 and noise suppression circuit 408 in gain combiner 414. Gain combiner 414 adjusts the volume levels of the different input signals and combines them into a unified output signal.

[0083] The final audio signal output from the gain combiner 414 is then routed to the speaker 416, allowing the user to hear their own speech with improved clarity and intelligibility.

[0084] In one or more specific implementations, the gain combiner 414 is implemented at least in part by hardware, firmware or software. In some specific implementations, some functionality of the gain combiner 414 may be performed by a processor such as a host device (such as a smart phone or smart watch). The gain combiner 414 adds the gain compensation signal of the hearing loss compensation circuit 412 to the input audio signal from the microphone 308, and provides the final audio signal output to the speaker 416. Because the gain compensation signal includes a separate gain between the user's own voice and the ambient sound, the final audio signal output includes the different gains applied to the own voice and the ambient sound. Based on the level of ambient noise in the input signal, the final audio signal output includes the noise-free audio from the microphone 308 and the user's own voice at a controlled level.

[0085] In one or more specific implementations, the user of the electronic device 210 may go through a hearing profile registration process to configure the electronic device 210 to meet any specific needs and preferences of the user. During the hearing profile registration, a fine-tuning slider representing a visual control interface may be presented to the user. The fine-tuning slider is a user-adjustable feature that is designed to allow individuals to customize and adjust their preferred sound gain during the hearing profile registration process. This feature provides a personalized and flexible method to meet the specific needs and preferences of the user by allowing the user to adjust the user-defined gain settings that are configurable via user input. For example, the fine-tuning slider may allow the user to manually adjust the user's own voice gain level for their own voice via the fine-tuning slider by user input according to their personal preferences. The user may move the slider along a continuum to indicate the desired amplification level for the gain level of their own voice. In one or more other specific implementations, another fine-tuning slider that allows the user to manually adjust the ambient sound gain level according to their personal preferences may be presented to the user.

[0086] When the user adjusts the fine-tuning slider, the hearing profile registration system dynamically modifies the gain setting (e.g., for the user's own voice) to reflect the user's preferences. The system captures the user's preferred sound gain as a user-defined gain setting and stores it as part of their personalized hearing profile. By providing this level of control and customization, the fine-tuning slider enables individuals to customize the amplification of their own voice according to their specific comfort and communication needs. This helps the listening device or audio processing system to deliver a personalized and optimized listening experience based on the user's preferred sound gain setting. This feature provides a user-friendly interface and enhances user participation by allowing them to actively participate in the hearing profile registration process. By fine-tuning the sound gain according to their own preferences, individuals can achieve a more personalized and satisfying listening experience.

[0087] Figure 5A A diagram 500 of an example loudness increase function for own speech and ambient sound for a hearing impaired person according to various aspects of the subject technology is illustrated. The diagram 500 shows different loudness compensations between own speech and ambient sound. The transmission and perception of a user's own speech exhibits distinguishable properties in both physical and psychoacoustic terms compared to ambient sound. In terms of physical properties, the sound pressure level is different due to different transmission distances to the ear.

[0088] For example, during a normal level conversation, when the user acts as a listener, the ambient sound (speech) source is approximately 1 meter away, resulting in the sound pressure level (SPL) reaching the user's ear entrance being at approximately 60 dB sound pressure level (or dBSPL). On the other hand, when the user plays the role of a speaker, the own voice at the ear entrance is approximately 72 dBSPL, as compared to the 60 dBSPL at a distance of approximately 1 meter. In this regard, since the perceived loudness (Song) of the own voice is higher than the loudness of the ambient sound (this is attributed to the reference Figure 1 The user's own voice and ambient sound described travel through different physical transmission paths), so the gain compensation required for the own voice will be less than the gain compensation expected for the gain compensation of the ambient sound.

[0089] In terms of psychoacoustic perception, the perceived loudness grows differently as the sound pressure level increases. For example, for a hearing-impaired user, own speech and ambient sound have different loudness growth functions. In one or more specific implementations, the operation encompasses utilizing different loudness growth functions for both normal hearing conditions and hearing-impaired conditions. The loudness growth function characterizes the relationship between perceived loudness and physical sound pressure level. In this context, the loudness unit used is the sonne, where 2 sones are perceived as twice as loud as 1 sonne. Specifically, 1 sonne is defined as the loudness of a 1kHz tone at 40dBSPL in a free field.

[0090] like Figure 5A As illustrated in FIG. 5 , the disclosed loudness growth function illustrates the difference between own speech and ambient sound represented by lines 502 and 504 , respectively. Figure 5A The lines 502, 504 show the increase in sound pressure level required to achieve double loudness in different scenarios. For hearing-impaired users, both loudness increase functions become steeper, but the line 502 representing own speech has a steeper slope than the line 504 representing ambient sound.

[0091] For example, to double the loudness perception, the ambient sound would require a physical sound pressure level that is approximately 10 dB (approximately 3 times) higher than the initial level. In contrast, the user's own voice would only require a physical sound pressure level increase of less than 5 dB (less than 2 times) to achieve the same doubling of loudness perception.

[0092] The above description highlights the consideration of both physical sound pressure level and psychoacoustic loudness growth due to the above reference Figure 1 Describes the differences in the transmission and perception of a user's own voice compared to ambient sound through different physical transmission paths for the user's own voice and ambient sound. These insights can provide considerations for the design and optimization of listening devices and audio processing algorithms to facilitate optimal perception and comfort for the user.

[0093] Figure 5B 550 illustrates an example loudness compensation for own speech and ambient sound for a hearing impaired person in accordance with various aspects of the subject technology. The disclosed operations involve restoring the loudness to a normal level, whereby it is determined that the gain required for own speech is less than the gain required for ambient sound. Figure 5B For example, graph 550 shows the different loudness compensation between the own speech and the ambient sound of the hearing impaired user, where the gain applied to the own speech (represented by line 552) will be less than the gain applied to the ambient sound (represented by line 554).

[0094] In one example, the same gain line can be applied to both the own voice and the ambient sound. However, this approach causes the own voice to be too loud for the user, causing discomfort. To address this problem and provide an improved solution, the present disclosure introduces a novel approach. By distinguishing the gain lines of the own voice from the ambient sound, restoring the loudness to a normal level can be achieved more accurately. This helps the amplified own voice to remain at an appropriate audibility level, thereby enhancing the user's comfort and satisfaction.

[0095] The described methods and systems represent an innovative advancement over the prior art by addressing the problem of excessive loudness when one's own voice is amplified at the same gain as ambient sound. Implementing the disclosed methods in a listening device and audio processing system enables the loudness to be restored to a normal level while taking into account the specific gain adjustment required for one's own voice, thereby improving the overall listening experience of the user.

[0096] Fig. 6A Graph 600 illustrates an example of dominant ambient sounds coexisting with the sound of own speech at frequency bins in accordance with various aspects of the subject technology. Graph 600 shows the sound pressure levels of own speech and ambient sounds at different frequency bins. One aspect of the operations performed by own speech adjustment circuit 410 involves estimating a bin-by-bin suppression gain for a speech signal. The bin-by-bin suppression gain estimation is based on analyzing the spectral characteristics of the speech signal in each frequency bin. The own speech adjustment circuit 410 performs a spectral analysis of the incoming speech signal to divide it into individual frequency bins, such as Fig. 6A exemplified in .

[0097] The self-speech adjustment circuit 410 may apply noise suppression to the dominant signal by attenuating the dominant signal based on determining that the dominant signal in the frequency interval in one or more frequency intervals corresponds to the ambient sound. The self-speech adjustment circuit 410 selectively reduces the amplitude or power of the noise in each frequency interval with the dominant signal characterized as the ambient sound by applying the estimated interval-by-interval suppression gain to the speech signal. In one or more specific implementations, the self-speech adjustment circuit 410 may stop attenuating the speech signal in the corresponding frequency interval when the dominant signal is characterized as the self-speech. For example, for non-dominant ambient sound signals, the self-speech adjustment circuit 410 may stop applying noise suppression to the dominant signal based on determining that the dominant signal in the frequency interval in one or more frequency intervals corresponds to the speech of the user of the electronic device 210. This processing helps enhance the clarity and intelligibility of the speech signal by minimizing the interference caused by background noise.

[0098] In one or more specific implementations, Figure 4 The own voice adjustment circuit 410 can, for example, determine the dominant signal between the ambient sound and the voice of the user of the electronic device 210 for one or more frequency intervals associated with the received audio signal based on at least partial overlap in frequency between the ambient sound and the voice of the user of the electronic device 210.

[0099] The OVPP values ​​generated by the own voice adjustment circuit 410 help determine the dominant signal between the user's own voice and the ambient sound within a given frequency interval. The hearing loss compensation circuit 412 uses this information to accurately distinguish and prioritize the relevant signals. When processing the audio signal, the hearing loss compensation circuit 412 analyzes the OVPP values ​​on a bin-by-bin basis. Each frequency bin is evaluated to assess the likelihood that the user's own voice is present compared to the ambient sound.

[0100] Based on the OVPP value of a particular frequency interval, the own voice adjustment circuit 410 determines the dominant signal. If the OVPP value indicates a high probability of the user's own voice, the own voice adjustment circuit 410 identifies the own voice as the dominant signal in the interval. Conversely, if the OVPP value implies a low probability of the user's own voice, the own voice adjustment circuit 410 identifies the ambient sound as the dominant signal.

[0101] By making this distinction, the own voice adjustment circuit 410 can adapt its processing techniques accordingly. When the user's own voice is identified as the dominant signal, the own voice adjustment circuit 410 helps the own voice be retained and appropriately amplified to maintain audibility and intelligibility. Conversely, when ambient sounds are identified as the dominant signal, the own voice adjustment circuit 410 focuses on suppressing or reducing the amplification of those sounds to enhance speech clarity and reduce background noise interference.

[0102] The use of OVPP values ​​enables the own voice adjustment circuit 410 to dynamically adjust its processing strategy to ensure that the user's own voice is accurately distinguished and prioritized within the audio signal. This enhances the overall listening experience by managing the presence of ambient sounds while maintaining the integrity of the user's own voice.

[0103] For each frequency interval, the self-speech adjustment circuit 410 calculates a suppression gain value. The gain value represents a reference Figure 6B The amount by which the ambient sound in this particular frequency interval should be attenuated is illustrated. The gain estimation takes into account the power or amplitude of the ambient sound relative to the desired own speech signal. Figure 6B A diagram 650 of example loudness compensation for frequency intervals coexisting with own voice and ambient sound according to various aspects of the subject technology is illustrated. Diagram 650 shows the gain of own voice and ambient sound applied to different frequency intervals. For the dominant signal corresponding to ambient sound, the gain of the speech signal containing the dominant unwanted signal (e.g., not the user's own voice) is attenuated (or reduced) in the corresponding frequency interval. For the dominant signal corresponding to the own voice, the gain of the speech signal containing the dominant wanted signal (e.g., the user's own voice) is maintained (or increased) in the corresponding frequency interval.

[0104] Figure 7 A flowchart illustrating an example process 700 for self-speech audio processing according to various aspects of the subject technology is shown. For purposes of explanation, this document primarily refers to Figure 2 The process 700 is described with reference to the audio processing circuit 230. However, the process 700 is not limited to Figure 2 The audio processing circuit 230 of the process 700, and one or more blocks (or operations) of the process 700 may be performed by one or more other components of other suitable devices (such as earplugs, headphones, headsets, etc.). Further for the purpose of explanation, the blocks of the process 700 are described herein as occurring sequentially or linearly. However, multiple blocks of the process 700 may occur in parallel. In addition, the blocks of the process 700 do not have to be performed in the order shown, and / or one or more blocks of the process 700 do not have to be performed and / or may be replaced by other operations.

[0105] At step 702, the device receives an audio signal corresponding to a microphone. For example, the own voice activity detector 406 receives the audio signal from the microphone 308. In another example, the noise suppression circuit 408 receives the audio signal from the microphone 308. In another example, the hearing loss compensation circuit 412 receives the audio signal from the microphone 308. In another example, the gain combiner 414 receives the audio signal from the microphone 308.

[0106] At step 704, the device detects one or more of the ambient sounds or the voice of the user of the device in the received audio signal. For example, the own voice adjustment circuit 410 detects one or more of the ambient sounds or the voice of the user of the electronic device 210 in the received audio signal. In one or more specific implementations, one or more of the ambient sounds or the voice of the user of the device is detected at least in part based on a classification indicating that the received audio signal contains the voice of the user of the device but does not contain the ambient sounds. For example, the own voice adjustment circuit 410 may detect one or more of the ambient sounds or the voice of the user of the electronic device 210 based at least in part on a classification indicating that the received audio signal contains the voice of the user of the electronic device 210 but does not contain the ambient sounds. In one or more other specific implementations, one or more of the ambient sounds or the voice of the user of the device is detected at least in part based on a classification indicating that the received audio signal contains the voice of the user of the electronic device 210 but does not contain the ambient sounds.

[0107] For example, the own voice adjustment circuit 410 may detect one or more of the ambient sound or the voice of the user of the electronic device 210 based at least in part on a classification indicating that the received audio signal contains ambient sound but does not contain the voice of the user of the electronic device 210. In one or more other specific implementations, one or more of the ambient sound or the voice of the user of the device is detected based at least in part on a classification indicating that the received audio signal contains a combination of ambient sound and the voice of the user of the device. For example, the own voice adjustment circuit 410 may detect one or more of the ambient sound or the voice of the user of the electronic device 210 based at least in part on a classification indicating that the received audio signal contains a combination of ambient sound and the voice of the user of the electronic device 210. In some specific implementations, the device detects the voice of the user of the device in the received audio signal based at least in part on one or more measurements from the accelerometer. For example, the own voice activity detector 406 may detect the voice of the user of the electronic device 210 in the received audio signal based at least in part on one or more measurements from the accelerometer 302.

[0108] In some implementations, the device may determine OVPP values ​​for one or more frequency bins associated with the received audio signal, wherein the own voice presence probability value indicates the likelihood that the voice of the user of the device is present in a frequency bin in the one or more frequency bins. For example, the own voice adjustment circuit 410 may determine the OVPP value of the received audio signal for the one or more frequency bins.

[0109] In one or more specific implementations, the device may determine the dominant signal between the ambient sound and the voice of the user of the device for one or more frequency intervals associated with the received audio signal based on at least partial overlap of the ambient sound and the voice of the user of the device in frequency. For example, the self-voice adjustment circuit 410 may determine whether the ambient sound or the self-voice is the dominant signal in the audio signal received for the corresponding frequency interval. In one or more specific implementations, the device may stop applying noise suppression to the dominant signal based on determining that the dominant signal in the frequency interval in one or more frequency intervals corresponds to the voice of the user of the device. For example, when it is determined that the dominant signal in the received audio signal corresponds to the self-voice, the self-voice adjustment circuit 410 may stop the attenuation (or suppress the attenuation) of the gain value of the audio signal received in the corresponding frequency interval. In one or more other specific implementations, the device may apply noise suppression to the dominant signal by attenuating the dominant signal based on determining that the dominant signal in the frequency interval in one or more frequency intervals corresponds to the ambient sound. For example, when it is determined that the dominant signal in the received audio signal corresponds to the ambient sound, the self-voice adjustment circuit 410 may apply attenuation (or suppression) to the gain value of the audio signal received in the corresponding frequency interval.

[0110] At step 706, when the ambient sound is detected in the received audio signal, the device applies a first gain to the ambient sound, and when the voice of the user of the device is detected in the received audio signal, the device applies a second gain different from the first gain to the voice of the user of the device. In one or more specific implementations, the device may adjust the second gain based at least in part on the probability value of the presence of the own voice. In one or more specific implementations, the first gain and the second gain correspond to different psychoacoustic loudness growth functions.

[0111] Figure 8 800 in which some aspects of the subject technology are implemented. In one or more specific implementations, the wireless communication device 800 can be a headset or earbud device of the subject technology, for example, Figure 2 , Figure 3 or Figure 4 The wireless communication device 800 may include a radio frequency (RF) antenna 810, a duplexer 812, a receiver 820, a transmitter 830, a baseband processing module 840, a memory 850, a processor 860, and a local oscillator generator (LOGEN) 870. In various aspects of the subject technology, Figure 8 One or more blocks presented in the embodiment may be integrated on one or more semiconductor substrates. For example, blocks 820 to 870 may be implemented in a single chip or a single system on a chip, or may be implemented in a chipset of multiple chips.

[0112] Receiver 820 may include suitable logic circuits and / or code that may be operable to receive and process signals from RF antenna 810. Receiver 820 may, for example, be operable to amplify and / or down-convert received wireless signals. In various aspects of the subject technology, receiver 820 may be operable to eliminate noise in received signals and may be linear over a wide range of frequencies. In this way, receiver 820 may be suitable for receiving signals according to a variety of wireless standards such as Wi-Fi, WiMAX, Bluetooth, and various cellular standards. In various aspects of the subject technology, receiver 820 may not use any sawtooth acoustic wave (SAW) filters and little or no off-chip discrete components, such as large capacitors and inductors.

[0113] Transmitter 830 may include suitable logic circuits and / or code that may be operable to process and transmit signals from RF antenna 810. Transmitter 830 may, for example, be operable to up-convert a baseband signal to an RF signal and amplify the RF signal. In various aspects of the subject technology, transmitter 830 may be operable to up-convert and amplify baseband signals processed according to a variety of wireless standards. Examples of such standards may include Wi-Fi, WiMAX, Bluetooth, and various cellular standards. In various aspects of the subject technology, transmitter 830 may be operable to provide a signal for further amplification by one or more power amplifiers.

[0114] The duplexer 812 may provide isolation in the transmit frequency band to avoid saturation of the receiver 820 or damage to components of the receiver 820 and relax one or more design requirements of the receiver 820. In addition, the duplexer 812 may attenuate noise in the receive frequency band. The duplexer 812 may be capable of operating in multiple frequency bands of various wireless standards.

[0115] The baseband processing module 840 may include suitable logic, circuitry, interfaces, and / or code that may be operable to perform baseband signal processing. The baseband processing module 840 may, for example, analyze received signals and generate control and / or feedback signals for configuring various components of the wireless communication device 800, such as the receiver 820. The baseband processing module 840 may be operable to encode, decode, transcode, modulate, demodulate, encrypt, decrypt, scramble, descramble, and / or otherwise process data according to one or more wireless standards.

[0116] The processor 860 may include suitable logic, circuitry, and / or code that may enable processing of data and / or control of the operation of the wireless communication device 800. In this regard, the processor 860 may enable providing control signals to various other components of the wireless communication device 800. The processor 860 may also control the transfer of data between various components of the wireless communication device 800. Additionally, the processor 860 may enable implementation of an operating system or otherwise execute code to manage the operation of the wireless communication device 800. In one or more implementations, the processor 860 may be configured to perform some functionality of the subject technology.

[0117] The memory 850 may include suitable logic, circuitry, and / or code that may enable storage of various types of information, such as received data, generated data, code, and / or configuration information. The memory 850 may include, for example, RAM, ROM, flash memory, and / or magnetic storage. In various aspects of the subject technology, the information stored in the memory 850 may be used to configure the receiver 820 and / or the baseband processing module 840. In some specific implementations, the memory 850 may store image information of processed and / or unprocessed fingerprint images from the under-screen fingerprint sensor of the subject technology.

[0118] LOGEN 870 may include suitable logic, circuitry, interfaces, and / or code that may be operable to generate one or more oscillating signals of one or more frequencies. LOGEN 870 may be operable to generate digital signals and / or analog signals. Thus, LOGEN 870 may be operable to generate one or more clock signals and / or sinusoidal signals. The characteristics of the oscillating signals, such as frequency and duty cycle, may be determined based on one or more control signals from, for example, processor 860 and / or baseband processing module 840.

[0119] In operation, the processor 860 may configure various components of the wireless communication device 800 based on the wireless standard according to which the signal is expected to be received. The wireless signal may be received via the RF antenna 810, amplified, and down-converted by the receiver 820. The baseband processing module 840 may perform noise estimation and / or noise cancellation, decoding, and / or demodulation of the baseband signal. In this way, the information in the received signal may be appropriately recovered and utilized. For example, the information may be audio and / or video to be presented to the user of the wireless communication device 800, data to be stored in the memory 850, and / or information that affects and / or enables the operation of the wireless communication device 800. The baseband processing module 840 may modulate, encode, and perform other processing on the audio, video, and / or control signals to be sent by the transmitter 830 according to various wireless standards.

[0120] In one or more specific implementations, the wireless communication device 800 may provide a system for training a machine learning model using training data, wherein the trained machine learning model is subsequently deployed to the wireless communication device 800. In addition, the wireless communication device 800 may provide one or more machine learning frameworks for training machine learning models and / or developing applications using such machine learning models. In one example, such machine learning frameworks may provide various machine learning algorithms and models for different problem domains in machine learning. In one example, the wireless communication device 800 may include a deployed machine learning model that provides an output of data corresponding to a prediction or some other type of machine learning output. In one or more specific implementations, training and inference operations involving individually identifiable information of a user of the wireless communication device 800 may be performed entirely on the wireless communication device 800 to prevent individually identifiable data from being exposed to devices and / or systems not authorized by the user.

[0121] In one or more implementations, the wireless communication device 800 may be connected to, or communicatively connected to, a server (not shown) that may provide a system for training a machine learning model using training data, wherein the trained machine learning model is subsequently deployed to the server and / or the wireless communication device 800. In one implementation, the server may train a given machine learning model for deployment to a client electronic device (e.g., the wireless communication device 800). In one or more implementations, the server may train a portion of the machine learning model using (e.g., anonymized) training data from a user population, and the wireless communication device 800 may train a portion of the machine learning model using individual training data from a user of the wireless communication device 800. The machine learning model deployed on the server and / or the wireless communication device 800 may then execute one or more machine learning algorithms. In one implementation, the server provides a cloud service that utilizes the trained machine learning model and / or continues to learn over time.

[0122] It is understood that the use of personally identifiable information should be subject to privacy policies and practices that are generally recognized to meet or exceed industry or government requirements for maintaining user privacy. Specifically, personally identifiable information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use, and the nature of the authorized use should be clearly stated to users.

[0123] The various functions described above may be implemented in digital electronic circuits and computer software, firmware or hardware. The technology may be implemented using one or more computer program products. A programmable processor and a computer may be included in a mobile device or packaged as a mobile device. The process and logic flow may be performed by one or more programmable processors and one or more programmable logic circuits. General and special computing devices and storage devices may be interconnected via a communication network.

[0124] Some specific implementations include electronic components such as microprocessors, storage devices, and memories that store computer program instructions in machine-readable or computer-readable media (or computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, or flash memory. Computer-readable media can store computer programs that can be executed by at least one processing unit and include instruction sets for performing various operations. Examples of computer programs or computer codes include machine code, such as machine code generated by a compiler, and files including higher-level code that can be executed by a computer, electronic component, or microprocessor using an interpreter.

[0125] Although the above discussion mainly involves microprocessors or multi-core processors that execute software, some specific implementations are performed by one or more integrated circuits such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some specific implementations, such integrated circuits execute instructions stored on the circuit itself.

[0126] As used in this specification and any claims of this patent application, the terms "computer", "processor" and "memory" all refer to electronic or other technical devices. These terms exclude people or groups of people. For the purposes of this specification, the terms "display" or "displaying" means displaying on an electronic device. As used in this specification and any claims of this patent application, the terms "computer-readable medium" and "computer-readable media" are entirely limited to tangible, touchable objects that store information in a form that can be read by a computer. These terms do not include any wireless signals, wired download signals, and any other transient signals.

[0127] To provide interaction with a user, implementations of the subject matter described in this specification may be implemented on a computer as described herein having a display device for displaying information to a user, and a keyboard and pointing device such as a mouse that a user can use to provide input to the computer. Other types of devices may also be used to provide interaction with a user; for example, feedback provided to a user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback. Input from a user may be received in any form, including acoustic, voice, or tactile input.

[0128] Many of the features and applications described above can be implemented as a software process designated as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, cores of a processor, or other processing units), these instructions cause the one or more processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media do not include carrier waves and electrical signals transmitted wirelessly or through wired connections.

[0129] In this specification, the term "software" is intended to include firmware residing in a read-only memory or applications stored in a magnetic storage device, which can be read into a memory for processing by a processor. Similarly, in some specific implementations, while retaining the different software aspects disclosed in this subject, multiple software aspects disclosed in this subject can be implemented as sub-parts of a larger program. In some specific implementations, multiple software aspects can also be implemented as independent programs. Finally, any combination of independent programs that jointly implement the software aspects described herein is within the scope of this subject disclosure. In some specific implementations, when installed to run on one or more electronic systems, the software program defines one or more specific machine implementations that execute and enforce the operations of the software program.

[0130] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages ​​and declarative or procedural languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on one computer or on multiple computers located at the same site or distributed across multiple sites and interconnected by a communications network.

[0131] It should be understood that the specific order or hierarchy of the frames in the process disclosed by the present invention is an illustration of the example method. Based on the design preference requirements, it should be understood that the specific order or hierarchy of the frames in the process can be rearranged or all the illustrated frames are executed. Some frames in these frames can be executed simultaneously. For example, in some cases, multitasking and parallel processing may be advantageous. In addition, the division of each system component in the above aspects should not be understood as requiring such division in all aspects, and it should be understood that program components and systems can be generally integrated together in a single software product or encapsulated in multiple software products.

[0132] The previous description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications of these aspects are apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but are intended to make the full scope consistent with the language claims, wherein reference to elements in singular values ​​is not intended to mean "one and only one", but to refer to "one or more", unless specifically noted. Unless otherwise specifically stated, the term "some" refers to one or more. Male pronouns (e.g., his) include female and neutral (e.g., her and its), and vice versa. Titles and subtitles (if any) are used only for convenience and do not limit the subject disclosure.

[0133] The predicate words "configured to," "operable to," and "programmed to" do not imply any specific tangible or intangible modification of a subject matter but are intended to be used interchangeably. For example, a component or a processor configured to monitor and control an operation may also mean that the processor is programmed to monitor and control the operation or that the processor is operable to monitor and control the operation. Likewise, a processor configured to execute code may be interpreted as a processor programmed to execute code or operable to execute code.

[0134] Phrases such as "aspects" do not mean that this aspect is essential to the subject technology or that this aspect applies to all configurations of the subject technology. Disclosures related to an aspect may apply to all configurations, or one or more configurations. Phrases such as aspects may refer to one or more aspects, and vice versa. Phrases such as "configurations" do not mean that this configuration is essential to the subject technology or that this configuration applies to all configurations of the subject technology. Disclosures related to configurations may apply to all configurations or one or more configurations. Phrases such as configurations may refer to one or more configurations, and vice versa.

[0135] The word “exemplary” is used herein to mean “serving as an example or illustration.” Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0136] All structural and functional equivalents of the elements throughout the various aspects described in this disclosure that are known or later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, anything disclosed herein is not intended to be provided to the public, regardless of whether the disclosure is explicitly stated in the claims. No claim element should be interpreted under the provisions of 35 U.S.C. §112(f) unless the element is explicitly stated using the phrase "a component for..." or, in the case of a method claim, the element is stated using the phrase "a step for..." In addition, the terms "including", "having", etc. are used in the specification or claims to the extent that such terms are intended to be included in a manner similar to how the term "including" is interpreted when used as a transitional word in a claim.

Claims

1. A device, comprising: microphone; and A processor, the processor being configured to: receiving an audio signal corresponding to the microphone; detecting one or more of ambient sounds or a voice of a user of the device in a received audio signal; and When the ambient sound is detected in the received audio signal, a first gain is applied to the ambient sound, and when the voice of the user of the device is detected in the received audio signal, a second gain different from the first gain is applied to the voice of the user of the device.

2. A device according to claim 1, wherein one or more of the ambient sounds or the voice of the user of the device is detected at least in part based on a classification indicating that the received audio signal contains the voice of the user of the device but does not contain the ambient sounds.

3. A device according to claim 1, wherein one or more of the ambient sounds or the voice of the user of the device is detected based at least in part on a classification indicating that a received audio signal contains the ambient sounds but does not contain the voice of the user of the device.

4. A device according to claim 1, wherein one or more of the ambient sounds or the voice of the user of the device is detected based at least in part on a classification indicating that a received audio signal contains a combination of the ambient sounds and the voice of the user of the device.

5. The device of claim 1 , further comprising an accelerometer, wherein the processor configured to detect one or more of the ambient sounds or the voice of the user of the device is further configured to detect the voice of the user of the device in a received audio signal based at least in part on one or more measurements from the accelerometer.

6. A device according to claim 1, wherein the processor configured to detect one or more of the ambient sound or the voice of the user of the device is further configured to determine a probability value of the presence of own voice for one or more frequency intervals associated with the received audio signal, the probability value of the presence of own voice indicating the possibility that the voice of the user of the device exists in a frequency interval in the one or more frequency intervals. 7 . The apparatus of claim 6 , wherein the processor configured to apply the second gain is further configured to adjust the second gain based at least in part on the own speech presence probability value.

8. A device according to claim 1, wherein the processor is further configured to determine a dominant signal between the ambient sound and the voice of the user of the device for one or more frequency intervals associated with the received audio signal based on at least partial overlap in frequency between the ambient sound and the voice of the user of the device.

9. The device of claim 8, wherein the processor is further configured to refrain from applying noise suppression to the dominant signal based on determining that the dominant signal in a frequency interval in the one or more frequency intervals corresponds to the speech of the user of the device.

10. The device of claim 8, wherein the processor is further configured to apply noise suppression to the dominant signal by attenuating the dominant signal based on determining that the dominant signal in a frequency interval of the one or more frequency intervals corresponds to the ambient sound.

11. The apparatus of claim 1, wherein the first gain and the second gain correspond to different psychoacoustic loudness growth functions.

12. The device of claim 1, wherein the second gain corresponds to a user-defined gain setting configurable via user input during a hearing profile registration process for the device.

13. A device, comprising: microphone; and A filter configured to: receiving an audio signal corresponding to the microphone; detecting ambient sounds and a voice of a user of the device in a received audio signal; determining, for one or more frequency bins associated with a received audio signal, a dominant signal between the ambient sounds and the speech of the user of the device based on at least a partial overlap in frequency between the ambient sounds and the speech of the user of the device; and When it is determined that the ambient sound is the dominant signal for the one or more frequency intervals, a first gain is applied, and when it is determined that the voice of the user of the device is the dominant signal for the one or more frequency intervals, a second gain different from the first gain is applied.

14. A device according to claim 13, wherein one or more of the ambient sounds or the voice of the user of the device is detected based at least in part on a classification indicating that the received audio signal contains the voice of the user of the device but does not contain the ambient sounds.

15. A device according to claim 13, wherein one or more of the ambient sounds or the voice of the user of the device is detected based at least in part on a classification indicating that a received audio signal contains the ambient sounds but does not contain the voice of the user of the device.

16. A device according to claim 13, wherein one or more of the ambient sounds or the voice of the user of the device is detected based at least in part on a classification indicating that a received audio signal contains a combination of the ambient sounds and the voice of the user of the device.

17. A device according to claim 13, wherein the device also includes an accelerometer, wherein the filter configured to detect one or more of the ambient sounds or the voice of the user of the device is further configured to detect the voice of the user of the device in a received audio signal based at least in part on one or more measurements from the accelerometer.

18. A device according to claim 13, wherein the filter configured to detect one or more of the ambient sound or the voice of the user of the device is further configured to determine a probability value of the presence of own voice for one or more frequency intervals associated with the received audio signal, the probability value of the presence of own voice indicating the possibility that the voice of the user of the device exists in a frequency interval in the one or more frequency intervals, and wherein the filter configured to apply the second gain is further configured to adjust the second gain based at least in part on the probability value of the presence of own voice.

19. The apparatus of claim 13, wherein the filter is further configured to: when a dominant signal in a frequency interval of the one or more frequency intervals corresponds to the speech of the user of the device, refraining from applying noise suppression to the dominant signal; and When a dominant signal in a frequency interval of the one or more frequency intervals corresponds to the ambient sound, noise suppression is applied to the dominant signal by attenuating the dominant signal.

20. A method comprising: receiving an audio signal corresponding to a microphone; detecting one or more of ambient sounds or a voice of a user of the device in the received audio signal; as well as When the ambient sound is detected in the received audio signal, a first gain is applied, and when the voice of the user of the device is detected in the received audio signal, a second gain different from the first gain is applied.

Images