Adaptive active noise control system with hearing aid mechanism

Abstract

The present disclosure provides systems and methods for adjusting the audio output of a wearable device based on a user's audio gain profile. The wearable device may receive an audiogram indicating one or more frequency ranges associated with hearing loss. The wearable device may determine the audio gain profile based on the audiogram. The wearable device may use one or more adjustment modules to determine a gain to apply to the audio output based on the audio gain profile. The adjustment modules may include an active noise control module, a hearing aid module, and a transparency control module. The wearable device may use a least mean square algorithm and / or a machine learning model to determine the amount of gain to apply.

Description

Background Art

[0001] Wearable and hearable devices can include active noise control systems that generate noise cancellation signals based on microphone input. The microphone input is filtered using a digital signal processing engine that generates sound waves. These waves are superimposed with the primary sound waves in the user's ear. This isolates and cancels ambient noise. However, active noise control generally does not provide hearing assistance for users with mild to moderate hearing loss. Summary of the Invention

[0002] The present disclosure provides systems and methods for adaptively adjusting audio output to provide a more natural listening experience to a user. A wearable device can receive an audiogram from another device, the audiogram indicating a frequency range associated with hearing loss. The wearable device can create an audio gain profile based on the audiogram. The audio gain profile can be based on one or more frequencies in the audiogram and can include information identifying whether positive or negative gain should be applied to the indicated frequencies. The wearable device can include one or more adjustment modules that can apply gain to the identified frequency range based on the audio gain profile. The use of such an adjustment module allows a wearable or hearable device, such as earbuds, to be personalized for a given user. It also allows the gain profile to be dynamically adjusted to compensate for changes in the audiogram, sounds in the surrounding environment, or changes in desired sounds.

[0003] One aspect of the present disclosure includes a wearable device comprising one or more microphones and one or more processors in communication with the one or more microphones. The one or more processors may be configured to receive an audiogram comprising a frequency range associated with a user's hearing loss; determine an audio gain profile based on the audiogram; receive audio content comprising at least one of external audio and playback audio from the one or more microphones; and adjust audio output based on the audio gain profile and the received audio content.

[0004] The audio gain profile may include a positive gain or a negative gain for at least one frequency range. The audio gain profile may include at least one positive gain for a frequency range associated with hearing loss. The audio gain profile may be based on one or more frequency ranges in an audiogram.

[0005] The one or more processors may be further configured to adjust the audio output using an adaptive hearing block. When adjusting the audio output using the adaptive hearing block, the one or more processors may be further configured to determine the gain of the frequency range using a least mean square algorithm. The adaptive hearing block may include at least one of noise cancellation, acoustic transparency control, and hearing assistance. The one or more processors may be further configured to periodically receive one or more updated audiograms and update the audio gain profile based on the one or more updated audiograms.

[0006] Another aspect of the present disclosure includes a method comprising receiving, by one or more processors, an audiogram comprising a frequency range associated with hearing loss; determining, by the one or more processors, an audio gain profile based on the audiogram; receiving audio content comprising at least one of external audio and playback audio from one or more microphones in communication with the one or more processors; and adjusting, by the one or more processors, an audio output based on the audio gain profile and the received audio content.

[0007] Yet another aspect of the present disclosure includes a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to receive an audiogram including a frequency range associated with hearing loss; determine an audio gain profile based on the audiogram; receive audio content including at least one of external audio and playback audio from one or more microphones; and adjust audio output based on the audio gain profile and the received audio content. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1A is a visual diagram of an example device according to aspects of the present disclosure.

[0009] Figure 1B is a functional block diagram of an example system according to aspects of the present disclosure.

[0010] Figure 2 is a graphical representation illustrating an example use according to aspects of the present disclosure.

[0011] Figure 3 is a graphical representation illustrating an example use according to aspects of the present disclosure.

[0012] Figure 4 is a flow chart illustrating a method of adjusting audio output according to aspects of the present disclosure. DETAILED DESCRIPTION

[0013] Wearable devices can include hearing-assistance mechanisms to provide natural frequency-related sound enhancement for users with mild to moderate hearing loss. Wearable devices can be earbuds, smart glasses, AR / VR headsets, helmets, and the like. The wearable device can receive content from a host device. For example, the wearable device can receive an audiogram indicating one or more frequency ranges associated with hearing loss. The device can also receive an audio content stream from the host device for output to the user.

[0014] The audiogram can be received by the adaptive hearing control ("AHC") block of the wearable device. Based on the audiogram, the AHC block can create an audio gain profile that indicates the frequency ranges that may require positive or negative gain to provide the user with a more natural listening experience. The audio gain profile can be used to determine adjustments to the audio output to compensate for mild to moderate hearing loss. The audio gain profile can indicate a specific amount of gain for a given frequency range in which the user experiences hearing loss. The gain for a given frequency range can compensate for the hearing loss. In some examples, each ear may have a different audio gain profile. For example, the right ear and the left ear may experience different or different degrees of hearing loss. The right ear may have moderate hearing loss, while the left ear may have mild hearing loss. In some examples, the frequency range in which the audiogram indicates hearing loss may be different for each ear. Therefore, the audio gain profile for the right ear may be different from that for the left ear. In other examples, using a host device (eg, a smartphone) and a wearable device (eg, earbuds), an audiogram may be periodically generated and used to adjust a gain profile for the wearer of the wearable device.

[0015] The AHC block may include an active noise control system ("ANC"), a hearing aid control mechanism ("HAC"), and a transparency control system ("XPC"). The AHC may automatically adapt or adjust the audio output based on input received by one or more microphones and an audio gain profile. The AHC block may adjust the audio output based on a least mean square algorithm. In some examples, the least mean square algorithm may determine the amount of gain that should be applied by each adjustment module. Additionally or alternatively, the AHC block may adjust the audio output based on a machine learning model.

[0016] Figure 1AAn example system 100A is described in which the features described herein may be implemented. This should not be considered to limit the scope of the present disclosure or the usefulness of the features described herein. In this example, the system 100A may include a host device 120 and a wearable device 110. The host device 120 may be a smartphone and the wearable device 110 may be a pair of earbuds. The host device 120 may be any device or accessory, such as a smartphone, a mobile phone, a wireless-enabled PDA, a tablet computer, a netbook capable of obtaining information via the Internet or other network, a wearable computing device (e.g., a smartwatch, headphones, smart glasses, a virtual reality player, other head-mounted displays, etc.), a wireless speaker, a home assistant, a game console, etc.

[0017] Host device 120 can wirelessly connect 130 to wearable device 110. For example, host device 120 and wearable device 110 can be coupled via short-range communication, such as Bluetooth, Bluetooth Low Energy (BLE), or the like. Wearable device 110 can receive content from host device 120 via wireless connection 130. The content can be a user-specific audiogram 132. The content can also include music, speech, or content from an audio source. Audiogram 132 can indicate certain frequencies that a user may not be able to hear with the same intensity as other frequencies. Audiogram 132 illustrates example intensities measured in decibels across a frequency range. Audiogram 132 can include data related to each ear of the user, where one ear may have greater hearing loss than the other. According to some examples, the content can be audio content to be output by wearable device 110. For example, host device 120 can stream audio content. The audio content can be transmitted to wearable device 110 via wireless connection 130. Wearable device 110 can output the audio content for the user to hear.

[0018] Figure 1B An example system 100B is illustrated in which the features described above and herein can be implemented. In this example, system 100B can include a wearable device 110 and a host device 120. Wearable device 110 can include one or more processors 111, memory 112, instructions 113, data 114, one or more microphones 115, a wireless communication interface or antenna 116, and an adaptive hearing control ("AHC") block 117.

[0019] The one or more processors 111 may be any conventional processor, such as a commercially available microprocessor. Alternatively, the one or more processors may be a dedicated device, such as an application specific integrated circuit (ASIC) or other hardware-based processor. Figure 1BThe processor, memory, and other elements of the wearable device 110 are functionally described as being within the same block, but one of ordinary skill in the art will understand that the processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical housing. Similarly, the memory may be a hard drive or other storage medium located in a housing different from the housing of the wearable device 110. Therefore, references to a processor or computing device will be understood to include references to a collection of processors, computing devices, or memories that may or may not operate in parallel.

[0020] The memory 112 can store information that can be accessed by the processor, including instructions 113 and data 114 that can be executed by the processor 111. The memory 112 can be a type of memory for storing information that can be accessed by the processor 111, including non-transitory computer-readable media, or other media that stores data that can be read by an electronic device, such as a hard drive, a memory card, a read-only memory ("ROM"), a random access memory ("RAM"), an optical disk, and other writable and read-only memories. The subject matter disclosed herein can include different combinations of the foregoing, whereby different portions of the instructions 113 and data 114 are stored on different types of media.

[0021] Memory 112 can be retrieved, stored, or modified by processor 111 according to instructions 113. For example, although the present disclosure is not limited to a particular data structure, data 114 can be stored in a computer register, a relational database, as a table with multiple different fields and records, an XML document, or a flat file. Data 114 can also be formatted in a computer readable format, such as, but not limited to, binary values, ASCII, or Unicode. By way of further example only, data 114 can be stored as a bitmap comprising pixels stored in compressed or uncompressed or various image formats (e.g., JPEG), vector-based formats (e.g., SVG), or computer instructions for drawing graphics. In addition, data 114 can include information sufficient to identify relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations), or information used by functions to calculate relevant data.

[0022] Instructions 113 may be any set of instructions executed directly by processor 111 (e.g., machine code), or indirectly executed (e.g., a script). In this regard, the terms "instructions," "application," "steps," and "program" are used interchangeably herein. Instructions may be stored in object code format for direct processing by the processor, or in any other computing device language, including scripts or collections of independent source code modules that are interpreted on demand or pre-compiled. The functions, methods, and routines of the instructions are explained in more detail below.

[0023] Wearable device 110 may include one or more microphones 115. A microphone 118 of the wearable device may be located on an exposed surface of the housing of wearable device 110 when wearable device 110 is worn on the body. In some examples, microphone 115 of wearable device 110 may be located on a surface of the housing of wearable device 110 that contacts the body when wearable device 110 is worn on the body. The microphones are capable of receiving audio input. The AHC block may process the audio input based on the user's audio gain profile.

[0024] Wearable device 110 may also include a wireless communication interface 116, such as an antenna, a transceiver, and any other device for wireless communication. For example, the antenna may be a short-range wireless network antenna. Wearable device 110 can be coupled to host device 120 via a wireless connection. For example, wireless communication interface 116 may be used to transmit and receive Bluetooth signals, WiFi signals, or signals using other short-range wireless technologies.

[0025] The wearable device 110 may include an AHC block 117. The AHC 117 block may receive content, such as an audiogram, from the host device 120. The audiogram may indicate certain frequency ranges associated with hearing loss. The AHC block 117 may create an audio gain profile for the user based on the audiogram. The audio gain profile may indicate frequency ranges to which positive and / or negative gains may be applied to provide the user with gain-adjusted sound, which will provide a better listening experience. For example, for frequency ranges associated with hearing loss, positive gains may be applied to increase or enhance the intensity of the audio in those frequency ranges. In frequency ranges not associated with hearing loss, or outside the normal expected human hearing range, negative gains may be applied to reduce and / or lower the audio in those frequency ranges.

[0026] AHC block 117 can receive input from microphone 115. For example, a first microphone can face outward or be located on a surface that is exposed when wearable device 110 is on the user's body. The outward-facing microphone can receive ambient or external noise as audio input. External noise can include noises occurring around the user, such as traffic, construction, machinery operation, or faint chatter. AHC block 117 can receive input from a second microphone, located on a surface that is configured to contact the user's body when wearable device 110 is on the user's body. The second microphone can receive audio output by the wearable device. For example, the second microphone can receive input from content being output to the user by wearable device 110. In the example where the wearable device is a pair of earbuds, the second microphone 115 can be located on a surface inside the user's ear when the earbuds are on the body. There may be propagation paths around the earbuds. Based on the audio input received by each microphone 115 and the propagation paths between microphones 115, AHC block 117 can determine what noise is ambient noise, what noise is intended to be output to the user, and so on. The AHC block 117 may adjust the audio output based on the audio gain profile and the received audio.In some examples, the AHC block 117 may output a set of noise control benefits to modify the way the user perceives the audio.

[0027] The AHC block 117 may use one or more adjustment modules to adjust the audio output. For example, the adjustment modules may include an active noise control system ("ANC"), transparency control ("XPC"), and / or hearing aid control ("HAC"). These modules may include instructions executable by the processor 111 or may be implemented as an integrated circuit, where the integrated circuit is, for example, an ASIC, a DSP resident on the processor 111, or part of a circuit resident on the processor 111. In some examples, the AHC block 117 may receive an audio input received by the microphone 115. The AHC block 117 may process the audio input based on an audio gain profile. Adjusting the audio output may include applying a positive or negative gain to a specific frequency range. Each adjustment module may apply a different gain. For example, the HAC module may apply a positive gain, while the ANC module applies a negative gain. The amount of gain for each module may be different, and in some examples, the amount of gain may depend on the frequency of the audio. The amount of gain may be determined using a least mean square algorithm and / or a machine learning model.

[0028] The AHC block 117 may include an ANC module to apply ANC gain to the audio output. The ANC module may modify the output of certain frequencies based on the user's audiogram. For example, the ANC module can isolate the audio output from ambient or background noise. The ambient or background noise may be traffic, weather-related noise (such as wind and thunder), indistinct background noise, the sound of an air conditioner or heater, etc. Isolating the audio output may include generating noise that is opposite to the ambient or background noise, thereby eliminating or significantly reducing the ambient or background noise. According to some examples, ANC can digitally eliminate low-frequency audio.

[0029] The AHC block 117, and therefore the ANC module, can receive audio input from one or more microphones. The audio input can be ambient noise, background noise, etc. A sound estimate can be determined based on the received audio. The sound estimate can include the intensity, frequency, etc. of the received audio. Based on the sound estimate, the AHC block 117 can determine the gain to be applied so that the audio output provides a natural enhancement of the ambient sound. In some examples, the ANC module can generate anti-noise or anti-sound corresponding to the ambient or background noise received by the one or more microphones. The anti-noise can have a sound wave or frequency that is opposite to the undesirable sound wave or frequency of the audio input. The anti-noise sound wave can cancel the sound wave of the audio input received by the one or more microphones.

[0030] The AHC block 117 may include an XPC module to apply an XPC gain to the audio output. XPC can allow the user to maintain a comfortable playback volume while still being able to hear ambient sounds. As described above and herein, the ANC module can reduce or eliminate most or all background or ambient noise. In order to provide the user with a more natural listening experience, the AHC module can modify or adapt the audio output using the XPC module to provide the ambient noise to the user as part of the audio output. That is, although the ANC module can eliminate ambient noise, the XPC module can apply a gain to the received ambient noise so that the audio output is the same or similar to the ambient noise heard by the user who is not wearing the wearable device 110. The gain applied by the XPC module can be based on an audiogram. For example, the gain can be determined based on certain frequency ranges associated with hearing loss so that the user can hear the ambient noise as part of the audio output. This can provide the user with a listening experience as if the user is not wearing the wearable device 110.

[0031] The AHC block 117 may include an HAC module. The HAC may apply HAC gain to the audio input to modify how the user perceives the audio output. The HAC module may apply positive gain to increase the intensity of certain frequencies or frequency ranges. For example, an audio gain profile may indicate that the user has hearing loss within a certain frequency range. The HAC module may apply gain to the audio within the identified frequency range. The amount of gain may be based on the severity of the hearing loss. By applying positive gain, the intensity or playback volume may be increased. Including the HAC module may provide a better user experience for users with mild to moderate hearing loss.

[0032] Wearable device 110 can perform passive noise control ("PNC") based on the materials used to manufacture wearable device 110. For example, the materials of wearable device 110 can block ambient or external noise from being received by the user. In an example where wearable device 110 is a pair of earbuds, the earpieces worn within the user's ears can block or prevent sound from entering the ears. According to some examples, PNC can physically isolate high-frequency audio input.

[0033] According to some examples, the AHC block 117 can apply a least mean square algorithm (LMS algorithm) to determine how to adjust the audio output using each adjustment module based on the audiogram. The LMS algorithm can use the external microphone input and the processed output from the audiogram module to generate the desired filter gain response for the AHC block.

[0034] In some examples, the AHC block 117 can use a machine learning model to determine how to adjust the audio output using each adjustment module based on the audiogram. The machine learning model can be trained to determine the amount of gain applied to each adjustment module. Each training example can consist of an audio output provided to the user. The input features of the machine learning model can be an audiogram, a volume command received by the device to increase or decrease the playback volume, environmental or background noise, etc. The machine learning model can use the input features to more accurately determine the amount of gain to be applied by each adjustment module. The output of the machine learning model can be the amount of gain to be applied by each adjustment module of the AHC block 117. In some examples, the device can request feedback from the user. For example, the user may be asked whether the background noise is too loud or whether the audio output is too quiet. The user can provide feedback (e.g., yes or no), which indicates that the applied gain is suitable for the user's listening preferences.

[0035] In some examples, host device 120 and wearable device 110 can be used to create an audiogram. Host device 110 can periodically prompt a user to participate in an audiogram test. While the user is wearing wearable device 110, host device 120 can perform the hearing test. The user can provide feedback during the test to create a new or updated audiogram. In some examples, a machine learning model can use the new or updated audiogram as input to modify the audio gain profile. The updated audiogram can be used to dynamically adjust the audio gain profile. This can provide continuous updates to the audio gain profile so that the audio gain applied by AHC block 127 is based on the most recent audiogram.

[0036] Host devices 120 may each include one or more processors 121 , memory 122 , instructions 123 , data 124 , microphone 125 , wireless communication interface 126 , and AHC block 127 , which are substantially similar to those components described herein with respect to wearable device 110 .

[0037] Figure 2 A graphical representation of example audio gains applied by an AHC block based on an audio gain profile is illustrated. Chart 200 illustrates gains that may be applied to the audio output by the adjustment module based on the audio gain profile. In some examples, the AHC block may adjust the audio output based on parameters determined by a least mean square algorithm. For example, the least mean square algorithm may determine that the HAC module requires greater granularity within a certain frequency range than the ANC module within the same and / or different frequency ranges. The AHC block may apply positive and negative gains to certain frequency ranges using the ANC, total noise cancellation ("TNC"), XPC, and HAC modules.

[0038] In the example shown in chart 200, the audiogram may indicate that the user has mild to moderate hearing loss for frequencies between approximately 3,000 Hz (hertz) and 8,000 Hz. The audio gain profile may indicate that HAC may apply positive gain to frequencies between approximately 3,000 Hz and 8,000 Hz, while ANC applies negative gain in other frequency ranges. This may allow the user to better hear audio output in frequencies between 3,000 Hz and 8,000 Hz, while blocking other or unwanted audio in frequencies outside the 3,000 Hz to 8,000 Hz range.

[0039] The ANC module can apply negative gain to frequencies other than those in the identified hearing loss range. According to some examples, ANC can be used to adjust the audio output heard by the user in a frequency range in which the user has no hearing loss. As shown in chart 200, the ANC can apply negative gain to frequencies between approximately 50 Hz and approximately 3,000 Hz. For example, as the frequency of the audio increases from approximately 50 Hz to approximately 100 Hz, the ANC can gradually apply increasing negative gain to eliminate the audio at those given frequencies. For example, audio at a frequency of approximately 75 Hz may require a gain of -20 dB to eliminate or cancel the audio. Audio at a frequency of approximately 90 Hz may require a gain of approximately -30 dB to eliminate or cancel the audio at that frequency.

[0040] Between approximately 100 Hz and 3,000 Hz, the negative gain applied by the ANC may gradually decrease. For example, as the audio frequency approaches 3,000 Hz, the gain applied by the ANC may decrease from approximately -30 dB to approximately 0 dB.

[0041] As shown in graph 200, when the gain applied by the ANC module approaches 0 dB at approximately 3,000 Hz, the HAC module may apply a maximum gain of approximately 20 dB in examples where the user has mild hearing loss and a maximum gain of approximately 60 dB in examples where the user has moderate hearing loss. According to the example shown in graph 200, the HAC module may apply positive gain for frequencies between approximately 3,000 Hz and 8,000 Hz, or frequencies associated with the user's hearing loss. The amount of gain applied may depend on the severity of the hearing loss. The severity of the hearing loss may be part of an audio gain profile based on an audiogram.

[0042] The PNC module can passively cancel noise due to the material and / or shape of the wearable device. In an example where the wearable device is a pair of earbuds, the portion of the earbud that is inserted into the user's ear can form a seal or tight fit between the earbud and the user's ear. This can prevent external or ambient noise from reaching the user's auditory system. Depending on the material of the wearable device, the PNC module may work best at certain frequencies. As shown, the PNC module can cancel audio with frequencies between approximately 1,000 Hz and higher. According to some examples, the PNC module may come out of the seal of the device when the user turns the device off.

[0043] The XPC module can apply positive or negative gain to amplify or reduce the intensity of ambient or background noise. This can allow the user to maintain a comfortable audio playback volume while still being able to hear ambient and / or background sounds. As shown in chart 200, the XPC module may not apply positive or negative gain until the audio reaches a frequency of approximately 10,000 Hz. For example, the XPC module may not apply gain to the audio received by the one or more microphones so that the audio output of the XPC module is the same as or similar to the audio output that the user would hear without wearing the wearable device.

[0044] The TNC module can apply negative gain across one or more frequency ranges. As shown in chart 200, the TNC module can apply negative gain to frequencies greater than 50 Hz. The maximum negative gain applied by the TNC module can be in a frequency range associated with hearing loss. In this example, the maximum negative gain applied by the TNC module can be for audio with frequencies between 3,000 Hz and 10,000 Hz. According to some examples, the TNC module can be a combination of a PNC module and an ANC module. For example, when the device is switched to ANC mode, the noise cancellation benefit can be total noise cancellation.

[0045] The AHC block can apply gains determined by one or more of the ANC, HAC, XPC, PNC, and TNC modules. In some examples, as described above, the amount of gain applied by the AHC block can be based on a least mean square algorithm. Additionally or alternatively, as also described above, the amount of gain applied to adjust the audio output can be based on a machine learning model.

[0046] Figure 3 A graphical representation of example audio gains applied by the AHC block based on another audio gain profile is illustrated. The audio gain of each adjustment module can be based on an audio gain profile created for a user based on a received audiogram. For example, the audio gain profile may indicate that the user has mild to moderate hearing loss for frequencies above 2,000 Hz. As shown in chart 300, the AHC block can apply positive gains to audio frequencies above 2,000 Hz. The amount of gain can be determined by the AHC block using a least mean square algorithm and / or a machine learning model.

[0047] The ANC module can apply negative gain to audio with a frequency below 2,000 Hz. In some examples, negative gain is applied to eliminate or reduce audio in certain frequency ranges. In this example, negative gain is applied to eliminate or reduce audio with a frequency below 2,000 Hz. The amount of gain can vary based on the audio and / or frequency. For example, background or ambient noise is more likely to be eliminated or reduced than playback audio or audio that the user is outputting through the wearable device. As shown in chart 300, the ANC module can apply a negative gain of approximately 15 Hz to audio with a frequency of approximately 100 Hz. In contrast, the ANC module can apply a negative gain of approximately 2 Hz to audio with a frequency of approximately 1,000 Hz. Thus, the amount of gain can vary based on the frequency of the audio.

[0048] The XPC module can apply gain to audio received by the external microphone according to the audio gain profile. For example, the XPC module can apply positive gain to audio input received by the external microphone with a frequency less than 100 Hz. This can increase or enhance ambient or background audio, allowing the user to hear audio that the user would not normally be able to hear. As shown in chart 300, the XPC module can apply positive gain to audio input received by the external microphone with a frequency greater than 10,000 Hz.

[0049] The HAC module can apply a gain to the audio received by the one or more microphones according to the audio gain profile. For example, the audio gain profile can indicate or identify a frequency range associated with hearing loss. As shown in chart 300, the frequency range can be 2,000 Hz and 10,000 Hz. The HAC module can apply a positive gain to increase or amplify audio with frequencies between 2,000 Hz and 10,000 Hz. The positive gain can amplify the audio output in the frequency range associated with hearing loss to ensure that the user can hear the audio output in the frequency range.

[0050] The TNC module can apply gain to the audio received by the one or more microphones according to the audio gain profile. For example, the TNC module can apply negative gain to one or more frequency ranges. The TNC module can apply the maximum negative gain in the frequency range associated with hearing loss.

[0051] Graph 300 may include a target equalization ("EQ") for the HAC module. The target EQ for the HAC module may be a target gain that will be applied by the HAC module within a frequency range associated with hearing loss. For example, the target EQ may be a smooth curve, such as a convex curve, that starts at or near the low frequencies of the frequency range and ends at or near the high frequencies of the frequency range. As shown, the curve may be at or near approximately 2,000 Hz and end at or near approximately 10,000 Hz. The curve may start at or near 2,000 Hz and form a smooth upward curve, indicating that the applied gain gradually increases until the gain reaches a target maximum gain. After reaching the target maximum gain, the curve may form a smooth downward curve, indicating that the applied gain gradually decreases until the gain reaches 0 dB. Although the target EQ shows an example of what the target gain for the HAC module may be, the target is merely exemplary. The HAC module may apply a gain that is greater or less than the gain shown by the target EQ.

[0052] Figure 4 An example method for adjusting the audio output of a wearable device based on an audio gain profile is described. The following operations do not have to be performed in the exact order described below. Instead, the various operations can be processed in a different order or simultaneously, and operations can be added or omitted.

[0053] For example, at block 410, the wearable device may receive an audiogram that includes a frequency range associated with hearing loss. The audiogram may include the frequency range associated with hearing loss. The wearable device may be earbuds, smart glasses, an AR / VR headset, etc. The wearable device may be wirelessly connected to a host device. The host device may be, for example, a smartphone, a tablet, a laptop, etc. In some examples, the wearable device may receive audio content from the host device. The audio content may be output by the wearable device for the user to hear.

[0054] The wearable device may determine an audio gain profile based on the audiogram at block 420. The audio gain profile may be based on one or more frequencies in the audiogram and may include information identifying whether positive or negative gain should be applied to the indicated frequencies.

[0055] At block 430, the wearable device may receive audio from one or more microphones. The audio content may include external audio and / or playback audio. External audio may be audio received by a microphone on an external surface, or an outward-facing surface, of the wearable device. An external surface, or an outward-facing surface, may be a surface that is not configured to contact the body when the wearable device is worn on the body. Playback audio may be audio being output by the wearable device. For example, a host device may stream music to the wearable device. The wearable device may output the music for the user to hear. An internal microphone, or an internal microphone, may receive playback audio. An internal microphone, or an internal microphone, may be a microphone on a surface of the wearable device that is configured to contact the body when the wearable device is worn on the body.

[0056] At block 440, the wearable device may adjust the audio output based on the audio gain profile and the received audio content. The device may use one or more adjustment modules to adjust the audio output. For example, the wearable device may include an AHC block that includes ANC, HAC, TNC, PNC, and XPC modules. Each module may determine a gain to apply to the audio output based on the audio gain profile. For example, the ANC module may apply gain to reduce or eliminate external or ambient noise, while the HAC module may apply gain to increase audio in a frequency range associated with hearing loss.

[0057] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be made by way of illustration rather than by way of limitation of the subject matter defined by the claims. In addition, the examples described herein and clauses expressed with phrases such as "for example," "including," etc. should not be construed as limiting the subject matter of the claims to specific examples; rather, the examples are intended to illustrate only one of many possible embodiments. Furthermore, the same reference numerals in different figures may identify the same or similar elements.

Claims

1. A wearable device comprising: one or more microphones; as well as one or more processors in communication with the one or more microphones, the one or more processors being configured to: receiving an audiogram including a frequency range associated with a hearing loss of a user; determining an audio gain profile based on the audiogram; receiving audio content from the one or more microphones, the audio content comprising at least one of external audio and playback audio; as well as Adjusting the audio output based on the audio gain profile and the received audio content, wherein adjusting the audio output includes applying a positive gain or a negative gain to at least one frequency range, wherein the one or more processors are further configured to use an adaptive hearing block to adjust the audio output, wherein the adaptive hearing block includes the following modules as adjustment modules: an active noise cancellation system ANC module, an acoustic transparency control system XPC module, and a hearing aid control mechanism HAC module, wherein the ANC module generates noise opposite to the external audio so as to cancel or reduce the external audio, wherein the XPC module applies a gain to the external audio so that the audio output is the same or similar to the external audio when the wearable device is not worn, and wherein the HAC module applies a positive gain to increase the intensity of the frequency range associated with the hearing loss, wherein the adaptive hearing block applies a least mean square algorithm or a machine learning model to determine the amount of gain that should be applied to the audio output by each adjustment module based on the audio gain profile and the received audio content, wherein the machine learning model is trained to determine the amount of gain to be applied for each adjustment module.

2. The wearable device according to claim 1, wherein: The audio gain profile includes a positive gain or a negative gain for at least one frequency range.

3. The wearable device according to claim 1, wherein: The audio gain profile includes at least one positive gain for the frequency range associated with the hearing loss of the user.

4. The wearable device according to claim 1, wherein: The audio gain profile is based on one or more frequency ranges in the audiogram.

5. The wearable device according to claim 1, wherein: The one or more microphones include a first microphone located on a surface exposed when the wearable device is on a user's body, and wherein the least mean squares algorithm uses input from the first microphone and processed output from the audiogram module to determine an amount of gain that should be applied by each adjustment module.

6. The wearable device according to claim 1, wherein: The machine learning model determines an amount of gain to be applied by each adjustment module using input features including the audiogram, volume commands received by the wearable device to increase or decrease playback volume, and the external audio.

7. The wearable device according to claim 1, wherein: The one or more processors are further configured to: receiving one or more updated audiograms periodically; and The audio gain profile is updated based on the one or more updated audiograms.

8. A method for adjusting audio output, comprising: receiving, by one or more processors, an audiogram including a frequency range associated with a hearing loss of a user; determining, by the one or more processors, an audio gain profile based on the audiogram; receiving audio content from one or more microphones in communication with the one or more processors, the audio content comprising at least one of external audio and playback audio; as well as The one or more processors adjust the audio output based on the audio gain profile and the received audio content, wherein adjusting the audio output includes applying a positive gain or a negative gain to at least one frequency range, wherein adjusting the audio output includes using an adaptive hearing block, wherein the adaptive hearing block includes the following modules as adjustment modules: an active noise cancellation system ANC module, an acoustic transparency control system XPC module, and a hearing aid control mechanism HAC module, wherein the ANC module generates noise opposite to the external audio so as to cancel or reduce the external audio, wherein the XPC module applies gain to the external audio so that the audio output is the same or similar to the external audio when the wearable device is not worn, and wherein the HAC module applies a positive gain to increase the intensity of the frequency range associated with the hearing loss, wherein the adaptive hearing block applies a least mean square algorithm or a machine learning model to determine the amount of gain that should be applied to the audio output by each adjustment module based on the audio gain profile and the received audio content, wherein the machine learning model is trained to determine the amount of gain to be applied for each adjustment module.

9. The method according to claim 8, wherein The audio gain profile includes a positive gain or a negative gain for at least one frequency range.

10. The method according to claim 8, wherein The audio gain profile includes at least one positive gain for the frequency range associated with the hearing loss of the user.

11. The method according to claim 8, wherein The audio gain profile is based on one or more frequency ranges in the audiogram.

12. The method according to claim 8, wherein The least mean square algorithm uses the external microphone input and the processed output from the audiogram module to determine the amount of gain that should be applied by each adjustment module.

13. The method according to claim 8, wherein The machine learning model determines an amount of gain to be applied by each of the adjustment modules using input features including the audiogram, volume commands received by the wearable device to increase or decrease playback volume, and the external audio.

14. The method according to claim 8, further comprising: receiving, by the one or more processors, one or more updated audiograms; as well as The audio gain profile is updated, by the one or more processors, based on the one or more updated audiograms.

15. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to: receiving an audiogram including a frequency range associated with a hearing loss of a user; determining an audio gain profile based on the audiogram; receiving audio content from one or more microphones, the audio content comprising at least one of external audio and playback audio; as well as Adjusting the audio output based on the audio gain profile and the received audio content, wherein adjusting the audio output includes applying a positive gain or a negative gain to at least one frequency range, wherein adjusting the audio output includes using an adaptive hearing block, wherein the adaptive hearing block includes the following modules as adjustment modules: an active noise cancellation system ANC module, an acoustic transparency control system XPC module, and a hearing aid control mechanism HAC module, wherein the ANC module generates noise opposite to the external audio so as to cancel or reduce the external audio, wherein the XPC module applies gain to the external audio so that the audio output is the same or similar to the external audio when the wearable device is not worn, and wherein the HAC module applies a positive gain to increase the intensity of the frequency range associated with the hearing loss, wherein the adaptive hearing block applies a least mean square algorithm or a machine learning model to determine the amount of gain that should be applied to the audio output by each adjustment module based on the audio gain profile and the received audio content, wherein the machine learning model is trained to determine the amount of gain to be applied for each adjustment module.

16. The non-transitory computer-readable medium of claim 15, wherein: The audio gain profile includes a positive gain or a negative gain for at least one frequency range.

17. The non-transitory computer-readable medium of claim 15, wherein: The audio gain profile includes at least one positive gain for the frequency range associated with the hearing loss of the user.

18. The non-transitory computer-readable medium of claim 15, wherein: The audio gain profile is based on one or more frequency ranges in the audiogram.

19. The non-transitory computer-readable medium of claim 15, wherein: The least mean square algorithm uses the external microphone input and the processed output from the audiogram module to determine the amount of gain that should be applied by each adjustment module.

20. The non-transitory computer-readable medium of claim 15, wherein: The machine learning model determines an amount of gain to be applied by each of the adjustment modules using input features including the audiogram, volume commands received by the wearable device to increase or decrease playback volume, and the external audio.

Images