Dynamic Seal Testing and Feedback for Audio Wearable Devices

Abstract

The present disclosure provides processes, methods, systems, and devices for providing feedback of a wearable device to a user. The feedback may indicate a level of seal, fit, or compatibility between the wearable device and the user. For example, the feedback provides an accurate measurement and a quantified report of how well a seal is created when the user wears the wearable device, which indicates the audio playback performance of the wearable device experienced by the user. This allows the user to identify the best adjustment or selection of components of the wearable device (e.g., tips, inserts, cups, etc.) to deliver the best audio performance achievable by the wearable device, as well as comfort levels. The wearable device may include earphones, headphones, headsets, or any audio device that is in physical contact with the user.

Description

[Technical Field]

[0001] This application claims priority to and the benefit of U.S. Patent Application No. 17 / 326,839, filed May 21, 2021, the entire contents of which are incorporated herein by reference as if fully set forth below.

[0002] Aspects of the present disclosure generally relate to audio reproduction performance. [Background technology]

[0003] To enjoy audio anywhere, listeners desire miniaturized, high-fidelity, responsive speakers with noise-canceling capabilities. To fully deliver the audio performance as designed, such speakers are often inserted into the listener's ear (e.g., in the case of earphones) or completely cover the listener's ear (e.g., in the case of headphones). To accommodate different sizes and shapes of listeners' ears, accessories such as tips, inserts, cups, or other sealing components are provided to allow listeners to identify the most comfortable fit. However, in some cases, selecting accessories based on comfort level may not result in the best audio performance achievable with the speaker. For example, listeners may prefer a loose fit to avoid pressure on the ear, but such a loose fit may degrade speaker performance. Currently, there are few quantifiable measurements available to help listeners make informed decisions regarding the trade-off between comfort and performance, identify the best accessories to achieve both comfort and performance, or identify audio performance through accurate measurements. Therefore, what is desired are methods for testing and receiving feedback to inform listeners of achievable audio performance, as well as devices and systems configured to implement these methods. Summary of the Invention [Means for solving the problem]

[0004] All examples and features mentioned in this specification can be combined in any technically possible manner.

[0005] Aspects of the present disclosure provide a method for providing feedback to a user of a wearable device. The method generally includes playing an audio signal through a speaker on the wearable device. The method further includes measuring audio data associated with the audio signal using a microphone on the wearable device. The wearable device is configured to be worn by a user such that the microphone shares a cavity with the user's ear canal. The method includes providing feedback to the user regarding the seal quality of an interface of the wearable device with at least a portion of the user's head. The feedback is provided continuously: i) while the wearable device is moved relative to the user; ii) while the audio signal played through the speaker is changed; or iii) while the wearable device is moved relative to the user and the audio signal played through the speaker is changed.

[0006] In an aspect, providing feedback includes providing a visual response.

[0007] In an aspect, providing feedback includes providing an audio response via a speaker on the wearable device.

[0008] In aspects, the placement of the flexible coupling element relative to the ear includes at least a position or orientation of the flexible coupling element relative to at least one of the user's ears. In some cases, the speaker and internal microphone on the wearable device are positioned inside the user's ear. The audio data measured by the internal microphone may include a low-frequency response indicative of a level of seal between the speaker and the user's ear, the low-frequency response comprising at least one of an amplitude response or a phase response.

[0009] In some cases, the method further includes indicating a fit quality indicator that indicates when a level of seal between the speaker and the user's ear is within a calibrated range. In some cases, the method further includes actively canceling ambient noise via the speaker when the fit quality indicator is above a threshold.

[0010] In an aspect, the method further includes generating an audio signal based on a profile of frequency variation to evoke a low frequency response.

[0011] Aspects of the present disclosure provide a wearable device configured to be worn by a user such that a microphone of the wearable device shares a cavity with the user's ear canal. The wearable device includes at least one speaker configured to play audio signals to the user. The wearable device includes a microphone adjacent to the at least one speaker. The microphone is configured to measure audio data associated with the audio signals played by the at least one speaker. The wearable device further includes a processor configured to process the audio data to dynamically determine, in a closed loop, feedback regarding the seal quality of an interface of the wearable device with at least one of the user's ears. The feedback is provided continuously: i) while the wearable device is moved relative to the user; ii) while the audio signal played through the speaker is changed; or iii) while the wearable device is moved relative to the user and the audio signal played through the speaker is changed. The processor is further configured to output the feedback to the user.

[0012] In aspects, the wearable device further includes a visual indicator configured to provide a visual display of feedback to the user.

[0013] In an aspect, the feedback is at least reproduced by at least one speaker.

[0014] In aspects, the placement of the wearable device includes at least a position or orientation of the flexible coupling element relative to at least one of the user's ears.

[0015] In aspects, the speaker and internal microphone on the wearable device are positioned inside the user's ear. In some cases, the audio data measured by the internal microphone includes a low-frequency response indicative of a level of seal between the speaker and the user's ear, the low-frequency response comprising at least one of an amplitude response or a phase response. In some cases, the feedback includes a fit quality indicator that indicates when the level of seal between the speaker and the user's ear is within a calibrated range.

[0016] In an aspect, the at least one speaker is further configured to actively cancel ambient noise via the speaker when the fit quality indicator is above a threshold.

[0017] In an aspect, the processor is further configured to generate an audio signal based on a profile of frequency variation to evoke a low frequency response.

[0018] An aspect of the present disclosure provides a system including a wearable device and a playback device. The wearable device includes at least one speaker configured to play audio signals to a user. The wearable device includes a microphone adjacent to the at least one speaker. The microphone is configured to measure audio data associated with the audio signals played by the at least one speaker. The wearable device further includes a processor configured to process the audio data to dynamically determine, in a closed loop, feedback regarding the seal quality of an interface of the wearable device with at least one of the user's ears. The feedback is provided continuously: i) while the wearable device is moved relative to the user; ii) while the audio signal played through the speaker is changed; or iii) while the wearable device is moved relative to the user and the audio signal played through the speaker is changed. The processor is further configured to output the feedback to the user. The playback device is configured to communicate with the wearable device and receive the feedback.

[0019] In aspects, the playback device transmits the audio signal to the wearable device. Two or more features described in this disclosure, including features described in this Summary section, may be combined to form implementations not specifically described herein.

[0020] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. [Brief explanation of the drawings]

[0021] [Figure 1] 1 illustrates an example of a system in which aspects of the present disclosure may be implemented. [Figure 2] 1A and 1B show cross-sectional views illustrating a seal between a speaker and an ear canal according to some aspects of the present disclosure. [Figure 3]1 is a flow diagram illustrating example operations that may be performed by a playback device according to some aspects of the present disclosure. [Figure 4] 10 illustrates an exemplary graph showing feedback or measurements for different seal qualities, according to some aspects of the present disclosure. [Figure 5] 1 illustrates an exemplary flow diagram for providing continuous feedback in accordance with some aspects of the present disclosure.

[0022] Like numbers refer to like elements. DETAILED DESCRIPTION OF THE INVENTION

[0023] The present disclosure provides processes, methods, systems, and devices for providing a user with feedback on a wearable device. The feedback may indicate the level of seal, fit, or compatibility between the wearable device and the user. For example, the feedback provides an accurate measurement and quantified report on how well the seal is created when the user wears the wearable device, indicating the audio playback performance of the wearable device as experienced by the user. This allows the user to identify the best adjustment or selection of components of the wearable device (e.g., tips, inserts, cups, etc.) to deliver the best audio performance achievable by the wearable device, as well as comfort levels. The wearable device may include earphones, headphones, a headset, or any audio device that physically contacts the user.

[0024] Generally, the techniques disclosed herein involve playing an audio signal through a speaker on a wearable device. For example, the audio signal includes any audible sound waves generated by the speaker. The wearable device includes at least one internal microphone adjacent to the speaker, and the internal microphone is used to measure audio data associated with the audio signal. Very often, the internal microphone is positioned within a space enclosed by the interface between the wearable device and the user's ear. Thus, the audio data measured by the internal microphone may include the audio signal and any reflections or propagations of the audio signal within the enclosed space. In some cases, the internal microphone may be a microphone used for active noise cancellation.

[0025] Based on the audio data, feedback regarding the seal quality of the wearable device's interface to the user's ear may be provided to the user. The feedback may include a visual response and / or an audio response. The feedback dynamically changes based on the wearable device's interface to the user's ear, as determined by at least one of application of a flexible sealing coupling element (e.g., a tip, insert, or cup) for a speaker or placement of the flexible coupling element relative to the user's ear. The placement of the flexible coupling element may include the position or orientation of the flexible coupling element relative to the user's ear. For example, if the flexible coupling element is a soft tip for an earbud, the user can insert the soft tip into the ear at different depth levels to create different placements and result in different seal qualities. Similarly, if the soft tip has an asymmetric shape, orienting the soft tip (i.e., rotating and angling it in different directions) within the ear will result in different seal qualities. Replacing the soft tip with another one having a different size, shape, material, or other property will similarly result in different seal qualities.

[0026] Conventionally, multiple flexible coupling elements are provided on wearable devices. For example, earphones have flexible tips of different sizes, shapes, and flexibilities. Headphones have cups of different materials, sizes, etc. Users often select one of the flexible coupling elements to use with a wearable device based solely on comfort. However, users have no standard for expected audio or noise cancellation, and therefore no basis for judging seal quality. The present disclosure provides techniques for dynamically providing feedback on seal quality, thus enabling users to identify a flexible tip that provides an optimized comfort level and sound performance. For example, continuous feedback may be provided at a specific rate (e.g., the number of feedbacks provided per specific period). Thus, even if a particular feedback may be presented to a user discretely (e.g., one at a time), such feedback is still considered continuous feedback.

[0027] The present disclosure provides various benefits for providing feedback. In some cases, by providing a user with a simple and clear indication of seal quality, the user can consider seal quality in selecting a flexible coupling element. Selecting an appropriate flexible coupling element can significantly improve the audio performance achieved, for example, by allowing the user to select a better-fitting coupling element, particularly one that not only achieves a seal when well-positioned (e.g., in the ear), but also provides a robust seal that is resistant to jaw movement, motion, and the like. Various techniques described herein provide the user with dynamic feedback regarding seal quality (e.g., based on the seal quality of the wearable device's interface with at least a portion of the user's head, such as a portion of the user's ear), thereby allowing the user to receive real-time feedback regarding seal quality as the user adjusts the wearable device to different positions (i.e., as the wearable device is moved relative to the user) and / or as different audio signals are played to determine seal quality at different frequencies or different sounds. For example, these techniques can be initiated when the user inserts the earbuds into the user's ears. Before insertion begins, there is no seal, and the feedback indicates a seal. When the earbuds are initially inserted, feedback can indicate that the seal is improving (e.g., using visual, auditory, and / or haptic feedback on the wearable device and / or a remote device connected to the wearable device). As the user completes insertion, the seal should improve, and feedback can indicate so. However, even when the user completes insertion of the earbuds, the seal may not provide threshold audio performance and / or active / passive noise reduction characteristics, and feedback can indicate so. This can prompt the user to adjust the fit of the earbuds while receiving real-time feedback regarding the fit quality of the earbuds.In some cases, the fit may not exceed a predetermined threshold due to the use of inappropriate eartips and / or retention members for the earbuds, and the present technology can therefore prompt the user to try different eartips and / or retention members. In this way, the present technology helps inform the user about the seal quality and allows the user to experiment with the fit of the wearable device to help achieve a desired balance between seal quality, comfort, and stability. This is particularly beneficial for wearable devices that have multiple configurations for fitting the device to the user, such as having different eartips, retention members, earcups, ear cushions, earhooks, etc. (where the differences can be based on, for example, size, shape, and / or material), because the present technology can use the measured seal quality to suggest switching to one or more different configurations for the wearable device (e.g., suggesting using larger or smaller eartips).

[0028] In some aspects, the wearable device may be paired with a playback device, which is a computing device operable to play multimedia documents or streamed broadcasts. The wearable device may receive an audio stream from the playback device. The wearable device may be paired with the playback device via a Bluetooth connection. The wearable device may include a speaker and a microphone. The speaker is configured to output the audio playback provided by the playback device. The microphones may be positioned in various locations, some near the speaker and some to capture the user's voice. The microphone near the speaker may be used to capture a feedback audio signal to determine seal quality.

[0029] 1 illustrates an example of a system 100 in which aspects of the present disclosure may be practiced. As shown, the system 100 includes a wearable device 110 communicatively coupled to a playback device 120. The wearable device 110 is shown as either an earphone set or a headset. The wearable device 110 includes at least two speakers, one for each ear. One speaker 111 is shown on the headset and one speaker 114 is shown in the earphones. The playback device 120 is shown as a smartphone or tablet computer.

[0030] In one aspect, wearable device 110 includes at least one respective internal microphone 112 or 118 adjacent to speaker 111 or 114. For example, internal microphone 112 is positioned inside an earcup of wearable device 110, next to the internal speaker relative to the earcup. Similarly, internal microphone 118 is positioned inside a tip or insert of wearable device 110. Internal microphone 112 or 118 is configured to measure respective audio data associated with the audio signal played by speaker 111 or 114. The measured audio data can be processed to indicate the seal quality between wearable device 110 and the user's ear (example shown in FIG. 2).

[0031] For example, if the wearable device 110 is in the form of a headset, the wearable device 110 may include a flexible seal 113 that conforms to the contours of the user's ear and face to form a seal between the user's ear and the speaker 111 (and internal microphone 112). If the wearable device 110 is in the form of an earphone or the like, the wearable device 110 may include a seal 116 in the form of a flexible insert or tip that is inserted into the user's ear and forms a seal with the opening of the ear canal. This seals the speaker 114 and the internal microphone 118 within the space within the user's ear. Because ear size and shape may vary across the population, a single seal may not be ideal for all users. Thus, users may prefer different seals 116. When multiple seals 116 are provided to a user as an accessory, the user can use the feedback techniques disclosed herein to identify a seal 116 that, for example, provides the best sound quality achievable by the wearable device 110 while also being comfortable based on their experience wearing it. The technique may also be used to help the user balance seal quality and perceived comfort so that the user may intentionally select an insert / tip and / or fitting that prioritizes one of seal quality or comfort over the other.

[0032] In some cases, wearable device 110 may include voice activity detection (VAD) circuitry capable of detecting the presence of a speech signal (e.g., a human speech signal) in a sound signal. Wearable device 110 may further include hardware and circuitry, including a processor / processing system and memory, configured to implement one or more sound management or other capabilities, including, but not limited to, noise canceling circuitry (not shown) and / or noise masking circuitry (not shown), body movement detection devices / sensors and circuitry (e.g., one or more accelerometers, one or more gyroscopes, one or more magnetometers, etc.), geolocation circuitry, and other sound processing circuitry.

[0033] In one aspect, wearable device 110 is wirelessly connected to playback device 120 using one or more wireless communication methods, including but not limited to Bluetooth, Wi-Fi, Bluetooth Low Energy (BLE), other RF-based techniques, etc. In one aspect, wearable device 110 includes a transceiver that transmits and receives data via one or more antennas to exchange audio data and other information with playback device 120. In some cases, playback device 120 is configured to receive feedback from wearable device 110 and may provide visual feedback to the user upon receiving the feedback. In some cases, playback device 120 may transmit an audio signal to wearable device 110, which converts the audio signal into sound waves to generate the feedback.

[0034] In one aspect, wearable device 110 includes communications circuitry capable of transmitting and receiving audio data and other information from playback device 120. Wearable device 110 also includes an incoming audio buffer, such as a render buffer, that buffers at least a portion of the incoming audio signal (e.g., audio packets) to allow time for retransmission of any missing or missing data packets from playback device 120. For example, when wearable device 110 receives a Bluetooth transmission from playback device 120, the communications circuitry typically buffers at least a portion of the incoming audio data in a render buffer before the audio is actually rendered and output as audio to at least one of the transducers (e.g., audio speakers) of wearable device 110. This is done to ensure that even if there is an RF collision that causes an audio packet to be lost during transmission, the lost audio packet has time to be retransmitted by playback device 120 before it needs to be rendered by wearable device 110 for output by one or more acoustic transducers of wearable device 110.

[0035] Wearable device 110 is shown as a headphone or earphone. However, the techniques described herein apply to other wearable audio devices, including any audio output device that fits at least partially around, on, in, or near one or both of a user's ears to create at least a partial seal with the ears. For example, this can include over-ear headsets or headphones that include earcups that at least partially seal with the user's ears, wired or wireless earphones (e.g., truly wireless earphones) where each earphone includes an eartip portion that at least partially seals with the user's ear, etc. Wearable device 110 can take any form, including a standalone device, a stationary device, headphones, earphones, earpieces, headsets, goggles, headbands, earbuds, sports headphones, neckbands, or glasses.

[0036] In one aspect, wearable device 110 connects to playback device 120 using a wired connection, with or without a corresponding wireless connection. Playback device 120 may be a smartphone, tablet computer, laptop computer, digital camera, or other playback device that connects wired and / or wirelessly with wearable device 110. As shown, playback device 120 can connect to network 130 (e.g., the Internet) and can access one or more services on the network. As shown, these services may include one or more cloud services 140.

[0037] In one aspect, playback device 120 can access cloud servers in cloud 140 on network 130 using a mobile web browser or a local software application or "app" running on playback device 120. In one aspect, the software application or "app" is a local application that is installed and executed locally on playback device 120. In one aspect, cloud servers accessible on cloud 140 include one or more cloud applications running on the cloud servers. The cloud applications can be accessed and executed by playback device 120. For example, the cloud applications can generate web pages that are rendered by a mobile web browser on playback device 120.

[0038] In one aspect, mobile software applications installed on playback device 120 or cloud applications installed on a cloud server, individually or in combination, can be used to implement techniques for low-latency Bluetooth communication between playback device 120 and wearable device 110 according to aspects of the present disclosure. In one aspect, examples of local software applications and cloud applications include gaming applications, audio AR applications, and / or gaming applications with audio AR capabilities. Playback device 120 can receive signals (e.g., data and control) from and send signals to wearable device 110.

[0039] Although some examples herein describe low latency Bluetooth® communication between a smartphone and earphones with flexible tips, these aspects can be used interchangeably with any portable playback device and any wireless audio output device that has a flexible coupling element.

[0040] 2 shows a cross-sectional view 200 illustrating a seal 206 between a speaker 202 and an ear canal 205 according to some embodiments of the present disclosure. As shown, a wearable device 110 includes a speaker 202 having a tip 208 applied thereto. A user may position the tip 208 inside and against the ear canal 205 to form the seal 206. The quality of the seal 206 depends on at least one of: (1) the size, shape, material properties, and other aspects of the tip 208; (2) how well the shape of the tip 208 conforms to the natural shape of the ear; and (3) its placement within the ear, including both its initial position when worn and how much that position may shift with the user's movements or activities. The placement 215 includes the position (e.g., translation in a coordinate system) and orientation (e.g., rotation in a coordinate system) of the flexible coupling element relative to at least one of the user's ears.

[0041] The embodiments primarily describe techniques for providing dynamic feedback of the seal quality of the seal 206. The seal quality corresponds to the acoustic performance of the speaker 202 coupled to the enclosed space bounded by the speaker 202, the tip 208, and the ear canal 205. Specifically, the seal quality measures the degree to which low-frequency sound emitted from the speaker escapes from that space due to an imperfect seal. The internal microphone 204 can accurately measure various sound waves within the enclosed space formed between the tip 208 and the ear canal 205. The audio data captured by the internal microphone 204 can be processed to determine a response (e.g., at least an amplitude response or a phase response) in a certain frequency range (e.g., a low-frequency range, particularly responsive to seal quality, as shown in FIG. 4 ). In some cases, the response may be interpreted to determine a fit quality indicator indicative of the level of the seal 206 between the tip 208 and the ear canal 205. Furthermore, this fit quality index can be divided into one or more calibrated ranges, which can be presented to the user to inform them of the degree of performance they should expect with their chip selection 208 and placement 215.

[0042] In some cases, when the wearable device 110 is in the form of a headset instead of earphones, the seal 206 may be created between the ear cup (not shown) and the ear 203, although other aspects are similar.

[0043] In aspects, wearable device 110 may provide feedback as a visual response, such as by outputting feedback on display 220. In some cases, the feedback may include an audio response played through speaker 202 on wearable device 110. As shown in FIG. 2, speaker 202 may be coupled to processor 210, which includes at least one memory 212. Processor 210 may be controlled by playback device 230, which allows a user to initiate the dynamic feedback generation process. The user may also control the manner in which feedback is received in playback device 230, i.e., visually, audibly, or both.

[0044] In some cases, wearable device 110 may include other components not explicitly shown in FIG. 2 , along with other components shown. For example, wearable device 110 includes an acoustic driver for converting audio signals into acoustic energy via speaker 202. Wearable device 110 also includes a network interface, at least one processor, audio hardware, a power supply for powering various components of wearable device 110, and memory 212. In one aspect, processor 210, network interface, audio hardware, power supply, and memory 212 are interconnected using various buses, and some of the components may be mounted on a common motherboard or in other manners as desired.

[0045] The network interface provides communication between wearable device 110 and other electronic playback devices via one or more communication protocols. The network interface provides either or both a wireless network interface and a wired interface 231 (optional). The wireless interface allows wearable device 110 to communicate wirelessly with other devices according to a wireless communication protocol such as IEEE 802.11. The wired interface 231 provides network interface functionality over a wired (e.g., Ethernet) connection for reliability and high transfer speeds, for use, for example, when wearable device 110 is not worn by a user.

[0046] All other digital audio received as part of a network packet can be passed straight from the network media processor through a USB bridge (not shown) to the processor 210, on to a decoder, a DSP, and finally rendered via an electroacoustic transducer.

[0047] The network interface may further include Bluetooth circuitry for Bluetooth applications (e.g., for wireless communication with Bluetooth-enabled audio sources such as smartphones or tablets) or other Bluetooth-enabled speaker packages. In some aspects, the Bluetooth circuitry may be the primary network interface due to energy constraints. For example, the network interface may use the Bluetooth circuitry solely for mobile applications when wearable device 110 adopts any wearable form factor. For example, BLE technology may be used in wearable device 110 to extend battery life, reduce package weight, and provide high-quality performance without other backup or alternative network interfaces.

[0048] In one aspect, the network interface supports communication with other devices using multiple communication protocols simultaneously at one time. For example, wearable device 110 may support Wi-Fi / Bluetooth coexistence and support simultaneous communication using both Wi-Fi and Bluetooth protocols at one time. For example, wearable device 110 may receive an audio stream from a smartphone using Bluetooth and further simultaneously rebroadcast the audio stream to one or more other devices over Wi-Fi. In one aspect, the network interface may include only one RF chain capable of communicating using only one communication method (e.g., Wi-Fi or Bluetooth) at a time. In the present context, the network interface may support Wi-Fi communication and Bluetooth communication simultaneously, for example, by time-dividing a single RF chain between Wi-Fi and Bluetooth according to a time division multiplexing (TDM) pattern.

[0049] The streamed data may be passed from the network interface to the processor 210. The processor 210 may execute instructions (e.g., for performing digital signal processing, decoding, and equalization functions, among others), including instructions stored in memory 212. The processor 210 may be implemented as a chipset of chips including separate analog and digital processors. The processor 210 may provide coordination of other components of the audio wearable device 110, such as controlling a user interface.

[0050] In some aspects, memory 212 stores software / firmware related to protocols and their versions used by wearable device 110 to communicate with other networked devices. For example, the software / firmware manages how wearable device 110 communicates with other devices for synchronized audio playback. In one aspect, the software / firmware includes lower-level frame protocols related to control path management and audio path management. Protocols related to control path management generally include protocols used to exchange messages between speakers. Protocols related to audio path management generally include protocols used for clock synchronization, audio distribution / frame synchronization, and audio decoder / audio stream playback. In one aspect, the memory can also store various codecs supported by speaker packages for audio playback of respective media formats. In one aspect, the software / firmware stored in memory can be accessible and executable by processor 210 for synchronized audio playback with other networked speaker packages.

[0051] In certain aspects, the protocols stored in memory 212 may include, for example, BLE according to Bluetooth Core Specification version 5.2 (BT5.2). Wearable device 110 and various components therein are provided herein to fully conform to or implement aspects of the protocols and associated specifications. For example, BT5.2 includes an enhanced attribute protocol (EATT) that supports concurrent transactions. To support EATT, a new L2CAP mode is defined. Thus, wearable device 110 includes sufficient hardware and software components to support the BT5.2 specifications and modes of operation, even if not explicitly shown or discussed in this disclosure. For example, wearable device 110 may utilize LE isochronous channels specified in BT5.2.

[0052] The processor 210 provides the processed digital audio signals to audio hardware, which includes one or more digital-to-analog (D / A) converters for converting the digital audio signals to analog audio signals. The audio hardware also includes one or more amplifiers that provide amplified analog audio signals to electroacoustic transducers for sound output. In addition, the audio hardware may include circuitry for processing analog input signals and providing digital audio signals for sharing with other devices, such as other speaker packages for synchronized output of digital audio.

[0053] The memory 212 may be any non-volatile or non-transitory memory device. The memory 212 may include, for example, flash memory and / or non-volatile random access memory (NVRAM). In some aspects, the instructions (e.g., software) are stored on an information carrier. When executed by one or more processing devices (e.g., processor 210), the instructions perform one or more processes, such as those described elsewhere herein. The instructions may also be stored by one or more storage devices, such as one or more computer-readable or machine-readable media (e.g., memory 212 or memory on a processor). The instructions may include instructions for performing decoding (i.e., a software module may include an audio codec for decoding a digital audio stream) and digital signal processing and equalization. In certain aspects, the memory 212 and processor 210 may cooperate with the internal microphone 204 in data acquisition and real-time processing.

[0054] 3 is a flow diagram illustrating example operations 300 that may be performed by a wearable device according to aspects of the present disclosure. For example, the example operations 300 may be performed by the wearable device 110 of FIGS. 1 and 2 to provide dynamic feedback to a user.

[0055] The example operations 300 begin by playing an audio signal through a speaker on a wearable device at 302. The wearable device measures audio data associated with the audio signal using a microphone on the wearable device at 304. The wearable device is configured to be worn by a user such that the microphone shares a cavity with the user's ear canal.

[0056] At 306, the wearable device provides feedback to the user regarding the seal quality of the wearable device's interface with at least a portion of the user's head, the feedback being provided continuously i) while the wearable device is moved relative to the user, ii) while the audio signal played through the speaker is changed, or iii) both.

[0057] In some cases, the feedback dynamically changes based at least in part on the audio data and the interface of the wearable device to at least one of the user's ears, as determined by at least one of the application of a flexible coupling element for the wearable device or the placement of a flexible seal tip in at least one of the user's ears. In aspects, the feedback includes a visual response, an audio response, or both. For example, the visual response may include displaying a judgment of seal quality on a display of the wearable device or another device connected to the wearable device (e.g., playback device 120 of FIG. 1 ). The feedback may be descriptive in terms of an assessment such as excellent, adequate, fair, or poor. Such descriptive feedback may be represented by different tones in the audio response and / or different colors, icons, or text in the visual response. The feedback may be quantitative, such as providing a score or a quantified result (e.g., a graphical representation of an amplitude or phase response).

[0058] In aspects, the placement of the flexible coupling element relative to the ear includes at least a position or orientation of the flexible coupling element relative to at least one of the user's ears. In some cases, the speaker and internal microphone on the wearable device are positioned inside the user's ear. The audio data measured by the internal microphone may include a low-frequency response indicative of a level of seal between the speaker and the user's ear, the low-frequency response including at least one of an amplitude response or a phase response. In some cases, both the amplitude response and the phase response may be used to determine the level of seal. In some cases, one of the amplitude response and the phase response can be used.

[0059] In some cases, the method further includes indicating a fit quality indicator when a level of seal between the speaker and the user's ear is within one of one or more calibrated ranges used to communicate to the user (e.g., corresponding to a "bad fit," or an "acceptable fit," or a "good fit." In some cases, the method further includes actively canceling ambient noise via the speaker when the fit quality indicator is above a threshold.

[0060] In an aspect, the method further includes generating an audio signal based on a profile of frequency variation to evoke a low frequency response.

[0061] In embodiments, the audio data measured by the internal microphone includes a low-frequency response that indicates the level of seal between the speaker and the user's ear. An example frequency response is shown in Figure 4 and described below.

[0062] FIG. 4 shows exemplary graphs 400 and 401 illustrating feedback or measurements for different seal qualities, according to some embodiments of the present disclosure. In exemplary embodiments, different seal qualities are provided using various flexible tips on the earbuds that form a seal with the user's ear. On the left, amplitude response (vertical axis) is plotted against frequency (logarithmic horizontal axis). On the right, phase response (vertical axis) is plotted against frequency (logarithmic horizontal axis). As shown, plots of the complex response from the driver to the feedback (system) microphone complex response (magnitude in dB, phase in degrees) illustrate the different results produced by different seal qualities. Exemplary measurements are taken at a test frequency of approximately 281 Hz (other frequencies may also be used to provide feedback).

[0063] Achieving a good seal is important for both passive and active performance. The noise-canceling earphones described herein aid in identifying and positioning flexible coupling elements, such as soft tips, of the earphones. The present disclosure provides quantified feedback that allows users to learn about better performance achievable by the earphones when tips whose size or shape closely match the shape of the ear are used and when the earphones with the tips are properly positioned. As shown, the low-frequency driver-to-feedback microphone response of earphones (e.g., noise-canceling earphones fitted with various tips) varies significantly with seal quality. Among the multiple recorded data sets, four main representative lines are shown: (1) a lower reference line 402 when the earbud and its tip are in free air (i.e., not inserted in a user's ear); (2) an upper reference line 404 when the earbud's tip is blocked; (3) a representative good fit line 410 when the tip forms an expected seal quality with the user's ear that allows for good noise cancellation and audio performance; and (4) a representative bad fit line 420 that averages cases when the earbud's tip is unable to form a good seal with the user's ear.

[0064] Using these four representation lines, different seal qualities may be indicated when the response falls within the zones bounded by the continuous lines. Accordingly, seal quality may be represented by a fit quality indicator. For example, if the measured response is between the lower reference line 402 and the poor fit line 420, visual feedback (e.g., an icon, color, notification, etc.) and / or audio feedback (e.g., a tone, recording, etc.) indicating a poor fit (i.e., a fit quality indicator of poor seal quality) may be provided, such as at the wearable device 110 and / or playback device 120. If the measured response is between the poor fit line 420 and the good fit line 410 or approximates the good fit line 410 within a certain defined tolerance, visual feedback and / or audio feedback indicating an acceptable fit (i.e., a fit quality indicator of adequate seal quality) may be provided. If the measured response is between the good fit line 410 and the upper reference line 404, visual feedback and / or audio feedback indicating an excellent fit (i.e., a fit quality indicator of good seal quality) may be provided.

[0065] In aspects, the dynamic feedback provided to the user includes a clear and simple auditory and / or visual indication of the quality of the seal. The feedback updates in near real time as the user moves the earphones (or another form of wearable device) relative to the user's ear (e.g., insertion depth, rotation, etc.). The visual feedback indication(s) may also have a clear indication of good design-intent performance. In some cases, the visual indication may be a simple meter or bar graph. The auditory feedback may be a series of chords played to the earphones, each chord associated with a range of seal quality. The sequence of chords may form a clear progression to a solution. For example, a poor fit chord may be dissonant or may sound like an orchestra with more instruments emerging as the fit quality improves. Furthermore, the complexity of the instrumentation in the sound can vary, as can the repetition rate of the sound aspect. For example, a user may hear this progression through three fit quality ranges as they slowly put on and move the earphones into a good position with an appropriately selected tip that achieves a good seal.

[0066] In some cases, the audio feedback may be from a multimedia recording (e.g., a video, a presentation, or a document with recorded audio aspects). The multimedia recording may be displayed as or along with the application in which the audio feedback is played. The multimedia recording may simultaneously provide one or more visual cues to prompt specific operations of the chip, such as while the audio feedback is played in parallel.

[0067] In some cases, a combination of both visual feedback (e.g., in a playback device 120 application communicating with wearable device 110 via Bluetooth) and audio feedback at wearable device 110 may best inform the user directly as to the exact performance of wearable device 110. The visual and audio feedback may be tailored to reinforce meaning, such as a 1, 2, or 3 bar display (like a cellular signal strength meter) corresponding to a three-stage progression of sound.

[0068] 5 shows an example flow diagram 500 for providing continuous feedback of fit quality (FQ) according to some aspects of the present disclosure. As shown, at 510, the wearable device can initialize an FQ value to 0. At 520, the current FQ value is measured or identified and compared to two range thresholds to determine a sound loop for playback. For example, the current FQ value can be fed back to replace the initialized FQ value and updated every execution cycle.

[0069] At 530, the current FQ value is sent to an application for visual display. The visual display may correspond to a sound recording being played, such as a test sound sequence to determine the FQ value.

[0070] At 540, the wearable device may acquire a 2048-point frame of driver and microphone data at 48 kHz (or another frequency), where the frame corresponds to a Hamming window (e.g., an extension of a Hann window with a raised cosine window format and a corresponding spectral format).

[0071] At 550, the wearable device applies the Goertzel algorithm to calculate the signal at a test frequency, such as 281.25 Hz (13th FFT bin) of the driver and microphone signals. The driver-to-microphone amplitude ratio is calculated along with the corresponding phase response.

[0072] At 560, the driver-to-microphone amplitude is compared to a free-air threshold, measured when air can freely enter and exit the cavity formed by the wearable device and the ear. If the magnitude is less than the free-air threshold, FQ is assigned to zero at 570. Otherwise, at 572, the wearable device applies a linear mapping to calculate FQ from phase.

[0073] At 580, frame-to-frame FQ value variations are smoothed using a low-pass filter and a slew rate limit. At 590, once the current sound loop is completed, a check step is performed to determine if the user has stopped the test. For example, the user may stop the test by interacting with the wearable device or an application running on the user device. The test may also be stopped using a timer. If the user has not stopped the test, the test continues by looping back to 520. If the user has stopped the test, the test sequence ends at 590.

[0074] In other aspects, the disclosed method is applicable to wireless earbuds, earhooks, or ear-to-ear devices. For example, a host such as a mobile phone may be connected to a bud (e.g., the right bud) via Bluetooth®, and the right bud further connects to the left bud using a Bluetooth® link or other wireless technology such as NFMI or NFEMI. The left bud is initially time-synchronized with the right bud. Audio frames (compressed to mono) are transmitted from the left bud with timestamps (synchronized with the right bud's timestamps) as described in the techniques above. The right bud forwards these encoded mono frames along with its own frames. The right bud does not wait for audio frames from the left bud with the same timestamp. Instead, the right bud transmits any frames that are available and ready to be transmitted using the appropriate packing. It is the responsibility of the receiving application in the host to assemble packets using the timestamps and channel numbers. Depending on how it is configured, the receiving application can choose to merge the decoded mono channel of one bud with the decoded mono channel of the other bud into a stereo track based on the timestamp included in the header of the received encoded frame. This disclosure allows the right bud to simply forward audio frames from the left bud without decoding the frames. This helps conserve battery power in truly wearable devices.

[0075] Although the description of the embodiments of the present disclosure has been presented above for purposes of illustration, it may be noted that the embodiments of the present disclosure are not intended to be limited to any of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0076] In the above, reference is made to embodiments presented in the present disclosure. However, the scope of the present disclosure is not limited to the particular described embodiments. Aspects of the present disclosure may take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware embodiments, all of which may be generally referred to herein as "components," "circuits," "modules," or "systems." Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-readable program code embodied thereon.

[0077] Any combination of one or more computer-readable media may be utilized. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media include an electrical connection having one or more wires, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present context, a computer-readable storage medium may be any tangible medium that can contain or store a program.

[0078] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects. In this regard, each block in the flowcharts or block diagrams may correspond to a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in the blocks may occur out of the order depicted in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or in some cases, the blocks may be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented in a dedicated hardware-based system performing a particular function or operating a combination of dedicated hardware and computer instructions. [Explanation of symbols]

[0079] 130 Network 140 Cloud 210 processor 212 memory 220 Display 230 Playback Devices

Claims

1. 1. A method for providing feedback regarding seal quality of a wearable device, the method comprising: playing an audio signal through a speaker on the wearable device, the audio signal being converted into acoustic energy through the speaker by a driver of the wearable device; measuring audio data associated with the audio signal using a microphone on the wearable device, the wearable device being configured to be worn by the user such that the microphone shares a cavity with the user's ear canal; providing feedback to the user regarding the seal quality of the interface of the wearable device with at least a portion of the user's head, the feedback being provided continuously i) while the wearable device is moved relative to the user, ii) while the audio signal played through the speaker is changed, or iii) while the wearable device is moved relative to the user and the audio signal played through the speaker is changed, after acquiring 2048 point frames of the driver and audio data of the wearable device at a set frequency, applying a Goertzel algorithm to calculate an amplitude ratio of a driver signal to a microphone signal and a corresponding phase response, the frames corresponding to a Hamming window, the feedback being calculated based on the corresponding phase response, and comparing the amplitude of the driver signal to a free air threshold to determine the fit quality of the wearable device as an amplitude response; A method comprising:

2. The method of claim 1 , wherein providing the feedback comprises providing a visual response via a remote device connected to the wearable device.

3. The method of claim 1 , wherein providing the feedback includes providing an audio response via the speaker on the wearable device.

4. The method of claim 1 , wherein the speaker and the microphone on the wearable device are positioned inside the user's ear.

5. 5. The method of claim 4, wherein the audio data measured by the microphone comprises a low-frequency response indicative of a level of seal between the speaker and the user's ear, the low-frequency response comprising at least one of the magnitude response or the phase response.

6. The method of claim 5 , further comprising indicating a fit quality indicator when the seal quality is within one of a plurality of different ranges.

7. The method of claim 6 , further comprising initiating active noise cancellation using the speaker in response to the fit quality indicator exceeding a threshold.

8. The method of claim 5 , further comprising generating the audio signal based on a profile of frequency variation to evoke the low frequency response.

9. The method of claim 1 , wherein the microphone is a microphone used for active noise cancellation.

10. The method of claim 1 , wherein the at least a portion of the user's head includes a user's ear.

11. 2. The method of claim 1, wherein the feedback dynamically changes based at least in part on the audio data and the interface of the wearable device relative to the at least one of the user's ears, as determined by at least one of application of a flexible coupling element for the wearable device or placement of the flexible coupling element relative to the at least one of the user's ears, wherein the placement of the flexible coupling element relative to the ear includes at least a position or orientation of the flexible coupling element relative to the at least one of the user's ears.

12. 1. A wearable device configured to be worn by a user such that a microphone of the wearable device shares a cavity with the user's ear canal, the wearable device comprising: at least one speaker configured to play audio signals to the user; a driver configured to convert the audio signal into acoustic energy via the at least one speaker; a microphone adjacent to the at least one speaker, the microphone configured to measure audio data associated with the audio signal reproduced by the at least one speaker; a processor, the processor processing the audio data to dynamically determine, in a closed loop, feedback regarding the seal quality of the interface of the wearable device with at least one of the user's ears, the feedback being provided continuously i) while the wearable device is moved relative to the user, ii) while the audio signal played through the speaker is changed, or iii) while the wearable device is moved relative to the user and the audio signal played through the speaker is changed; after acquiring 2048-point frames of the driver and audio data of the wearable device at a set frequency, applying a Goertzel algorithm to calculate a driver signal-to-microphone signal amplitude ratio and a corresponding phase response, the frames corresponding to a Hamming window; and calculating the feedback based on the corresponding phase response; and comparing the driver signal-to-microphone signal amplitude to a free-air threshold to determine the fit quality of the wearable device as an amplitude response; a wearable device configured to output the feedback to the user.

13. The wearable device of claim 12 , further comprising a visual indicator configured to provide a visual indication of the feedback to the user.

14. The wearable device of claim 12 , wherein the feedback is reproduced by at least the at least one speaker.

15. The wearable device of claim 12 , wherein movement of the wearable device includes at least a position or orientation of the wearable device relative to the at least one of the user's ears.

16. The wearable device of claim 15 , wherein the speaker and microphone on the wearable device are positioned inside the user's ear.

17. 17. The wearable device of claim 16, wherein the audio data measured by the microphone comprises a low-frequency response indicative of a level of seal between the speaker and the user's ear, the low-frequency response comprising at least one of the amplitude response or the phase response.

18. 18. The wearable device of claim 17, wherein the feedback includes a fit quality indicator that indicates when the level of the seal between the speaker and the user's ear is within a calibrated range.

19. 1. A system comprising: A wearable device, at least one speaker configured to play audio signals to a user; a driver configured to convert the audio signal into acoustic energy via the at least one speaker; a microphone adjacent to the at least one speaker, the microphone configured to measure audio data associated with the audio signal reproduced by the at least one speaker; 1. A processor, comprising: processing the audio data to dynamically determine, in a closed loop, feedback regarding the seal quality of the interface of the wearable device with at least one of the user's ears, the feedback being provided continuously i) while the wearable device is moved relative to the user, ii) while the audio signal played through the speaker is changed, or iii) while the wearable device is moved relative to the user and the audio signal played through the speaker is changed; after acquiring 2048-point frames of the driver and audio data of the wearable device at a set frequency, applying a Goertzel algorithm to calculate a driver signal-to-microphone signal amplitude ratio and a corresponding phase response, the frames corresponding to a Hamming window; and calculating the feedback based on the corresponding phase response; and comparing the driver signal-to-microphone signal amplitude to a free-air threshold to determine the fit quality of the wearable device as an amplitude response; a wearable device comprising: a processor configured to output the feedback to the user; a playback device in communication with the wearable device, the playback device configured to receive the feedback; and A system comprising:

20. The system of claim 19 , wherein the playback device transmits the audio signal to the wearable device.

Images