Wearable audio device placement detection

Abstract

An earplug has an electro-acoustic transducer for generating sound, a proximity sensor configured to detect when the earplug is close to a skin of a user, an orientation sensor configured to detect an orientation of the earplug, and a processor configured to estimate whether the earplug has been inserted into an ear canal of the user based on the proximity sensor and the orientation sensor.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a continuation-in-part of and claims priority to International Patent Application No. PCT / US2021 / 034049, filed May 25, 2021, which itself claims priority to U.S. Patent Application No. 16 / 882,673, filed May 25, 2020, now published as U.S. Patent No. 11,202,137, published December 14, 2021. The entire disclosures of both prior applications are incorporated herein by reference in their entirety and for all purposes. Background Art

[0003] The present disclosure relates to wearable audio devices.

[0004] Wearable audio devices are designed to work best when they are in their proper use position on the body and when the user has stopped operating the device. Summary of the invention

[0005] All examples and features mentioned below can be combined in any technically possible way.

[0006] In one aspect, a wearable audio device includes: an electroacoustic transducer for generating sound, a proximity sensor configured to detect when the wearable audio device is close to a user, an orientation sensor configured to detect the orientation of the wearable audio device, an internal microphone positioned within a housing of the wearable audio device and configured to detect sound in a cavity formed at least in part by a portion of the housing, and a processor configured to (i) estimate whether the wearable audio device is in place on the user's body based on the proximity sensor and the orientation sensor, and (ii) estimate whether the user has completed an operation of the wearable audio device based on the internal microphone.

[0007] Some examples include one of the above and / or following features or any combination thereof. In one example, the proximity sensor includes an infrared sensor. In one example, the orientation sensor includes an inertial measurement unit (IMU). In one example, the IMU includes an accelerometer for detecting the orientation of the wearable audio device. In one example, the proximity sensor has an output, and the processor's estimation of whether the wearable audio device is in place on the user's body is based in part on the level of the proximity sensor output. In one example, after the processor estimates whether the wearable audio device is in place on the user's body, the processor is configured to enable a predetermined wearable audio device function. In one example, the predetermined wearable audio device function includes at least one of the following: telephone call answering capability, Bluetooth connection, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of user interface functions, and tuning of an active noise reduction system.

[0008] Some examples include one or any combination of the above and / or following features. In some examples, the wearable audio device also includes an external microphone configured to sense sounds outside the housing, and the processor estimate of whether the wearable audio device is in place on the user's body is also based on a transfer function between the external microphone and the internal microphone. In one example, the processor is further configured to cause the electroacoustic transducer to generate sounds based on the transfer function. In one example, the transfer function is determined at a frequency of up to 1000 Hz. In one example, the processor is further configured to cause the electroacoustic transducer to generate sounds of two different frequencies, wherein the first frequency is at least 1500 Hz and the processor is configured to determine whether the wearable audio device nozzle is blocked based at least in part on the first frequency, and the second frequency does not exceed 1000 Hz and the processor is configured to determine whether the wearable audio device is in place on the user's body based at least in part on the second frequency.

[0009] Some examples include one or any combination of the above and / or following features. In one example, the generated sound is part of a wearable audio device startup tone sequence. In one example, the processor estimates whether the wearable audio device is in place on the user's body, including two sequential steps: a first step based at least on the proximity sensor, the orientation sensor, and the internal microphone, and a second step based at least on the internal microphone and the external microphone. In some examples, the processor is configured to initiate a first group of wearable audio device functions based on the first step and initiate a second group of wearable audio device functions based on the second step, wherein the functions of the second group are different from the functions of the first group. In one example, the first group and the second group of functions include at least one of the following: phone call answering capability, Bluetooth connection, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of a user interface function, and tuning of an active noise reduction system. In one example, during at least the first step, the internal microphone is monitored at a frequency in the range of 0 Hz-20 Hz.

[0010] In another aspect, a method of detecting when a wearable audio device is in place on a user's body, wherein the wearable audio device includes: an electroacoustic transducer for generating sound, a proximity sensor configured to detect when the wearable audio device is close to the user, an orientation sensor configured to detect the orientation of the wearable audio device, an internal microphone positioned within a housing of the wearable audio device and configured to detect sound in a cavity formed at least in part by a portion of the housing, and an external microphone configured to sense sound outside the housing, including estimating whether the wearable audio device is in place on the user's body based on the proximity sensor, the orientation sensor, and the internal microphone, and estimating whether the user has completed an operation of the wearable audio device based on the internal microphone.

[0011] Some examples include one or any combination of the features described above and / or below. In one example, the processor is further configured to cause the electroacoustic transducer to generate sound after the processor has estimated that the wearable audio device is in place on the user's body, and wherein the processor is further configured to calculate a transfer function between the external microphone and the internal microphone over a specified frequency or frequency range of the generated sound. In one example, the processor's estimate of whether the wearable audio device is in place on the user's body includes two sequential steps: a first step based at least on the proximity sensor, the orientation sensor, and the internal microphone, and a second step based at least on the internal microphone and the external microphone, and wherein the processor is configured to initiate a first set of wearable audio device functions based on the first step and initiate a second set of wearable audio device functions based on the second step, wherein the functions of the second set are different from the functions of the first set.

[0012] On the other hand, a wearable audio device (e.g., headphones) includes: an electroacoustic transducer for generating sound; a proximity sensor configured to detect when the wearable audio device is close to a user; an internal microphone positioned within a housing of the wearable audio device (e.g., within an earmuff of the headphones) and configured to detect sound in a cavity formed at least in part by a portion of the housing; and a processor configured to estimate whether the wearable audio device is in place on the user's body based on at least the proximity sensor, and to estimate whether the user has completed an operation of the wearable audio device based on the internal microphone.

[0013] Some examples include one or any combination of the above and / or following features. In one example, the wearable audio device further includes: at least one earmuff, the at least one earmuff being configured to be located on or above the ear of the wearer when the wearable audio device is placed on the head, wherein both the proximity sensor and the internal microphone are positioned in the earmuff. In some examples, the proximity sensor includes a capacitive proximity sensor. In one example, the capacitive proximity sensor includes a conductive member inside the earmuff of the wearable audio device. In one example, the conductive member has a curved outer periphery that mimics the curve of a human auricle. In one example, the wearable audio device further includes: two earmuffs, the two earmuffs being configured to be located on or above the ear of the wearer when the wearable audio device is placed on the head, wherein there is an internal microphone positioned within each earmuff, and wherein an estimate of whether the user has completed operation of the wearable audio device is based on an internal microphone positioned within each of the earmuffs.

[0014] Some examples include one or any combination of the above and / or following features. In some examples, the processor is configured to initiate a first set of wearable audio device functions based on an estimate of whether the wearable audio device is in place on the user's body. In one example, the processor is configured to initiate a second set of wearable audio device functions based on an estimate of whether the user has completed an operation of the wearable audio device, wherein the functions of the second set are different from the functions of the first set. In one example, the first and second sets of functions include at least one of the following: phone call answering capability, Bluetooth connectivity, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of user interface functions, and tuning of an active noise reduction system.

[0015] Some examples include one or any combination of the above and / or following features. In one example, the internal microphone is monitored at a frequency in the range of 0 Hz-20 Hz during at least an estimate of whether the user has completed operation of the wearable audio device. In one example, the proximity sensor has an output, and the processor's estimate of whether the wearable audio device is in place on the user's body is based in part on the level of the proximity sensor output. In some examples, after the processor estimates whether the wearable audio device is in place on the user's body, the processor is configured to enable a predetermined wearable audio device function. In one example, the predetermined wearable audio device function includes at least one of the following: telephone call answering capability, Bluetooth connection, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of a user interface function, and tuning of an active noise reduction system. In one example, the processor is further configured to initiate an active noise reduction (ANR) filter determination procedure after the estimate of whether the user has completed operation of the wearable audio device.

[0016] Some examples include one or any combination of the above and / or following features. In some examples, the wearable audio device also includes an orientation sensor configured to detect the orientation of the wearable audio device. In one example, the processor is further configured to estimate whether the user has removed the wearable audio device from the user's body based on at least the orientation sensor. In one example, the processor is configured to reduce power usage based on an estimate of whether the user has removed the wearable audio device from the user's body. In one example, the estimate of whether the user has removed the wearable audio device from the user's body is also based on the proximity sensor. In some examples, the processor estimate of whether the wearable audio device is in place on the user's body is based on the proximity sensor and the orientation sensor. In one example, the orientation sensor includes an accelerometer. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a perspective view of a wearable audio device.

[0018] Figure 2 is a partial cross-sectional view of components of a wearable audio device.

[0019] Figure 3 is a flow chart of the operation of an earbud placement detection method.

[0020] Figure 4 The transfer function between the external microphone and the internal microphone of an earbud is illustrated, with the earbud located in free space and in the ear.

[0021] Figure 5 is a schematic partial cross-sectional view of a supra-aural headphone in position on a user's head.

[0022] Figure 6 is a partial interior view of an ear cup for an over-ear headphone.

[0023] Figure 7 is a flow chart of the operation of the headphone wearing and removal detection method. DETAILED DESCRIPTION

[0024] The present disclosure relates to wearable audio devices. Some non-limiting examples of the present disclosure describe a wearable audio device known as an earbud. An earbud typically includes an electroacoustic transducer for generating sound and is configured to deliver the sound directly into the ear canal of a user. The earbud may be wireless or wired. In the examples described herein, the earbud is wireless and therefore carries a power source (typically a rechargeable battery), a wireless communication system (in one example a Bluetooth-based communication system), and any necessary processing. Other aspects of the earbud not covered in the present disclosure are not shown or described.

[0025] Some examples of the present disclosure also describe a wearable audio device known as an open audio device. An open audio device has one or more electroacoustic transducers (i.e., audio drivers) positioned away from the ear canal opening. In some examples, the open audio device also includes one or more microphones; the microphones may be used to pick up the user's voice and / or for noise cancellation. Open audio devices are further described in U.S. Patent No. 10,397,681, the entire disclosure of which is incorporated herein by reference for all purposes.

[0026] Headphones refer to a device that is usually mounted around, on or in the ear and radiates sound energy directly or indirectly into the ear canal. Headphones are sometimes referred to as earphones, earphones, headsets, earplugs or sports headphones and can be wired or wireless. Headphones include an electroacoustic transducer (driver) for converting electrical audio signals into sound energy. The acoustic driver may or may not be accommodated in an earmuff or housing configured to be located on the head or on the ear or directly inserted into the user's ear canal. The headset may be a single independent unit or one headset in a pair of headphones (each headset includes at least one acoustic driver), each headset corresponding to one ear. The headset can be mechanically connected to another headset, for example, by a headband and / or by a lead that conducts the audio signal to the acoustic driver in the headset. The headset may include a component for wirelessly receiving an audio signal. The headset may include a component of an active noise reduction (ANR) system, which may include an internal microphone in the headset housing. The headset may also include other functionality, such as an additional microphone for the ANR system, or one or more microphones for picking up the user's voice.

[0027] In headphones that are around or on or away from the ear, the headphones may include a headband or other support structure, and / or at least one housing or other structure containing a transducer and arranged to be placed on or above or near the user's ear. The headband may be retractable or foldable and may be made from multiple parts. Some headbands include a slider that can be positioned inside the headband to facilitate any desired translation of the housing. Some headphones include a yoke pivotally mounted to the headband, wherein the housing is pivotally mounted to the yoke to provide any desired rotation of the housing.

[0028] Open-type audio devices include, but are not limited to, off-ear headphones, i.e., devices having one or more electroacoustic transducers that are coupled to the head or ears (usually via a support structure) but do not block the ear canal opening. In some examples, the open-type audio device is an off-ear headphone including audio glasses, but this is not a limitation of the present disclosure, as in an open-type audio device, the device is configured to deliver sound to one or both ears of the wearer, where earmuffs and earplugs are typically absent. The wearable audio systems contemplated herein may include a variety of devices including over-ear hooks, such as wireless headphones, hearing aids, glasses, protective hard hats, and other open-ear audio devices.

[0029] In various examples and combinations, one or more of the devices, systems, and methods described herein can be used in a wide range of wearable audio devices or systems, including wearable audio devices of various form factors. Unless otherwise specified, as used in this document, the term active portion of a wearable audio system includes headphones and various other types of wearable audio devices, such as acoustic devices worn on the head, shoulders, or body (e.g., audio glasses or other head-mounted audio devices), which include one or more acoustic transducers for receiving and / or generating sound with or without contacting the user's ears.

[0030] It should be noted that although specific implementations of wearable audio devices that primarily serve the purpose of acoustically outputting audio are presented with a certain degree of detail, such presentation of specific implementations is intended to facilitate understanding by providing examples and should not be taken as limiting the scope of the present disclosure or the scope of coverage of the claims.

[0031] In some examples, the wearable audio device includes a proximity sensor configured to detect when the earbud is close to the user. In one example, the proximity sensor detects the user's skin. In one example, the proximity sensor is an infrared (IR) sensor that can detect when the wearable audio device is close to or against the skin of the ear. In one example, the IR sensor output is within a predetermined range or at least a predetermined threshold level for making a position determination in use. In some examples, the wearable audio device also includes an orientation sensor configured to detect the orientation of the wearable audio device. In one example, the orientation is determined on three mutually orthogonal axes in space. Since the wearable audio device is designed to be worn in a specific orientation, knowing the orientation via the orientation sensor can indicate whether the device is in place. For example, when the earbud is in the ear of a head that is kept upright, the internal orientation sensor will have a known orientation in three-dimensional space. The orientation sensor result may be within a nominal predetermined range for making a position determination in use. The wearable audio device also includes a processor configured to estimate whether the wearable audio device is in place on the user's body based on the proximity sensor and the orientation sensor. In some examples where the wearable audio device is an earbud, the appropriate position is in the user's ear canal.

[0032] In some examples, the wearable audio device also includes an internal microphone. In one example, the internal microphone is positioned within the housing of the wearable audio device. In one example, the microphone is positioned to detect sound in a cavity formed at least in part by a portion of the housing. When the internal microphone is used in the wearable audio device, the processor may also be configured to estimate whether the user has completed the operation of the wearable audio device. In some examples, the operation includes placing the wearable audio device in place on the body or removing it from the body. In some examples, the wearable audio device also includes an external microphone. In one example, the external microphone is positioned to detect sound outside the housing of the wearable audio device. When the external microphone is used in the wearable audio device, the processor may also be configured to calculate the audio transfer function between the internal microphone and the external microphone, and estimate whether the wearable audio device is in its proper use position (e.g., in the ear) based on the transfer function.

[0033] Figure 1 is a perspective view of a wireless in-ear earbud 10. An earbud is a non-limiting example of a wearable audio device. Earbud 10 includes a body or housing 12 that houses the active components of the earbud. Portion 14 is coupled to body 12 and is flexible so that it can be inserted into the entrance of the ear canal. Sound is transmitted through opening 15. Retaining ring 16 is constructed and arranged to be positioned in the outer ear, such as in the anti-helix, to help retain the earbud in the ear. Earbuds are well known in the art (e.g., as disclosed in U.S. Pat. No. 9,854,345, the disclosure of which is incorporated herein by reference in its entirety for all purposes), and therefore certain details of earbuds are not further described herein.

[0034] Figure 2 is a partial cross-sectional view of only certain elements of the earplug 20 that can be used to better understand the present disclosure. The earplug 20 includes a housing 21 that encloses an electroacoustic transducer (audio driver) 30. The housing 21 includes a front housing portion 50 and rear housing portions 60 and 62. The transducer 30 has a diaphragm 32 that is driven to generate sound pressure in the front cavity 52. ​​The sound pressure is directed out of the front housing portion 50 via the sound outlet 54. An internal microphone 80 is located inside the housing 21. In one example, the microphone 80 is located in the sound outlet 54, as shown in FIG. Figure 2 As shown. External microphone 81 is configured to sense sound outside housing 21. In one example, internal microphone 80 is used as a feedback microphone for active noise reduction, and external microphone 81 is used as a feedforward microphone for active noise reduction. Earplugs, such as Figure 11, and generally includes a flexible tip (not shown) engaged with the neck 51 of the housing portion 50 to help conduct sound into the ear canal. The earbud housing 21 also includes a rear shell made of rear housing portions 60 and 62 and a grille 64. It is noted that the details of the earbud 20 are exemplary aspects of an earphone and do not limit the scope of the present disclosure, as the in-ear detection presented can be used in various types and designs of earbuds and earphones and other wearable audio devices.

[0035] The transducer 30 also includes a magnetic structure 34. As is well known in the art of electroacoustic transducers, the magnetic structure 34 includes a transducer magnet 38 and magnetic material for confining and directing the magnetic field from the magnet 38 so that the magnetic field interacts appropriately with the coil 33 to drive the diaphragm 32. The magnetic material includes a cup 36 and a front plate 35, both of which are preferably made of a material with a relatively high magnetic susceptibility, as is also known in the art. A transducer printed circuit board (PCB) 40 carries the electrical and electronic components (not shown) involved in driving the transducer. Pads 41 and 42 are locations where wires (not shown) can be coupled to the PCB 40.

[0036] In one example, an inertial measurement unit (IMU) 72 is used to detect the orientation of the earbud in three-dimensional space. The IMU may include a three-axis accelerometer that can be used to determine the orientation. Using an accelerometer to determine the orientation of a device including or carrying the accelerometer is known in the art and is not further described herein. The IMU may also include a gyroscope, or three gyroscopes configured to determine the rotational speed about three mutually orthogonal axes. The gyroscope may be used additionally or alternatively to determine the orientation of the earbud. Using a gyroscope to determine the orientation of a device including or carrying the gyroscope is known in the art and is not further described herein. In one example, the IMU 72 is mounted on the PCB 70, but the IMU may be located elsewhere in the earbud or on the earbud.

[0037] In some examples, the proximity sensor 76 is used to detect when the earplug 20 is close to the user's skin. In one example, the proximity sensor 76 can be an infrared (IR) sensor or a capacitive sensor. The IR sensor can be used to detect close proximity to the skin, while the capacitive sensor can detect when the device contacts the skin. The IR sensor can be used to detect when the earplug is in close proximity to the skin. Since the earplug needs to be located in the ear when it is in the appropriate use position, part of the earplug will be in contact with the skin or close to the skin. Positioning the IR sensor in a part of the earplug that will be in contact with the skin or close to the skin thus allows the IR sensor to be used as a proximity sensor. In one example, the IR sensor is positioned so that it will detect the tragus. In another example, a distance sensor such as a time-of-flight sensor can be used to detect the distance between the wearable audio device and the desired position on the body (e.g., in the ear). Since the wearable audio device is appropriately located at a known position on the body (e.g., in the ear, on the ear, or elsewhere on the head), when the device is in place, the distance from the device to the appropriate position should be zero or close to zero.

[0038] The earbud 20 also includes a processor 74. In some examples, the processor 74 is configured to process the outputs of the IMU 72, the proximity sensor 76, the internal microphone 80, and the external microphone 81. Of course, the processor is generally involved in other processing required for the functionality of the earbud, such as processing digital sound files to be played by the earbud, as is obvious to a person skilled in the art. In one example, the processor is configured to estimate whether the wearable audio device is properly positioned on the user's body based on both the proximity sensor and the orientation sensor. In some examples, the processor's estimate of whether the wearable audio device is in place on the user's body is also based on the transfer function between the external microphone and the internal microphone. In one example, the processor is configured to estimate whether the user has completed the operation of the wearable audio device based on the internal microphone. In some examples, the proximity sensor has an output, and the processor's estimate of whether the wearable audio device is in place on the user's body is based in part on the level of the proximity sensor output. In one example, the processor's estimate of whether the wearable audio device is in place on the user's body is based in part on whether the proximity sensor output has reached a threshold level. For example, as the wearable audio device approaches the skin, the IR sensor output will increase. For any particular IR sensor, an output reaching a predetermined level may be equated with the sensor being within a predetermined distance of the skin.

[0039] In some examples, the processor estimates whether the wearable audio device is in place on the user's body, including multiple sequential steps. In one example, the first step is based on a proximity sensor and an orientation sensor, the second step is based on an internal microphone, and the third step is based on an internal microphone and an external microphone. In one example, the processor is configured to initiate a first set of wearable audio device functions based on the first step and the second step and initiate a second set of wearable audio device functions based on the third step. In one example, the functions of the second group are different from the functions of the first group. In some examples, the first and second groups of functions include at least one of the following: phone call answering capability, Bluetooth connection, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of user interface functions, and tuning of an active noise reduction system.

[0040] In some examples, the processor is configured to cause the electroacoustic transducer to generate a sound used in a high confidence determination of whether the wearable audio device is in place. In one example, this high confidence determination occurs only after the processor has made a first level lower confidence determination that the wearable audio device is in place on the user's body and the user has stopped operating the device. In one example, the processor is configured to calculate a transfer function between the external microphone and the internal microphone over a specified frequency range. In one example, the generated sound is part of a startup tone sequence of the wearable audio device.

[0041] In one example, the processor is configured to cause the electroacoustic transducer to generate two (or more) sounds of different frequencies for in-position / out-of-position determination. In one example, the first frequency is at least 1500 Hz (e.g., 1500 Hz-3000 Hz), and the processor is configured to determine whether the wearable audio device nozzle is blocked based at least in part on the frequency. In one example, the second frequency is less than 1000 Hz, and the processor is configured to determine whether the wearable audio device is in place on the user's body based at least in part on the frequency. When describing frequencies or frequency ranges in this article, it should be understood that the frequencies or ranges are approximate in many cases. When specifying a specific frequency, it should be understood that the actual frequency can be approximately or approximately the specified frequency. One reason is that if the actual frequency is different from the frequency, the results may not be significantly different.

[0042] Internal and external microphones can be used to detect sounds that are analyzed in different ways and for different purposes. There may be multiple functions that the microphone is used for. One is to determine when a user is adjusting a wearable audio device. The internal microphone can be monitored for this function. In one example, the monitoring can be at a frequency of up to 20 Hz, and the monitoring is passive; that is, the microphone is used to monitor the sound inside the wearable audio device without using a driver to play the sound. When the wearable audio device is an earplug, the internal microphone can be a feedback microphone located in the earplug nozzle. When the earplug is being operated (i.e., the earplug is being inserted into the ear or the earplug is being removed from the ear), the operation results in sounds in the range of up to 20 Hz. Therefore, detecting sounds in this range can be equivalent to detecting the operation of the earplug. In addition, when the earplug is inserted into the ear, the cavity defined by the driver, the earplug nozzle, the ear canal, and the eardrum becomes sealed, which results in a DC pressure peak. The DC pressure peak occurs at 0 Hz, but extends a little in this frequency range. Therefore, monitoring the internal microphone in the 0 Hz-20 Hz range is useful for estimating one or both of when the earbud is inserted in the ear and when it is no longer being operated.

[0043] The second function that the microphone can be used for is to make a higher confidence determination that the wearable audio device is in place on the user's body. Internal microphones and external microphones can be used for this function. In one example, the audio transfer function between the external microphone and the internal microphone is determined. The determination can be made at a frequency or frequency range where there is a good transfer function magnitude separation between the in-position and out-position (e.g., for earplugs, in the ear and outside the ear). In the example for earplugs, at frequencies in the range of 0Hz-1000Hz, the transfer function magnitude is different between the in-ear state and the out-of-ear state. Therefore, determining the transfer function in this frequency range can indicate with confidence whether the earplug is in the ear. The same principle can be used for other types of earplugs, for headphones, and for other types of wearable audio devices. The specific frequency or frequency range where there is a good transfer function magnitude separation between the in-position and out-of-position will be unique to any given product. So a "in-position" determination can be made at a frequency or frequency range where there is a good magnitude separation.

[0044] Another function that the microphone can be used for is to determine whether the nozzle of the earplug is blocked. When the earplug is being operated (e.g., when it is being placed in place in the ear or removed from the ear), the earplug nozzle may be blocked by a finger. Therefore, the determination of a blocked nozzle can be used as a lower confidence indication that the earplug is being operated and is therefore not in its proper use position. In one example, at a frequency range, the audio transfer function between the external microphone and the internal microphone has one value when the nozzle is blocked by a finger, and has another value when the wearable audio device is in free air or in a proper use position. For example, in the 1500Hz-3000Hz range, there is another response if the earplug nozzle is blocked by the thumb compared to the earplug in the ear or outside the ear (which has a similar response). As another example, the transfer function of the earplug in the frequency range of 0Hz-1000Hz can be low when the earplug is in free air, and can be higher when the nozzle is blocked (blocked by a finger or blocked by the earplug sealing in the ear canal). Thus, the transfer function can be an indication that the earbud is being operated, and can be an indication that the earbud is in the ear. This determination can be used with other determinations described herein to make a final decision as to whether the wearable audio device is in its proper position for use.

[0045] Figure 3The steps of a method 100 for detecting when a wearable audio device such as described above is properly in place on a user's body are illustrated. A proximity sensor is monitored for proximity detection events, step 102. An orientation sensor is monitored for proper wearable audio device orientation, step 104. In one example, both sensors are monitored simultaneously. Once the proximity sensor and the orientation sensor reach a threshold indicating a possible in-ear event, the internal microphone is monitored, step 105. In one example, microphone monitoring occurs in the 0Hz-20Hz range to detect when the wearable audio device is being operated by the user and when the earbud is inserted into the ear, as described elsewhere herein. Based on the proximity sensor, orientation sensor, and internal microphone monitoring, a low confidence determination is made as to whether the device is in place (i.e., in place on the user's body), step 106. In this non-limiting example, the device is an earbud, so the determination is whether the device is in place in the ear. If a low confidence in-ear determination is made, a first set of earbud functions is enabled, step 108. In some examples, earbud functions that may be enabled based on the low confidence determination include some or all of automatic power on / off, automatic pause / play of audio, automatic phone call answering capability, Bluetooth connection / reconnection, beamforming of a microphone carried by the wearable audio device, enabling of user interface functions, and tuning of an active noise reduction system. More specifically, in one example, the functions enabled after the low confidence determination are background functions, such as starting the processor, sensors, and Bluetooth. The result is that chips, sensors, and other aspects of the audio device that use power can remain turned off until a first level determination is made, thereby saving battery power.

[0046] The method 100 then moves to a second level, high confidence determination that the earbud is in place in the ear. The internal microphone and the external microphone are monitored, step 110. The driver is then enabled to play one or more tones, step 112. The purpose of playing the tone is to receive the tone at the microphone and determine whether the reception of the tone is as expected if the wearable audio device is in the proper use position and the user has stopped operating the wearable audio device. In the example of earbuds, microphones used for this purpose include internal microphones for feedback-based active noise reduction and external microphones for feedforward-based active noise reduction. Both types of microphones are known in the art of active noise reduction for wearable audio devices and are not further described here. The internal microphone is typically positioned so that it can sense sound in the cavity formed by the driver, the earbud nozzle, and the ear canal. An example is Figure 2 The external microphone is enabled to sense sounds outside the earbud housing. An example is Figure 2 In method 100, the transfer function (G sd ), step 114. Gsd The determination is described in U.S. Patent 10,462,551, entitled “Wearable Audio Device with Head On / Off State Detection,” issued on October 29, 2019, the entire disclosure of which is incorporated herein by reference for all purposes. The acoustic coupling of the driver to the microphone (as exemplified by the transfer function) will change (at least at one or more frequencies) when the earbud is in the ear compared to when the earbud is outside the ear. In one example, the processor of the wearable audio device is configured to both cause the electroacoustic transducer to generate sound and calculate the transfer function after the processor has made a first-level estimate of whether the wearable audio device is in place on the user's body. In one example, the transfer function is calculated over a specified frequency range, which may be in the range of 0 Hz-1000 Hz. In some examples, the sound played is part of a device startup series of tones of a type commonly found in electronic devices.

[0047] If the transfer function is as expected, then a high confidence in-ear determination is made, step 116. In one example, this high confidence second level determination also relates to the first level determination (proximity to the user, proper orientation, and microphone-based determination that the user has stopped operating the wearable audio device). In other words, when the device is in the correct physical position and the user is not operating the device (which is expected to occur once the user is satisfied that the device is in place), the device is considered (with low confidence) to be in its proper use position. In one example, as part of the second level high confidence determination, a G in the range of 0 Hz-1000 Hz is used to determine whether the user has stopped operating the wearable audio device. sd The blocked nozzle indicator is calculated at the same time. If the blocked nozzle calculation indicates that the nozzle is blocked, the high confidence decision can be delayed for a short time to ensure that the user has stopped operating the earplug. If there is still an indication of the in-ear state after this time, a high confidence decision is made. After making the second-level high confidence decision, the second set of earplug functions is enabled, step 118. In some examples, the functions of the second group are different from the functions of the first group. In one example, the functions of the second group include two functions involving interaction with the user (such as switches, capacitive touch sensors, and microphone beamforming) and tuning of the active noise reduction system. In some examples, multiple levels can run on different subsystems. For example, there may be a sensor subsystem used in the first low confidence level (using a proximity sensor, an orientation sensor, and an internal microphone) and a microphone subsystem used in the second high confidence decision (using an internal microphone and an external microphone). This will enable the microphone subsystem to be activated only when needed to save power. This design will require some message transmission between subsystems.

[0048] In some examples, monitoring of the proximity sensor and orientation sensor continues during the second-level decision process. One reason is to allow the wearable audio device to determine whether it has been removed from its position of use. For example, an earbud may be placed in the ear and then quickly removed. Or perhaps the user may keep adjusting it for a short time after it has been placed in the ear. By monitoring the proximity sensor and / or orientation sensor throughout the startup sequence, there may be more confidence in the determination that the device is in place and the user has completed the operation on it. In addition, continuous or periodic monitoring of the microphone and calculation of the transfer function may be used to confirm when the device has been moved from its position of use (e.g., removed from the ear). In some examples, the processor may enable a timer that may be used as part of the in-place / out-of-place determination. For example, if a high confidence determination is maintained for at least a short time (perhaps 1 second-2 seconds), the user may have greater confidence that he has stopped adjusting his position. Moreover, if the proximity sensor stops sensing proximity (which will occur when the earbud is removed from the ear), the timer may be used to turn off the earbud function only after a short time; this can help prevent erroneous out-of-place determinations that may annoy the user. For example, if the drivers and external microphones are disabled when the device is removed from its use location, a false positive could cut off a phone call or music or podcast against the user's will.

[0049] Figure 4 is an example of the transfer function (G) between the external microphone and the internal microphone of the earplug over a frequency range up to about 10,000 Hz (as described above). sd ). The solid line 134 illustrates the transfer function when the earplug is in the ear, while the dashed line 132 illustrates the transfer function when the earplug is in free space. It can be seen that at frequencies up to 1000 Hz, the magnitude of the transfer function is different. Therefore, determining the transfer function in the range of 0 Hz-1000 Hz indicates whether the earplug is in the ear.

[0050] Some examples of the present disclosure relate to detecting placement of headphones around or on the ears, where the ear cups are configured to rest over or on the ears, respectively. This placement detection can be based on proximity sensors, such as capacitive proximity sensors, microphones that sense sound inside the ear cups (as opposed to external ambient sound), and accelerometers (which can be, but need not be, a function of the IMU). As described below, in some examples, these sensors are used to make a first level determination that the headphones have been placed on the head, a second level determination that the headphones are in place on the head and ready for use, and a third level determination that the headphones have been removed from the head.

[0051] In some examples, wearable audio device (e.g., on-ear or over-ear headphones) placement detection relies on three types of sensors - capacitive sensors, microphones, and IMU / accelerometers. The placement detection algorithm can be considered to have multiple different stages. In the first stage, a capacitive sensor is used to detect whether one or both earmuffs of the earmuffs are above the user's ears or around the user's ears (i.e., the headphones are worn). In some examples, there is a single capacitive sensor in / on one of the earmuffs of the earmuffs. In other examples, there is a capacitive sensor in each of the two earmuffs. The capacitive sensor (e.g., an electrode or conductive member of the sensor) is shaped to pick up the most certain part of the ear using self-capacitance. The input from the capacitive sensor is monitored to determine whether / when a threshold indicating an event on the head is met. This can be considered as a lower confidence decision that the user has placed the headphones on the head. If an event on the head is detected, the first set of one or more headphone functions is enabled. The first set of headset functions may include one or more of the following: power on, automatic playback of audio, automatic phone call answering capability, Bluetooth connectivity, beamforming of an external microphone carried by the headset, and enabling of user interface (UI) functions. In one example, the functions enabled after a low confidence decision are background functions, such as starting the processor, sensors, and Bluetooth. The result is that chips, sensors, and other aspects of the audio device that use power can remain turned off until a first level decision is made, thereby saving battery power.

[0052] Then, once the input from the capacitive sensor has reached a threshold indicating an event on the head, a second phase is performed in which input from one or more internal microphones (e.g., feedback microphones used in active noise reduction or ANR systems) is monitored to detect "end of wearing", which is when the user has finished adjusting the headphones (i.e., the headphone cover is fully in place or "worn"). In one example, microphone monitoring occurs in the 0Hz-20Hz range to detect when the wearable audio device is being operated by the user. After making a second-stage determination that the user has stopped operating the headphones, a second set of earbud functions is enabled. In some examples, the functions of the second group are different from those of the first group. In one example, the functions of the second group include two functions involving interaction with the user (such as switches, capacitive touch sensors, and microphone beamforming) and tuning of the ANR system. In this regard, delays are used to provide measurements of customized ANR until the "end of wearing" state is detected. These measurements typically consist of playing a tone and measuring the response with a feedback microphone to measure the unique response of the user's ear canal. Customized ANR for headphones is described in U.S. Patent 10,937,410, the entire disclosure of which is incorporated herein by reference for all purposes.

[0053] In some examples, an inertial monitoring unit (IMU) or accelerometer is used to detect when the headset is laid flat / horizontally, indicating an off-head state (i.e., the headset is removed or "removed"). In response, the headset will be powered off. In some examples, input from a capacitive sensor is used in conjunction with input from the IMU / accelerometer to make a determination that the headset is in an off-head state. That is, input from a capacitive sensor can be monitored to detect when the user has removed the headset, but the headset will delay powering off until input from the IMU / accelerometer indicates that the headset is laid down horizontally. This can help prevent false off-head detection. Thus, for example, if the user only briefly removes the headset to readjust, the headset may not change any functionality until the IMU / accelerometer exceeds a certain threshold. This helps avoid interruption of headset functionality if, for example, the user has picked up one or both earmuffs (such as might be done if the headset is adjusted after having been worn).

[0054] In an example herein, a wearable audio device includes an electroacoustic transducer for generating sound; a proximity sensor configured to detect when the wearable audio device is close to a user; an internal microphone positioned within a housing of the wearable audio device and configured to detect sound in a cavity formed at least in part by a portion of the housing; and a processor configured to estimate whether the wearable audio device is in place on the user's body based on at least the proximity sensor, and to estimate whether the user has completed an operation of the wearable audio device based on the internal microphone.

[0055] In one example, the wearable audio device further includes: at least one earmuff configured to be located on or above the ear of the wearer when the wearable audio device is placed on the head, wherein the proximity sensor and the internal microphone are both positioned in the earmuff. In some examples, the proximity sensor includes a capacitive proximity sensor. In one example, the capacitive proximity sensor includes a conductive member inside the earmuff of the wearable audio device. In one example, the conductive member has a curved outer periphery that mimics the curve of a human auricle. In one example, the wearable audio device further includes: two earmuffs configured to be located on or above the ear of the wearer when the wearable audio device is placed on the head, wherein there is an internal microphone positioned within each earmuff, and wherein an estimate of whether the user has completed operation of the wearable audio device is based on the internal microphone positioned within each of the earmuffs.

[0056] In some examples, the processor is configured to initiate a first set of wearable audio device functions based on an estimate of whether the wearable audio device is in place on the user's body. In one example, the processor is configured to initiate a second set of wearable audio device functions based on an estimate of whether the user has completed an operation of the wearable audio device, wherein the second set of functions is different from the first set of functions. In one example, the first and second sets of functions include at least one of: phone call answering capability, Bluetooth connectivity, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of user interface functions, and tuning of an active noise reduction system.

[0057] In one example, the internal microphone is monitored at a frequency in the range of 0 Hz-20 Hz during at least an estimation of whether the user has completed operation of the wearable audio device. In one example, the proximity sensor has an output, and the processor's estimation of whether the wearable audio device is in place on the user's body is based in part on the level of the proximity sensor output. In some examples, after the processor estimates whether the wearable audio device is in place on the user's body, the processor is configured to enable a predetermined wearable audio device function. In one example, the predetermined wearable audio device function includes at least one of the following: telephone call answering capability, Bluetooth connection, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of a user interface function, and tuning of an active noise reduction system. In one example, the processor is further configured to initiate an active noise reduction (ANR) filter determination procedure after the estimation of whether the user has completed operation of the wearable audio device.

[0058] In some examples, the wearable audio device also includes an orientation sensor configured to detect the orientation of the wearable audio device. In one example, the processor is further configured to estimate whether the user has removed the wearable audio device from the user's body based on at least the orientation sensor. In one example, the processor is configured to reduce power usage based on an estimate of whether the user has removed the wearable audio device from the user's body. In one example, the estimate of whether the user has removed the wearable audio device from the user's body is also based on the proximity sensor. In some examples, the processor estimate of whether the wearable audio device is in place on the user's body is based on the proximity sensor and the orientation sensor. In one example, the orientation sensor includes an accelerometer.

[0059] Figure 51 is a schematic cross-sectional view of a supra-aural headset 150 in place on a user's head 152. Only one earmuff 160 is shown positioned on the ear 154, and an internal cavity 161 is defined between the earmuff 160 and the head 152 / ear 154. In most but not all cases, the headset includes two substantially identical earmuffs, one on each ear, and mechanically connected by a headband 162. For supra-aural headsets, the earmuffs are designed to rest on the outer ear or auricle rather than on it. The headset 150 includes an electroacoustic transducer 170 and an internal microphone 164, which are positioned so that the transducer 170 generates sound pressure in the cavity 161 and the microphone 164 senses the sound pressure in the cavity 161. An external microphone 166 is positioned so as to sense sound outside the earmuff 160. A processor 168 is operatively connected to the microphones 164 and 168 and the transducer 170. The headset 150 also includes a proximity sensor (not shown) configured to detect when the headset is close to the user. Figure 6 An exemplary proximity sensor is further described.Other typical hardware and functionality of a headset is not shown or described.

[0060] Figure 6 1 is an interior view of a frame 180 of an earmuff for an over-ear headphone. The frame edge 182 will carry a cushion (not shown). Area 184 is the area where the electroacoustic transducer (not shown) is located. A conductive member 186 is located on the inner surface of the frame 180. The conductive member 186 is configured as a capacitive proximity sensor for detecting the proximity of the earmuff to the user's skin (e.g., ear or auricle). When the conductive member 186 is close to the skin, the capacitance changes. In some cases, the capacitance level indicates the proximity of the conductive member 186 to the skin. Therefore, the capacitance level can be used as a threshold indication of the placement of the earmuff on or above the ear. A processor (not shown) is configured to measure the capacitance change via a trace 188 leading to the processor. The position and shape of the conductive member 186 in the earmuff makes it configured to cover at least a portion of the outer ear or auricle of a person when the earmuff is worn. The size, position and shape of the conductive member 186 therefore increase the likelihood that the ear will be sensed. In some examples, sensing of the ear is used as a first, low-level indication that the user has placed the headset on the head. As explained in more detail elsewhere herein, in some examples, the processor is configured to enable one or more functions of the headset based at least in part on the proximity sensor. The proximity sensor can be located in one ear cup, or in both ear cups.

[0061] Figure 7Method 190 illustrating the operation of a headphone wearing and removal detection method. The method involves a first determination as to whether the headphone has been placed on the head, a second determination as to whether the headphone has been fully worn (in other words, the headphone is in place on or over the ear and ready for use), and a third determination as to whether the headphone has been removed or removed from the head.

[0062] In step 192, the processor is monitoring a proximity sensor to detect proximity to the head. In some examples, there is a proximity sensor that is part of only one of the two earmuffs. In other examples, there are two proximity sensors, one of which is part of each of the two earmuffs. When the head is sensed, a determination is made on the head, step 194, and then a first set of headphone functions is enabled, step 196. This determination on the head can be considered a low confidence determination because it is based only on a single sensor. In some examples, the first set of headphone functions includes one or more of the following: power on, automatic playback of audio enabled, automatic phone call answering enabled, Bluetooth connection, beamforming of a microphone carried by the headset, and enabling of UI functions. In one example, the enabled functions are background functions, such as starting a processor, sensors, and Bluetooth. The result is that the chip, sensors, and other aspects of the headset that use power can remain turned off until a determination is made on the head. This saves power.

[0063] The second level decision about the headset being fully worn and ready for use is based on monitoring one or more internal microphones of one or both earmuffs, step 198. The microphone can be a feedback microphone of the headset ANR system. In one example, at least one microphone in each earmuff is monitored to detect when the user stops touching or moving the two earmuffs, which usually occurs within a short time after the headset is placed on the head, while the user is adjusting the position of the earmuffs to obtain more comfortable and better audio results. In one example, the microphone output is monitored in the range of about 0Hz-20Hz. Part of the reason for selecting this range is that the sound / vibration in this range is generally not affected by the ANR system, so the expected response is the same regardless of whether any ANR is enabled. When a low threshold of the sound monitored in the 0Hz-20Hz range is reached, a wearing end decision is made, step 200, and a second set of headset functions is enabled, step 202. In some examples, the functions of the second group are different from the functions of the first group. In one example, the functions of the second group include two functions involving interaction with the user (such as switches, capacitive touch sensors, and microphone beamforming) and tuning of the ANR system. In some examples, a delay is used to provide a measurement of customized ANR until the "wearing end" state is detected. These measurements typically consist of playing a tone and measuring the response with a feedback microphone to measure the unique response of the user's ear canal.

[0064] The third sensed state is when the user has removed or removed the headset. In some examples, the proximity sensor and one or both of the inertial monitoring unit (IMU) or accelerometer are monitored, step 204, to detect when the headset is laid flat / horizontally, indicating a determination made by the processor about an off-head state (i.e., the headset is taken off or "removed"), step 206. In response, the processor is enabled to disable one or more headset functions, step 208. In some examples, the headset will be powered off in response to off-head detection. In an example in which an accelerometer is used to detect orientation, when the headset is on the head in a normal upright head position, there is a steady-state acceleration due to gravity in one direction (which may be referred to as the "vertical" direction) and no acceleration in two orthogonal directions. When the headset is removed and laid more or less flat, the steady-state acceleration will not be on the initial axis. The processor is therefore able to use the accelerometer to determine whether the headset has been laid flat.

[0065] In some examples, input from a proximity sensor (e.g., a capacitive proximity sensor) is used in conjunction with input from an IMU / accelerometer to make a determination that the headset is in an off-head state. In some examples, input from the proximity sensor is monitored to detect when the user has removed the headset, but the headset delays powering off until input from the IMU / accelerometer indicates that the headset is placed horizontally. This can help prevent false off-head detections. Thus, for example, if the user only briefly removes the headset or earmuffs to readjust them, the headset may not change any functionality until the proximity sensor exceeds a certain threshold. This helps avoid interruption of headset functionality if, for example, the user has picked up one or both earmuffs (such as might be done if adjusting the headset after having been worn, or if the head is briefly tilted to the side).

[0066] When a process is represented or implied in a block diagram, the steps may be performed by one element or multiple elements. The steps may be performed together or at different times. The elements performing the activity may be physically identical or close to each other, or may be physically separated. An element may perform the actions of more than one box. The audio signal may be encoded or not, and may be sent in digital or analog form. In some cases, conventional audio signal processing equipment and operations are omitted from the figure.

[0067] Examples of the systems and methods described herein include computer components and computer-implemented steps that will be apparent to those skilled in the art. For example, it will be appreciated by those skilled in the art that computer-implemented steps can be stored as computer-executable instructions on a computer-readable medium, such as, for example, a floppy disk, a hard disk, an optical disk, a flash ROM, a non-volatile ROM, and a RAM. In addition, it will be appreciated by those skilled in the art that computer-executable instructions can be executed on various processors, such as, for example, a microprocessor, a digital signal processor, a gate array, and the like. For ease of illustration, not every step or element of the above-mentioned systems and methods is described herein as part of a computer system, but it will be appreciated by those skilled in the art that each step or element can have a corresponding computer system or software component. Therefore, it is within the scope of the present disclosure to implement such computer systems and / or software components by describing their corresponding steps or elements (i.e., their functions).

[0068] A number of specific implementations have been described. However, it should be appreciated that additional modifications may be made without departing from the scope of the inventive concepts described herein, and therefore, other examples are within the scope of the following claims.

Claims

1. A wearable audio device comprising: an electroacoustic transducer for generating sound; a proximity sensor configured to detect when the wearable audio device is in proximity to a user; an internal microphone positioned within a housing of the wearable audio device and configured to detect sound in a cavity formed at least in part by a portion of the housing; and A processor is configured to estimate whether the wearable audio device is in place on the user's body based on at least the proximity sensor, and to estimate whether the user has completed an operation of the wearable audio device based on the internal microphone.

2. The wearable audio device according to claim 1, further comprising: At least one ear cup configured to be positioned on or over a wearer's ear when the wearable audio device is placed on the head, wherein both the proximity sensor and the internal microphone are positioned in the ear cup.

3. The wearable audio device of claim 1, wherein the proximity sensor comprises a capacitive proximity sensor.

4. The wearable audio device of claim 3, wherein the capacitive proximity sensor comprises a conductive member inside an ear cup of the wearable audio device.

5. A wearable audio device according to claim 4, wherein the conductive member is configured to cover at least a portion of a human auricle.

6. The wearable audio device according to claim 1, further comprising: Two ear cups, the two ear cups are configured to be located on or above the ears of the wearer when the wearable audio device is placed on the head, wherein there is an internal microphone positioned within each ear cup, and wherein an estimate of whether the user has completed an operation of the wearable audio device is based on the internal microphone positioned within each of the ear cups.

7. The wearable audio device of claim 1, wherein the processor is configured to initiate a first set of wearable audio device functions based on the estimation as to whether the wearable audio device is in place on the user's body.

8. The wearable audio device of claim 7, wherein the processor is configured to initiate a second set of wearable audio device functions based on the estimate as to whether the user has completed operation of the wearable audio device, wherein the second set of functions is different from the first set of functions.

9. The wearable audio device of claim 8, wherein the first and second groups of functions include at least one of: telephone call answering capability, Bluetooth connectivity, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of user interface functions, and tuning of an active noise reduction system.

10. The wearable audio device of claim 1, wherein the internal microphone is monitored at a frequency in the range of 0 Hz-20 Hz during at least the estimation as to whether the user has completed operation of the wearable audio device.

11. The wearable audio device of claim 1 , wherein the proximity sensor has an output, and the processor's estimate of whether the wearable audio device is in place on the user's body is based in part on a level of the proximity sensor output.

12. The wearable audio device of claim 1, wherein after the processor estimates whether the wearable audio device is in place on the user's body, the processor is configured to enable a predetermined wearable audio device function.

13. The wearable audio device of claim 12, wherein the predetermined wearable audio device functionality comprises at least one of: telephone call answering capability, Bluetooth connectivity, beamforming of a microphone carried by the wearable audio device, playback of audio received from an external audio source, enabling of user interface functionality, and tuning of an active noise reduction system.

14. The wearable audio device of claim 1, wherein the processor is further configured to initiate an active noise reduction (ANR) filter determination procedure after an estimation that the user has completed operating the wearable audio device.

15. The wearable audio device of claim 1, further comprising an orientation sensor configured to detect an orientation of the wearable audio device.

16. The wearable audio device of claim 15, wherein the processor is further configured to estimate whether the user has removed the wearable audio device from the user's body based at least on the orientation sensor.

17. The wearable audio device of claim 16, wherein the processor is configured to reduce power usage based on the estimate as to whether the user has removed the wearable audio device from the user's body.

18. The wearable audio device of claim 16, wherein the estimate as to whether the user has removed the wearable audio device from the user's body is further based on the proximity sensor.

19. The wearable audio device of claim 15, wherein the processor estimate as to whether the wearable audio device is in place on the user's body is based on the proximity sensor and the orientation sensor.

20. The wearable audio device of claim 15, wherein the orientation sensor comprises an accelerometer.

21. The wearable audio device of claim 1, further comprising: Two ear cups configured to be positioned on or over a wearer's ears when the wearable audio device is placed on the head, wherein a proximity sensor is present in each ear cup.

Images