Method for in-ear or on-skin detection of a wearable device and corresponding sensor package

Abstract

The invention provides a robust and accurate method for in- ear or on-skin detection of a wearable device (4), or a non- wearable biometric device, with a sensor package (2), comprising: - emitting light from a first light emitter (10) with a first dominant emission wavelength; - measuring a reflectance of the first light emitter's (10) emitted light from a target (6) using a first light detector (14); - emitting light from a second light emitter (12) with a second dominant emission wavelength different from the first dominant emission wavelength; - measuring a reflectance of the second light emitter's (12) emitted light from said target (6) using the first light detector (14) or a second light detector (16) or using both the first light detector (14) and a second light detector (16); - monitoring temperature with the help of a temperature sensor (22) at the sensor package (2) location; and - processing data from the first light detector (14) and, if present, the second light detector (16), the temperature sensor (22), and conducting optional additional checks to determine the in-ear or on-skin status of the wearable device (4).

Description

[0001] Method for in-ear or on-skin detection of a wearable device and corresponding sensor package

[0002] DESCRIPTION

[0003] TECHNICAL FIELD

[0004] The present invention is situated in the field of in-ear or on-skin detection technology, speci fically targeting the accurate determination of the in-ear or out-of-ear status of wearable devices , such as earbuds , hearing aids , and in-ear biometric devices . In particular, the invention provides a method for in-ear detection of a wearable device , a corresponding sensor package , and wearable device with a sensor package .

[0005] BACKGROUND

[0006] In recent years , wearable devices have become increasingly popular and have found applications in various domains , including audio playback, health monitoring, and communication . One essential aspect of in-ear wearable device functionality is the ability to detect whether the device is positioned within the user ' s ear or has been removed . Accurate in-ear detection is imperative for ef ficient power management , seamless user experience , and precise biometric measurements .

[0007] Conventional in-ear detection methods have primarily relied on single-wavelength optical reflectance measurements . While these methods have provided some level of accuracy, they are not without limitations . They often result in false positives and false negatives , leading to suboptimal user experiences .

[0008] SUMMARY

[0009] The task of the present invention is to provide a robust and accurate method for in-ear detection of a wearable device , as well as a sensor package tailored to achieve this task . The invention is primarily tasked with resolving the technical problem of distinguishing whether a wearable device is properly positioned within the ear or has been removed, thereby enabling various functionalities and optimi zations .

[0010] According to claim 1 , the method-related obj ective is met by a method for in-ear or on-skin detection of a wearable device , comprising :

[0011] • emitting light from a first light emitter with a first dominant emission wavelength;

[0012] • measuring a reflectance of the first light emitter' s emitted light from a target using a first light detector ;

[0013] • emitting light from a second light emitter with a second dominant emission wavelength di f ferent from the first dominant emission wavelength;

[0014] • measuring a reflectance of the second light emitter' s emitted light from said target using the first light detector or a second light detector or using both the first light detector and a second light detector ;

[0015] • monitoring temperature with the help of a temperature sensor at the sensor package location; and

[0016] • processing data from the first light detector and, i f present , the second light detector, the temperature sensor, and conducting optional additional checks to determine the in-ear or on-skin status of the wearable device .

[0017] As explained further below, this includes the ( time- multiplexed) case of only one light detector or of two light detectors acting as one .

[0018] The present invention of fers an innovative solution to the challenges associated with in-ear ( generally : on-skin) detection . By combining dual-band light reflectance measurements , in particular in the near-infrared (NIR) wavelength range , temperature monitoring, and a range of optional checks , the invention signi ficantly improves the accuracy and reliability of in-ear detection for wearable devices . The use of two distinct wavelengths , coupled with the monitoring of temperature changes , allows for the precise identi fication of in-ear skin or tissue , reducing the likelihood of false positives and false negatives .

[0019] The monitored temperature at the sensor package location, compared to the user ' s body temperature ( typically in the range of 33 to 37 ° C ) , further refines the in-ear detection process .

[0020] In a preferred embodiment , not only is the current temperature monitored, but the temporal convergence of the measured temperature towards the predefined common body temperature range is checked .

[0021] This underscores the pivotal role of temperature monitoring and optional checks in correcting false positives associated with spectral approaches for skin identi fication . By evaluating both temperature and additional checks , the method ensures that even challenging scenarios , where skin and nonskin appear indistinguishable via infrared ( IR) reflectance , can be accurately resolved .

[0022] Preferably, the dominant emission wavelengths used in the method for in-ear detection fall within the near-infrared

[0023] (NIR) range . The first light emitter typically has a dominant emission wavelength of approximately 850 nm, while the second light emitter has a dominant emission wavelength of approximately 940 nm . This selection of NIR wavelengths of fers advantages , including compatibility with human skin, which is more transparent in the NIR range , thus enhancing the accuracy of in-ear detection . In principal , this can be done with many other wavelengths . Preferred values 940 / 850 nm are chosen for a concrete example embodiment mostly for reasons of cost / si ze and compatibility with pre-existing proximity devices . Higher wavelengths can of fer better performance but yield more cost / si ze . Lower wavelengths can of fer better per- formance but are visible and not attractive to customers who want no visible light .

[0024] Relative and absolute reflectance levels of the di f ferent wavelengths of light can be used to distinguish skin at close range from many other materials and surfaces at close range . In order to further improve the distinction of skin and nonskim materials , several additional checks can be used .

[0025] By monitoring ambient light levels , the method adds an extra layer of accuracy to the process . It can distinguish between indoor and outdoor scenarios based on ambient light conditions , improving the overall accuracy of in-ear detection . The introduction of this check ensures that the method remains adaptable to di f ferent environments , providing reliable detection in various conditions .

[0026] Another critical aspect to the in-ear detection process is the monitoring of proximity levels between the wearable device and the target ( typically the user ' s ear ) . Proximity levels provide insight into the closeness of the device to the ear, a crucial indicator of in-ear status . By assessing proximity levels , this method further refines the accuracy of in-ear detection, ensuring that it accurately identi fies whether the device is in contact with the ear .

[0027] Incorporating noise level assessment as an optional check adds an additional layer of precision to the in-ear detection process .

[0028] The accuracy of in-ear detection may be further enhanced by introducing the assessment of motion or variance events associated with the measured optical and thermal signals . For instance typical variance events of an in-ear insertion have di f ferent time signatures than that of an in-pocket insertion .

[0029] In an embodiment the additional checks may include measuring blood flow in in-ear blood vessels , preferably by using pho- toplethysmography ( PPG) . PPG is a non-invasive technique that involves illuminating the blood vessels with light and measuring the modulating ef fect caused by the periodic blood flow . By measuring blood flow, the method can reliably confirm in-ear status by directly assessing physiological factors .

[0030] In summary, the optional checks , including monitoring ambient light levels , proximity to the target , noise levels , motion events , variance events , and even blood flow in in-ear blood vessels using photoplethysmography ( PPG) , further enhance the system ' s performance and help correct any potential errors in skin identi fication based on spectral approaches .

[0031] The invention is preferably implemented through a sensor package , which includes a first light emitter, a second light emitter, a first light detector, a second light detector, and a temperature sensor . Processing circuitry within the sensor package analyzes data from these components and conducts the necessary checks to determine the in-ear status of the wearable device .

[0032] The disclosed method and sensor package of fer a comprehensive solution to in-ear detection challenges , with applications in earbuds , hearing aids , in-ear biometric devices , and other wearable technologies . This technology ensures improved user experiences and paves the way for more ef ficient power management and reliable biometric measurements in the world of wearable devices .

[0033] Summari zing, the primary obj ective of the present invention is to provide an improved method for in-ear detection of a wearable device , as well as a corresponding sensor package designed to achieve highly accurate in-ear detection . The invention aims to address the technical problem of identi fying whether a wearable device , such as an earbud, hearing aid, or in-ear biometric device , is positioned within the ear, which is crucial for purposes such as optimi zing power consumption, ensuring proper functionality, and facilitating biometric measurements . To achieve this obj ective , the invention combines dual-band near-infrared (NIR) reflectance measurements (preferably using absolute and relative levels ) , temperature monitoring (preferably using absolute and gradient values ) , and optional checks to enhance the accuracy of in-ear detection while minimi zing false positives . Additionally, the invention provides the option to employ motion detection and photoplethysmography ( PPG) for further refinement in identifying the in-ear status of the wearable device . By introducing this innovative approach, the invention of fers a reliable and energy-ef ficient solution that outperforms existing methods , ensuring seamless and accurate in-ear detection for a variety of wearable devices , thereby enhancing user experience and device functionality .

[0034] While the main area of application is in-ear detection as described above , the method and the sensor package may also be used for general near-skin or on-skin detection of a wearable device , for example a smartwatch, smart glasses , or any other device supposed to be in contact with or resting against a part of a human body .

[0035] BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Preferred embodiments of the invention will now be described with reference to the accompanying drawings .

[0037] FIG . 1 shows a schematic top view of an exemplary sensor package for in-ear detection of an in-ear-wearable device .

[0038] FIG . 2 shows a corresponding side view of a sensor package with a target above it .

[0039] FIG . 3 shows an in-ear wearable device with a sensor package for in-ear detection .

[0040] FIG . 4 shows a variant of a sensor package for in-ear detection with an optical barrier for cross-talk reduction . FIG. 5 shows a flow diagram illustrating an algorithm for in- ear detection of a wearable device with the help of a sensor package .

[0041] FIG. 6 shows a further flow diagram regarding timing aspects of an algorithm for in-ear detection.

[0042] FIG. 7 shows a further flow diagram regarding voting and filtering aspects of an algorithm for in-ear detection.

[0043] DETAILED DESCRIPTION

[0044] FIG. 1 shows a schematic top view of an exemplary sensor package 2 for in-ear detection of an in-ear wearable device 4, such as an earplug (earbud) , a hearing aid, or an in- ear biometric device. FIG. 2 shows a corresponding side view with a target 6 (such as inner ear skin) above the sensor package 2. An in-ear wearable device 6 with such a sensor package 2 is illustrated purely schematically in FIG. 3.

[0045] The sensor package 2 is designed to detect if the in-ear wearable device 4 rests within an ear as intended or has been removed / released and therefore is outside of the ear. Such in-ear detection is crucial, among others, in order to save energy consumption (i.e. for power savings) , properly manage autoplay / autostop in sound and media applications (earbud replay) , and / or to start / pause biometric readings.

[0046] To this end, the sensor package 2 comprises a base substrate 8 on which a first light emitter 10 (or light source) and a second light emitter 12 are arranged to emit light beams towards a common target 6. Each of the light emitters 10, 12 has a spectral characteristic which is preferably centered around a dominant emission wavelength, wherein both emission wavelengths are different. Preferably, both emission wavelengths are in the infrared (IR) range, in particular in the near-infrared (NIR) range. In this case, the respective light emitter 10, 12 may be called IR emitter or NIR emitter. In a particularly preferred embodiment the first light emitter 10 is a low IR emitter with a dominant emission wavelength of, e.g., 850 nm, and the second light emitter 12 is a high IR emitter with a dominant emission wavelength of, e.g., 940 nm. In a practice-oriented implementation the respective light emitter 10, 12 may be a vertical-cavity surface-emitting laser (VCSEL) arranged on a die or chip or directly on the base substrate 8.

[0047] Each light emitter 10, 12 is assigned a corresponding light detector 14, 16 (or light receiver) whose spectral sensitivity or response matches the dominant emission wavelength of the associated light emitter 10, 12. Each light detector 14, 16 is arranged on the base substrate 8 such as to receive light emitted by the associated light emitter 10, 12 and reflected by a target 6. That is, the first light detector 14 is arranged and configured to receive light emitted by the first light emitter 10 and reflected by the target 6, and the second light detector 16 is arranged and configured to receive light emitted by the second light emitter 12 and reflected by the target 6. The closer the target 6 and the higher the target's reflectance in the wavelength region of interest, the higher the measured reflectance signal will be. In practical implementations, the light detectors 14, 16 may be photodetectors such as photodiodes.

[0048] There are two basic options for collecting the reflectance of the two emitted wavelengths of light which can either be implemented on their own or combined with each other:

[0049] 1) Time Multiplexing: Each light emitter 10, 12 pulses sequentially and the relative signal levels are determined by comparing the reflected light at different times. This can be done fast on the scale of most device / skin interactions and thus negative effects of time separation are minimal. This allows the light detectors 14, 16 to have the same or overlapping spectral sensitivity bands. It is possible to use a single broadband detector ( instead of two detectors ) which covers both dominant emission wavelengths . In this case there is , for example , only the first light detector 14 but no second light detector 16 . It is also in time-multiplexing possible to combine the signals from the two light detectors 14 , 16 into one which basically means that they act as one (with enlarged detection area ) . It is also possible to arrange the light detectors ( typically photodiodes ) and emitters in a way that a distance to target can be determined, which can be an additional check used to help with the in-ear or skin identi fication . It is also possible to arrange the light detectors and emitters in a way that the tilt of the target can be determined, which can be an additional check used to help with the in-ear or skin identi fication .

[0050] 2 ) Spatial Multiplexing : There are multiple light detectors 14 , 16 , each one preferably with an optical filter to select one wavelength range of interest . The relative levels are determined by comparing the signal levels of the di f ferent detectors . This allows them to be detected at exactly same time . Another alternative is to have emitter / detector pairs that are physically or optically separated by barriers such that they cannot crosstalk with each other . Then, the receiver does not have to be as selective about the wavelength band .

[0051] Optical barriers may not only be suited for reducing crosstalk between emitter / detector pairs but also for reducing crosstalk between an emitter and an associated detector within a pair . FIG . 4 provides in a purely schematic manner a side view of a sensor package 2 with such an optical barrier 18 for either scenario .

[0052] In combination, the two emitter / detector pairs together with integrated circuits within a chip or die 20 on the base substrate 8 form a dual-band optical proximity sensor, in a particular a NIR proximity sensor, which makes it possible to detect the approach to or release from a target 6 . The two di f ferent operating wavelengths also allow for, to some extent , di f ferentiating among various targets 6 , in particular for identi fying the target as in-ear skin or tissue . To this end, absolute and / or relative reflectance signal levels and / or ratios may be obtained / sensed and compared to predetermined thresholds or against database entries . This can be achieved in a relatively fast measurement cycle , also called skin detection phase .

[0053] However, due to the relative simplicity of the measurement there may be false positives ( device deemed to be in the ear while actually outside ) and false negatives ( device deemed to be outside the ear while actually inside ) . Usually, in particular in the case of earbuds , false negatives can be more annoying to the user and even less acceptable than false positives .

[0054] In order to improve in-ear detection, the sensor package 2 further comprises a temperature sensor 22 to sense or measure the temperature and its temporal evolution or change at the sensor location, preferably directly at the sensor package 2 or in its immediate vicinity . In particular, the sensing algorithm implemented in the sensor package 2 compares the measured temperature to the well-known human body temperature of , typically, 33 to 37 ° C, also taking into account typical lower temperatures seen given the expected distance from skin to sensor, and expected material conductivities of surrounding product system . In other words , by monitoring the convergence ( or non-convergence ) of the measured temperature towards said body temperature an independent assessment of in- ear or outside-ear status of the wearable device 4 with the integrated sensor package 2 may be achieved . In particular, this temperature change monitoring can be used to veri fy or falsi fy a prior assessment based on the dual-band reflectance measurement .

[0055] This is further explained with reference to a process scheme ( flow diagram) shown in FIG . 5 . The leftmost box in the scheme represents an initial outside-ear status , in which the sensing algorithm implemented in the sensor package 2 is armed for a new proximity detection, wherein the absolute signal level of at least one of the light emitter / detector pairs may be used for proximity detection with respect to any target . This rules out cases where the wearable device 4 is not resting against anything . In a next step, one box further to the right , by comparing the signal ratio of the light emitter / detector pairs , in particular for the preferred 850 nm versus 940 nm wavelengths , a first assessment with respect to the material constitution of the target 6 can be made .

[0056] This rules out many cases where the targe consists of or comprises inorganic material . However, i f the material may be classi fied as in-ear tissue or skin the process proceeds with further validation steps to be described below . The two boxes described so far represent the fast optical skin detection phase .

[0057] As a side-note , VCSEL / package dri ft monitoring using nonproximity states may be evaluated for reducing dri ft ef fects on the measured ratio . So , for example , when in a state of known or inferred no-target , the crosstalk levels of the device can be used to check whether the emitter signal levels have dri fted relative to one another . This information can be used to adj ust calibration .

[0058] Turning now to the further validation steps , the most important one is illustrated in the middle box of the diagram . It comprises monitoring of the measured temperature over time and comparing it against the usual body temperature as described above . This rules out room-temperature organ- ic / inorganic cases ( targets ) where the temperature does not converge against the body temperature .

[0059] The rate of change of temperature is in general dependent on the design of the in-ear wearable device 4 . In particular, it may depend on the thermal conductivity of a plastic housing and / or overmold . Therefore , the check or validation based on the temperature-change is usually considerably slower than the prior optical assessment . As illustrated by a further ( second-rightmost ) box in the process flow of FIG . 5 , optional further checks , or valida- tions / veri fications , may be conducted in addition to the temperature check . This may include sensing the level or intensity of ambient light with either the light detectors 14 , 16 ( in a time-multiplexed measurement mode , alternating with the reflectance measurement ) or with additional , dedicated ambient light detectors . This may alternatively or additionally include assessing or sensing a level of noise , or a motion associated with at least one of the measured optical and / or thermal signals or with a separately measured signal . The underlying idea is that human movement will generally produce noisier signals in comparison with an outside-ear storage of the wearable device 4 . In other words , this rules out excessively "quiet" organic / inorganic cases or targets .

[0060] A similar idea underlies an optional photoplethysmography ( PPG) measurement of the blood movement in in-ear bloodvessel . That is , the blood vessels are illuminated by a light source , and the modulating ef fect of the periodic blood flow is measured by a light detector in reflection and transmission mode . Again, at least one of the light emitters 10 , 12 and / or light detectors 14 , 16 may be used in a time- multiplexed measurement setup, or an additional , dedicated PPG light source and light detector may be employed . This provides a very reliably in-ear detection at the expense of a more complex setup .

[0061] When one of the checks concludes that the target 6 is not skin or tissue , the decision algorithm goes back to the start ( at the leftmost box ) into new proximity detection mode . However, i f the chain of checks including the final check concludes that the target 6 is ( in-ear ) skin or tissue and therefore the wearable device 4 is deemed to be placed in- ear, the sensing algorithm of sensor package 2 enters a release detection mode which basically works in an analogous way as the proximity detection mode . Like the temperature check, the optional further checks are usually slow in comparison to the dual-band reflectance or proximity check but in typical applications help reduce power or other ef fects of false positives . While in the flow diagram of FIG . 5 the additional checks follow the temperature check, the order of the decision flow may be reversed . In practical implementations , the measurements required for the decisions can be performed simultaneously or in overlapping time intervals . The di f ferent checks ( signal ratios , signal levels , temperature , temperature convergence , variance , PPG evidence , etc . ) can also be applied di f ferent importance weights and di f ferent confidence values , so that that it is a mathematical combination of the checks that ultimately decides skin / no-skin . This mathematical combination of the check can also include certain high-confidence overrides , where one check results in high confidence enough result to make decision alone .

[0062] The flow diagram in FIG . 6 provides further details about a preferred timing regime of the fast and the slower measurements .

[0063] The flow diagram of FIG . 7 provides further details about a preferred embodiment of the decision-making process . A voting module 24 implemented in the circuitry 28 of the sensor package 2 receives proximity data from the first emitter / detector pair, as well as proximity data from the second emitter / detector pair . Further input comprises temperature data from the temperature sensor 22 . On the basis of the current values of these input data and on previously provided model parameters the voting module 24 periodically performs a weighted voting whether or not the wearable device 4 containing the sensor package 2 is in-ear or not . The resulting preliminary vote may be considered a noisy signal which is fed into a temporal filter 26 which also receives as input a time series or a continuous signal derived from at least the temperature data and possibly other input signals as well . The temporal filter 26 may then be considered smoothing or filtering the noisy voting signal by taking into account time- dependent ef fects represented by signal gradients and / or time integrals . By applying previously provided thresholds to the filtered voting signal a more substantiated or accurate vote or decision whether or not the wearable device 4 containing the sensor package 2 is in-ear or not can be obtained .

[0064] In summary, by combining reflectance measurements from two emitter / detector pairs with di f ferent wavelengths , in particular NIR wavelength, with temperature measurements , human skin (within the ear ) can be identi fied more accurately . The identi fication accuracy can even be improved by the use of additional signal monitoring such as ambient light levels , proximity levels with respect to a target , motion detection, and biometric detections to correct false positives .

[0065] LIST OF REFERENCE SIGNS sensor package 2 wearable device 4 target 6 substrate 8 first light emitter 10 second light emitter 12 first light detector 14 second light detector 16 optical barrier 18 die 20 temperature sensor 22 voting module 24 temporal filter 26 circuitry 28

Claims

CLAIMS1. A method for in-ear or on-skin detection of a wearable device (4) , or a non-wearable biometric device, with a sensor package (2) , comprising:• emitting light from a first light emitter (10) with a first dominant emission wavelength;• measuring a reflectance of the first light emitter' s(10) emitted light from a target (6) using a first light detector (14) or a second light detector (16) or using both the first light detector (14) and a second light detector (16) ;• emitting light from a second light emitter (12) with a second dominant emission wavelength different from the first dominant emission wavelength;• measuring a reflectance of the second light emitter' s(12) emitted light from said target (6) using the first light detector (14) or a second light detector (16) or using both the first light detector (14) and a second light detector (16) ;• monitoring temperature with the help of a temperature sensor (22) at the sensor package (2) location; and• processing data or signals from the first light detector(14) and, if present, the second light detector (16) , and the temperature sensor (22) , and optionally conducting additional checks to determine the in-ear or on-skin status of the wearable device (4) .

2. The method of claim 1, further comprising comparing the measured temperature to the body temperature of a user to verify the in-ear status.

3. The method of claim 2, further comprising checking temporal convergence of the measured temperature towards a predefined common body temperature, preferably in the range from 33 to 37°C.

4. The method of any of the preceding claims, wherein the optional checks include monitoring ambient light levels.

5. The method of any of the preceding claims, wherein the additional checks include monitoring proximity levels between the wearable device (2) and the target (6) .

6. The method of any of the preceding claims, wherein the additional checks include assessing noise levels.

7. The method of any of the preceding claims, wherein the additional checks include detecting motion and / or variance events associated with measured optical and / or thermal signals .

8. The method of any of the preceding claims, wherein the additional checks include measuring blood flow in in-ear blood vessels, preferably by using photoplethysmography(PPG) .

9. The method of any of the preceding claims, wherein the temperature monitoring and the additional checks correct false positives in spectral approaches for skin identification .

10. The method of any of the preceding claims, further comprising performing time-multiplexed and / or spatially multiplexed measurements for light detection with respect to the two different dominant emission wavelengths.

11. The method of any of the preceding claims, wherein the wearable device (4) is an earbud, a hearing aid, or an in-ear biometric device.

12. The method of any of the preceding claims, wherein the first dominant emission wavelength and the second dominant emission wavelength are in the near-infrared range.

13. The method of any of the preceding claims, wherein the first light emitter (10) has a dominant emission wavelength of approximately 850 nm and the second light emitter (12) has a dominant emission wavelength of approximately 940 nm.

14. A sensor package (2) for in-ear or on-skin detection, comprising :• a first light emitter (10) configured to emit light with a first dominant emission wavelength;• a second light emitter (12) configured to emit light with a second dominant emission wavelength different from the first dominant emission wavelength;• a first light detector (14) configured to measure a reflectance of the first light emitter's (12) emitted light from a target (6) and optionally configured to measure a reflectance of the second light emitter's (10) emitted light from said target (6) ;• optionally a second light detector (16) configured to measure a reflectance of the second light emitter's (10) emitted light from said target (6) and optionally configured to measure a reflectance of the first light emitter's (12) emitted light from said target (6) ;• a temperature sensor (22) for monitoring temperature at the sensor package (2) location;• processing circuitry (28) for analyzing data or signals from the first light detector (14) , the second light detector (16) if present, and the temperature sensor (22) , and optionally conducting additional checks to determine the in-ear or on skin status of a wearable device (4) comprising the sensor package (2) .

15. Wearable device (4) with a sensor package (2) according to claim 15, wherein the wearable device (4) is one of the following: an earbud, a hearing aid, or an in-ear biometric device .

Images