Cardiac monitoring and user authentication using wireless earbuds

Abstract

The invention relates to the methods, apparatus, systems, software and mechanisms for detecting and determining biometric of a wearer of wireless earbuds, including, for instance biometric user authentication and monitoring of various cardiac conditions via the wireless earbuds.

Description

[0001] Attorney Docket No. 24636.777WO1

[0002] CARDIAC MONITORING AND USER AUTHENTICATION USING WIRELESS EARBUDS

[0003] RELATED APPLICATIONS

[0004] [1] This application claims priority to U.S. provisional patent application No. 63 / 705,985 filed on October 10, 2024, entitled CARDIAC MONITORING AND USER AUTHENTICATION USING WIRELESS EARBUDS, which is incorporated by reference herein in its entirety.

[0005] FIELD OF THE INVENTION

[0006] [2] The invention relates to the methods, apparatus, systems, software and mechanisms for user authentication and / or monitoring of various cardiac conditions of a user via wireless earbuds worn by the user.

[0007] BACKGROUND

[0008] [3] Cardiovascular diseases (CVDs) are the leading cause of global mortality, accounting for approximately 17.9 million deaths annually. This staggering figure underscores the critical need for innovative solutions in cardiac care, particularly non-invasive, continuous monitoring technologies that can enhance early detection of CVDs while alleviating the burden on healthcare systems. The widespread adoption of true wireless stereo (TWS) earbuds, expected to reach over 700 million users by 2025, offers a promising mobile health platform to meet this demand. The ears’ anatomical structure, rich in vital blood vessels like the superficial temporal and posterior auricular arteries, provides a stable site for accurate cardiac monitoring. Current research in earable computing leverages these vascular advantages to develop cardiac monitoring solutions. Representative techniques include integrating photoplethysmography (PPG) sensors for in-ear PPG monitoring and using ultrasonic transducers to measure cardiac dynamics. Headphone drivers have also been repurposed to detect heartbeats and reconstruct phonocardiogram (PCG) signals. However, these solutions necessitate additional hardware design and complex software processing. This poses significant challenges for commercial TWS earbuds, which are constrained by their form factor and resources, including energy, communication, and computation capabilities. Attorney Docket No. 24636.777WO1

[0009] SUMMARY

[0010] [4] In some example embodiments, there may be provided systems, methods, and articles of manufacture for cardiac monitoring via wireless earbuds.

[0011] [5] According to one embodiment, an apparatus comprises

[0012] BRIEF DESCRIPTION OF THE DRAWINGS

[0013] [6] The accompanying drawings, which are incorporated in and constitute a part of this specification, show certain aspects of the subject matter disclosed herein and, together with the description, help explain some of the principles associated with the disclosed implementations. In the drawings,

[0014] [7] FIG. 1 is a partial representation of components of a human heart illustrating blood flow through the heart and related blood vessels;

[0015] [8] FIG. 2 if a partial representation of a human ear illustrating BCG propagation in the arteries surrounding the ear;

[0016] [9] FIG. 3A is a series of graphs showing, in black, the raw 6-axis IMU measurement data recorded for an Apple Airpods Pro (2ndgeneration) earbud for the left ear of a first subject instructed to remain stationary for the duration of the measurements and, in red, the raw 6-axis IMU measurement data recorded for an Apple Airpods Pro (2ndgeneration) earbud when placed stationary on a flat surface;

[0017]

[0010] FIG. 3B is a series of graphs showing the raw 6-axis IMU measurement data recorded for an eSense open earable platform earbud for the right ear of a second subject instructed to remain stationary for the duration of the measurements;

[0018]

[0011] FIG. 4 are plot diagrams showing frequency domain bandpass SNR benchmark across different IMU axes and subjects, in which each marker represents a specific subject, with * markers and ☆ markers denote male and female subjects using Apple AirPods, respectively, while □ and 0 markers indicate male and female subjects using eSense earable system earphones, respectively and wherein all plots share the same x axis representing the subject's BMI;

[0019]

[0012] FIG. 5 is a graph showing the measured timestamp skips in the IMU data stream for Apple AirPods Series earbuds under playback, active noise cancelling and transparency modes of operation for four different locations of the data gathering device (cell phone) relative to the earbuds; Attorney Docket No. 24636.777WO1

[0020]

[0013] FIG. 6A is a graph showing packet loss for each axis of a three axis accelerometer of an IMU under four different motion conditions, namely, stable, speaking, nodding, shaking, and walking;

[0021]

[0014] FIG. 6B is a graph showing packet loss for each axis of a three axis gyroscope of an IMU under four different motion conditions, namely, stable, speaking, nodding, shaking, and walking;

[0022]

[0015] FIG. 7 is a block diagram illustrating the logical components of a cardiac monitoring system using wireless earbuds, including data capturing, signal processing, and neural network model inference, as a single iOS APP on the iPhone 15 Pro running iOS 18 in accordance with embodiments;

[0023]

[0016] FIG. 8A is a graph showing SNR for the accelerometers of an in ear IMU with and without SWT-based denoising under four different motion artifacts, namely, speak, saccade, type, shake and walk;

[0024]

[0017] FIG. 8B is a graph showing SNR for the gyroscopes of an in ear IMU with and without SWT-based denoising under four different motion artifacts, namely, speak, saccade, type, shake and walk;

[0025]

[0018] FIGS. 9A, 9B, 9C, 9D, and 9E are time-frequency domain visualizations for motion artifacts denoising for a clean cardiac harmonic signal, a noisy signal, a cardiac harmonic signal with residual noise, a denoised cardiac harmonic signal, and a reference cardiac harmonic signal, respectively;

[0026]

[0019] FIG. 10 is a block diagram illustrating the logical components of a deep denoising model for motion artifacts in accordance with embodiments;

[0027]

[0020] FIG. 11 is a series of graphs showing a real-world example of an IMU data stream received by a smartphone when a stationary subject wore the earbuds in automatic noise cancelling mode illustrating packet losses;

[0028]

[0021] FIG. 12 is a block diagram illustrating the logical components of super-resolution cardiogram reconstruction neural network that translates multi-axis in-ear BCG into clinically reliable SCG signals with a high sampling rate in accordance with embodiments;

[0029]

[0022] FIG. 13 is a pictorial representation of an experimental setup using a cell phone, TWS earbuds, and a Polar H10 chest strap heart monitor in accordance with embodiments;

[0030]

[0023] FIG. 14A is a graph illustrating results of a case study using EarCardio to monitor heart rate of a subject in accordance with embodiments; Attorney Docket No. 24636.777WO1

[0031]

[0024] FIG. 14B is a graph illustrating results of a case study using EarCardio to monitor heart rate variance of a subject in accordance with embodiments;

[0032]

[0025] FIG 15 is a graph illustrating results of a case study using EarCardio to monitor heart rates of a two patients with tachycardia and bradycardia, respectively, in accordance with embodiments;

[0033]

[0026] FIG. 16 is a graph illustrating the true positive rate of EarCardio when used for user authentication according to embodiments;

[0034]

[0027] FIGS. 17A and 17B are graphs illustrating the mean error for diastolic blood pressure and systolic blood pressure measurements, respectively, using EarCardio according to embodiments;

[0035]

[0028] FIG. 18A is a graph showing reconstructed ECG waveforms using EarCardio for ECG reconstruction according to experiments in accordance with embodiments;

[0036]

[0029] FIG. 18B is a graph showing the Similarity and SOI for the experiments represented in FIG. 18 A;

[0037]

[0030] FIG 19 is a block diagram of the components of a computer system that may be used to implement the various techniques described herein; and

[0038]

[0031] FIG. 20 is a block diagram illustrating the basic components of a typical COTS TWS earbud.

[0039] DETAILED DESCRIPTION

[0040]

[0032] The human ear offers a unique opportunity for cardiac monitoring due to its physiological and practical advantages. However, existing earable solutions require additional hardware and complex processing, posing challenges for commercial TWS earbuds, which are limited by their form factor and resources. A system (with associated methods and articles of manufacture) that uses the inertial measurement unit (IMU) sensors in TWS earbuds for cardiac monitoring is disclosed herein. The system (as well as associated methods and articles of manufacture) is sometimes referred to herein as “EarCardio.” These sensors capture in-ear ballistocardiogram (BCG) signals. The ballistocardiogram (BCG) signals capture ultra-low frequency mechanical characteristics of a pumping heart. The EarCardio may reuse an unstable Bluetooth channel to stream the IMU data to, for example, a user equipment (e.g., a smartphone or other processorbased device) for BCG processing. The EarCardio incorporates a signal enhancement Attorney Docket No. 24636.777WO1 framework to address issues related to missing data and low sampling rate, while mitigating motion artifacts by fusing multi-axis information. Furthermore, the EarCardio employs a region- focused signal reconstruction method to translate the multi-axis in-ear BCG signals into finegrained seismocardiogram (SCG) signals.

[0041]

[0033] The system disclosed herein uses (e.g., repurposes) commercial TWS earbuds for cardiac monitoring applications. Standard IMU sensors (e.g., accelerometers, gyroscopes, and the like) embedded in TWS earbuds and originally intended for head motion tracking may be used to capture in-ear BCG signals. Based on this observation, TWS earbuds may be augmented, e.g., via software without the need for any hardware modifications, with BCG monitoring capabilities. EarCardio may be used to convert the acquired BCG signals into SCG waveforms that are associated with clinically relevant cardiac metrics. EarCardio is designed to be compatible with commercial off-the-shelf (COTS) TWS earbuds, such as Apple AirPods, eSense, and / or the like, and may be implemented through software enhancements, without any hardware modifications.

[0042]

[0034] Reusing the TWS earbuds hardware and software for such purposes presents some challenges. To preserve the earbuds’ normal functions and minimize software overhead, EarCardio utilizes the existing Bluetooth Low Energy (BLE) communication channel to stream raw IMU data to a user equipment (e.g., mobile device, smartphone, or other processor-based device), which then processes the IMU data to derive BCG signals. However, the BLE-based IMU stream is primarily designed for real-time head motion tracking, which tends to be deprioritized by the TWS earbuds and allocated only minimal resources. Thus, the IMU data stream may suffer from low sampling rate and unreliable sampling issues, especially when contending with more audio streaming functions. To overcome this challenge, timestamps are used in the IMU stream to identify the locations of missing data. A neural-based cardiographic continuity enhancement method is configured by employing data fusion across multiple axes and ensemble learning from consecutive beats to fill in the missing data. Additionally, a superresolution scheme is implemented to reconstruct cardiographic signals at a high sampling rate.

[0043]

[0035] Reliable in-ear BCG sensing also presents some challenges. The BCG signals extracted from IMU typically exhibit low signal to noise ratio (SNR) compared to those from specialized sensors, especially in the presence of motion artifacts. Additionally, variability in ear shapes and vessel locations results in differing sensitivities across the axes. To address these challenges, a two-stage denoising solution may be implemented. In an embodiment, first, the motion artifacts Attorney Docket No. 24636.777WO1 in the frequency domain are mitigated using stationary wavelet transform (SWT). Moreover, a multi-hcad attenuation network may be implemented to select and fuse the axes that arc more sensitive to BCG signals. Moreover, to enhance model generalization, an IMU simulator may be used to create highly diverse training data, incorporating different motion artifacts and user profiles.

[0044]

[0036] Utilizing in-ear BCG for practical applications presents additional challenges. The ultimate goal of EarCardio is to harness in-ear BCG for practical applications, such as CVD monitoring as well as other applications. However, existing neural-based signal enhancement solutions inadvertently suppress anomalous heartbeat signals. Similarly, anomalous heartbeat peak magnitude might be mistakenly normalized due to biases inherent in the training dataset. To maintain the fidelity of crucial cardiac features, one or more neural networks may be configured to achieve super-resolution BCG waveform reconstruction, without changing the interval of the signals, thereby preserving essential time-domain information. Additionally, a cardiac-focused loss function may be configured that prioritizes the accurate reconstruction of peak regions of the cardiographic signals, while diminishing focus on less critical areas. To further enhance accuracy, a transformer-based neural network may be used to fuse and translate the multi-axis in-ear BCG signals into clinically reliable SCG signals, capitalizing on their shared waveform characteristics derived from mechanical heart movements.

[0045]

[0037] In an example embodiment, the EarCardio was configured end-to-end as Android and iOS apps, running on smartphones which acquire and process the IMU data from the TWS earbuds. A real-world dataset was collected consisting of 40 hours of in-ear BCG recordings from over 100 participants, and synthesized an additional 150 hours of data with 202 types of motion artifacts for model training. Evaluation results may show that EarCardio effectively reconstruct SCG signals, with an average cosine similarity of 0.92 across participants with varying Body Mass Index (BMI), ages, and genders, although results may vary. EarCardio estimates heart rate (HR) and inter-beat interval (IB I) with average errors of 2.73% and 1.74%, respectively, even in the presence of significant motion artifacts, missing data, and low sampling rates (although results may vary). EarCardio can operate in real-time on COTS TWS earbuds and paired smartphones with power consumption of only 1.14 mAh / h and 7.26 mAh / h, respectively (although results may vary). The recovered SCG signals may be used to support various healthcare and Human Computer Interaction (HCI) applications, including long-term HR Attorney Docket No. 24636.777WO1 and Heart Rate Variability (HRV) monitoring, biometric user authentication, pathological case detection, Blood Pressure (BP) estimation, and electrocardiogram (ECG) reconstruction.

[0046]

[0038] Cardiographic signals are categorized into four types based on sensing modalities, as summarized in Table 1 below.

[0047] Table 1: Sensing mechanisms for cardiographic signals.

[0048]

[0039] PPG utilizes a light source and a photodetector to measure blood volume changes. ECG captures the heart’s electrical activity using electrodes. PCG employs microphones or specialized sensors to record heart sounds. SCG detects subtle chest wall vibrations from the heart’s mechanical actions, particularly the movement of the heart muscle and the ejection of blood. The BCG used in EarCardio is akin to SCG, measuring the body’s recoil movements caused by the heart’s mechanical activity during blood ejection, as shown in FIG. 1.

[0049]

[0040] ECG, SCG, and PCG are considered relatively invasive. ECG requires electrodes to be firmly attached to the skin, limiting their use for continuous cardiac monitoring. SCG and PCG require firm attachment of the sensor to the subject’s chest. In contrast, PPG and BCG are less invasive and more accessible through wearable sensing. A PPG sensor, for instance, is a standard module found on many smartwatches, and BCG can be measured using ubiquitous accelerometers positioned on the body or an adjacent surface, such as a bed or chair.

[0050]

[0041] Motion artifacts pose a significant challenge for reliable monitoring of cardiographic signals. Predominant studies assume that motion has minimal impact on blood pulsation, treating motion artifacts as independent disturbances. Under this assumption, advanced signal Attorney Docket No. 24636.777WO1 processing techniques such as independent component analysis, adaptive noise cancellation, and Fourier scries analysis may be utilized to mitigate these effects. Additionally, machine learning methods like time-delay neural networks may be employed to refine signals using motion references from accelerometers and gyroscopes. While effective for subtle motion artifacts in PPG and ECG monitoring, their efficacy diminishes under complex and substantial motion conditions. BCG faces more pronounced challenges because it is inherently a motion signal and is much weaker compared with the movements of other body parts. EarCardio addresses this challenge by empowering its neural enhancement model with massive and accurate simulationbased training data, and by selectively fusing the multi-axis IMU data.

[0051]

[0042] ECG and PCG are considered clinical gold standards for CVD diagnostics due to the tight sensor attachment and hence highly sensitive measurement. Recent studies highlight that the less-invasive PPG and SCG technologies also closely correlate with myocardial activities. These technologies offer potential for translating PPG / SCG to ECG. For instance, CardioGAN

[0021] employs a GAN model with a dual discriminator strategy to learn mappings in both time and frequency domains, while Performer

[0025] uses a Transformer model to enhance performance and aid in CVD detection. Region-Disentangled Diffusion Model applies a diffusion model to focus on regions of interest, such as the QRS complex, achieving accurate PPG-to-ECG translation. Wang et al. [9] employ an encoder-decoder architecture using smartphone accelerometers to reconstruct ECG signals from SCG signals. While these methods are promising, they largely depend on controlled, high-quality datasets, which may limit practical applicability. In contrast, EarCardio captures in-ear BCG signals characterized by low SNR, low sampling rates, unreliable sampling, and significant motion interference. Case studies involving 100 participants demonstrate EarCardio’s effectiveness in real-world cardiac monitoring applications.

[0052]

[0043] Emerging TWS earbuds are not only equipped with advanced acoustic features like active noise cancellation (ANC), but are also increasingly used for mobile health. For example, Apple has incorporated hearing aid functionalities that allow users, particularly the elderly, to personalize their sound profiles based on hearing capabilities. This demographic trend, coupled with the devices’ technological advancements, amplifies their suitability for ubiquitous health monitoring.

[0053]

[0044] Table 2 compares existing carablc cardiographic sensing solutions. In-ear PPG: Research has incorporated PPG sensors in TWS earbuds. These sensors facilitate respiratory and blood Attorney Docket No. 24636.777WO1 pressure monitoring by analyzing variations between the left and right earbuds.

[0054] Audioplcthysmography (APG) transmits low-intensity ultrasound signals into the car and captures reflected echoes, producing signals with waveforms similar to PPG signals. However, both the PPG and APG sensors require adding dedicated sensors to the earbuds.

[0055] Table 2: Comparison between various in-ear cardiographic techniques (HW: Hardware).

[0056]

[0045] With regard to in-ear PCG, the ear canal’s natural acoustics amplify heart sounds, facilitating in-ear PCG measurements. Utilizing in-ear microphones, studies have captured these sounds for biometric verification and blood pressure estimation. However, standard TWS earbuds lack such in-ear microphones as their microphones, designed for ANC and speech recording, are facing outside. Moreover, ambient noise remains a significant hurdle for in-ear PCG, even with in-ear microphone setups. Alternatively, HeadFi [4] and Asclepius [6] repurpose headphone drivers as sensors to detect heartbeats and reconstruct PCG signals. These approaches, however, involve intricate hardware additions like low-noise amplifiers and voltage converters, which increase the complexity, cost, and energy consumption of the earbuds. Attorney Docket No. 24636.777WO1

[0057]

[0046] With regard to around-ear BCG, FaceReader [35, 36] explores the potential of using IMU sensors in an AR / VR (Augmented Reality / Virtual Reality) headset for heartbeat detection. Since these sensors are positioned far from the arteries, they primarily detect bone vibrations caused by heartbeats, resulting in heart-rate estimation only. On the other hand, EarCardio integrates IMU sensors into a custom-designed hearing aid, firmly attaching them behind the ear to capture behind-ear BCG signals.

[0058]

[0047] EarCardio represents the first system to use the existing IMU sensors in TWS earbuds to measure in-ear BCG signals and reconstruct SCG / ECG signals. This method is implementable on, for example, COTS TWS earbuds without any hardware modifications, changes to wireless communication protocols, or significant system overhead, although EarCardio may be implemented using non-COTs carbuds as well.

[0059]

[0048] FIG. 1 shows the basic in-ear BCG model, which can be interpreted as the blood pressure gradients in the ascending and descending aorta

[0038] . When an instantaneous force is exerted on the blood in the main artery , the BCG force FBCG can be estimated as

[0060] FBCGW = Sd[Pi(t) - P2(t) ]-Sa[Po(t)-Pi(t) ] Eq. 1 wherein SdandSa are the average cross-sectional areas of the descending and ascending aorta, respectively. Po(t), Pi(t), and P2(t) represent the blood pressure waves of the ascending aorta inlet, arch outlet / inlet, and descending aorta outlet, respectively. As shown in FIG. 2, two arteries around the ear, i.e., the superficial temporal artery and the occipital artery, introduce significant mechanical vibration inside the ear. It has been newly discovered that in-ear BCG signals can propagate into the ear canal, allowing them to be measured by the IMU sensors in COTS TWS earbuds.

[0061]

[0049] To assess the feasibility of in-ear BCG, preliminary studies were conducted using two commercial TWS earbuds that provide access to raw accelerometer and gyroscope data from their IMU sensors, i.e., the Apple AirPods series and the eSense open earable platform. During the experiment, subjects were instructed to wear the earbuds as they normally would while remaining stationary. FIG. 3A shows the raw IMU waveform recorded by a set of Apple AirPods Pro (2nd generation), which exhibits the same pattern to typical BCG signals.

[0062] Particularly, FIG. 3A is a series of graphs showing, in black, the raw 6-axis IMU measurement Attorney Docket No. 24636.777WO1 data recorded for an Apple Airpods Pro (2nd generation) earbud for the left ear of a first subject instructed to remain stationary for the duration of the measurements and, in red, the raw 6-axis IMU measurement data recorded for Apple an Airpods Pro (2nd generation) earbud when placed stationary on a flat surface.

[0063]

[0050] This experiment was repeated using eSense. FIG. 3B is a series of graphs showing the raw 6-axis IMU measurement data recorded for an eSense open earable platform earbud for the right ear of a second subject instructed to remain stationary for the duration of the measurements. It was found that the eSense earbud also consistently captured in-ear BCG signals.

[0064]

[0051] This is attributable to the high resolution of existing IMUs. According to the datasheet for the eSense earable product, earable accelerometers have a typical measurement range of ±2g and a digital resolution of 16 bits, translating to a resolution of 0.06 mg / LSB (milli-G per least significant bit). The gyroscopes feature a measurement range of ±125dps (degrees per second) and a digital resolution of 16 bits, resulting in a resolution of Q.0Q4-dps / LSB. A preliminary study indicates that the average maximum peak amplitude of in-ear BCG, characterized by accelerometer and gyroscope, is approximately Q.QQ5in s2and 0.03dps, respectively, resulting in an average empirical peak SNR of 3 dB and \9dB. This suggests that the IMU sensors in TWS earbuds have sufficient resolution to sense in-ear BCG signals.

[0065]

[0052] To measure the reliability of in-ear BCG, a frequency domain Bandpass SNR is used, which evaluates how effectively the 1 ~ 10Hz cardiac frequency components are preserved amidst noise. The Bandpass SNR (in dB) is calculated as: 10 ]og \ )(Psignal / Pnoise). where we compute the power spectral density (PSD) to determine the mean power of both the signal

[0066] (P signal) and noise (Pnoise). Pnoise is sampled when placing the earbuds on a stationary surface.

[0067]

[0053] The SNR of multi-axis in-ear BCG signals was measured across 100 subjects. Findings are illustrated in FIG. 4 and summarized as follows. In FIG. 4, the plots show frequency domain bandpass SNR benchmark across different IMU axes and subjects, in which each marker represents a specific subject, wherein asterisk markers denote male subjects using Apple AirPods, star markers denote female subjects using Apple AirPods, square markers denote male subjects using eSense earable system earphones, and 0 markers denote female subjects using eSense earable system earphones, and wherein all plots share the same x axis representing the subject's BMI. Attorney Docket No. 24636.777WO1

[0068]

[0054] Significant variation in SNR was observed across different IMU axes. The axis oriented toward the occipital artery (Y-axis for Apple AirPods and X-axis for cScnsc) achieves the highest SNR, averaging I 8.2 E / B and 20.5 k / B, respectively. Conversely, the axis facing outward from the subject (X-axis for Apple AirPods and Z-axis for eSense) exhibit the lowest SNRs, with averages of 10.04dB and 13.93dB, respectively.

[0069]

[0055] Female subjects exhibit slightly lower SNRs than male subjects, with an average difference of 1 ,53<7B across different axes. BMI affects the SNR, but insignificantly, with only a 1.87 dB reduction observed between underweight (BMI < 18.5&g / m2) and obese subjects (BMI > 30 kg / m2). With respect to age, the average SNR is 12.33< B for subjects under 30 years of age, 10.67 dB for those between 30 and 40 years, and 9.56 dB for those over 40 years. This decrease in SNR with age is likely due to the higher BMI typically associated with older subjects.

[0070]

[0056] The SNR is consistent for different ear tip size and wearing styles. Apple provides four ear tip sizes and a fit test to select the best size for users. When users chose an unsuitable ear tip without passing the ear tip fit test, the SNR drops by only 1.34 dB and 1.13 dB for the accelerometer and gyroscope, respectively.

[0071]

[0057] IMU sensors in AirPods are not configurable, whereas eSense provides IMU sensor configuration. The eSense was configured to maximize SNR, by minimizing the measurement range of accelerometer with ±2g and gyroscope with ±250dps, respectively, while disabling the on-chip low-pass filter. eSense achieves higher SNR than AirPods Pro, with an average SNR of 18.84 dB compared to 13.12 dB. This may be because eSense can be better secured on the subject’ s ears as it offers specific designs to ensure a firm attachment.

[0072]

[0058] Despite the potential of in-ear BCG sensing, several key challenges (which can be resolved) should be addressed for practical usage. Because of the limited computational resources, the earbuds’ IMU data have to be streamed to an associated mobile device for post processing, which raises two challenges. Moreover, standard IMU sensors typically support sampling rates exceeding 500Hz. However, TWS earbuds are limited to much lower rates due to BLE constraints. COTS TWS earbuds limit each BLE packet to transfer one 6-axis IMU data sample without buffering to guarantee real-time processing. The minimal packet interval for Bluetooth 5.4 is 7.5ms, theoretically allowing a maximum rate of 133.3Hz. In practice, factors such as energy consumption and computational limitations further reduce this rate. For example, Apple AirPods Series is configured with a minimal packet interval of 4-Oms. resulting in a 20 - Attorney Docket No. 24636.777WO1

[0073] 25 Hz. rate. These rates are sufficient for head motion tracking, but inadequate for cardiac monitoring and analysis, which typically demands a minimum sampling rate of 100 - 200Hz. And, the IMU data stream transmitted over BLE exhibits inconsistent sampling intervals. The IMU stream from the Airpods Pro as received by an iPhone was observed. Based on the received timestamps, it was found that the IMU data is uniformly sampled at 40 / n.s intervals with a 25 Hz. sampling rate. However, the sampling intervals deviate significantly, by multiples of 4-Oms, indicating data loss. FIG. 5 shows the measured timestamp skips in the IMU stream under different scenarios. Significant data loss occurs due to several factors. First, the computational resources allocated to other real-time audio functions impact the IMU sampling reliability. IMU sample loss is particularly severe during “ANC” and “Trans” modes. Second, data loss is also influenced by BLE channel conditions. When the smartphone is within 5 meters of the subject or on their body, packet loss is minimal. However, with 15 m separation and wall obstructions, packet loss exceeds 30%. Besides, the real-time control algorithms for head motion tracking and spatial sound generation in TWS earbuds necessitate a short supervision timeout (i.e., 100ms) on BLE channels to ensure low latency. The supervision time-out is the maximum period a BLE device can wait before considering the connection lost. A short supervision timeout exacerbates data loss, especially in a busy wireless channel. This observation aligns with other measurement studies of BLE wearable devices.

[0074]

[0059] With respect to Low SNR, the SNR of in-ear BCG is compared with that of other body parts. During the measurement, users were instructed to firmly attach an AirPod earbud to the “Back of the ear,” “Chest”, and “Neck” using tape. In-ear BCG has 7.97 dB, 6.76 dB, and 2.27 dB lower SNR than “Back of the ear,” “Chest”, and “Neck”, respectively. This is because the two near-ear arteries are not proximate to the in-ear measurement site (FIG. 2). Besides, in-ear BCG were captured with normally worn earbuds, rather than the tape-secured setting.

[0075]

[0060] As illustrated in FIGS. 6A and 6B, the SNR of in-ear BCG was quantified across several common motions, including “Speak”, “Nod”, “Head shake”, and “Walk”. “Speak” motion introduces interference at intensities comparable to that of stable in-ear BCG. In contrast, head movements and walking produce substantial motion noise, surpassing the desired BCG signals by several orders of magnitude.

[0076]

[0061] Although different subjects achieve similar SNR, their in-ear BCG waveforms exhibit significant variability. FIGS. 3A and 3B demonstrate an example for two subjects, subject A and Attorney Docket No. 24636.777WO1 subject B, respectively.. Subject #A’s in-ear BCG (FIG. 3A) is sensitive to the Y-axis and Z-axis of the accelerometer and the Y-axis and Z-axis of the gyroscope, while Subject #B (FIG. 3B) is sensitive only to the X-axis and Y-axis of the gyroscope. This variability is attributed to differences in ear shapes, vessel locations, and wearing styles among subjects, which affect the waveform and the sensitivity of different IMU axes.

[0077]

[0062] The following provides an example of a system design for the EarCardio.

[0078]

[0063] Although motion artifacts obscure the subtle in-ear BCG in the time-frequency domain, as visualized in FIG. 9B, two key factors offer new opportunities to de-noise such interference. First, in-ear BCG has a unique cardiac frequency pattern distinct from arbitrary body motion. This pattern enables a determination of whether the IMU signals are dominated by motion artifacts or in-ear BCG, allowing for targeted enhancement of the relevant signals. Second, certain sensors and axes among the 6-axis IMU experience less motion interference during specific motions, as highlighted in the example in FIGS. 6 A and 6B. In accordance with embodiments, a motion artifact denoising pipeline leverages these two findings, comprising two stages, namely, -based denoising and neural-based multi-axis denoising.

[0079]

[0064] Common motion artifacts, particularly those generated by slow head movements and systemic body motions, typically have frequency components below 2Hz. In contrast, the critical peaks of BCG signals, including the H, I, J, K and L peaks, occur at frequencies above 3Hz. The SWT-based algorithm is designed to reduce the low-frequency dominant motion interference while maintaining the critical high-frequency BCG signals. The SWT-based denoising method can be expressed as: where D and D™ t) represent the zth wavelet coefficients corresponding to high-frequency BCG signals and lower-frequency motion artifacts, respectively. N is the order of SWT, and / is the decomposition level separating the frequencies of motion artifacts and BCG signals. We set N = 5 and J= 2 when the sampling rate of IMU sensor is fs= 25 Hz. Therefore, the filtered signal frequency band is >fs / 2J+ I= 3.25 Hz, ensuring the preservation of critical peak information in BCG signals. In practice, first the IMU signals are decomposed using SWT, and then the Attorney Docket No. 24636.777WO1 enhanced time-domain cardiac signals se (t) are reconstructed by applying inverse SWT to high- frequency component { DJ, }. Note that these steps are performed separately for each axis.

[0080]

[0065] An advantage of SWT compared to traditional discrete wavelet transform is its ability to decompose signals into different frequency bands without downsampling. This ensures that the processed signals retain the time-domain details of the BCG signals. However, due to the aforementioned problem of unreliable sampling, directly applying SWT to the raw IMU data would significantly corrupt both the time and frequency information. Fortunately, the IMU stream records the timestamp of each received IMU sample, allowing one to identify and calculate the missing patterns in the data stream. Thus, before applying SWT-based denoising, first, a simple linear interpolation is performed at the timestamps missing IMU samples, which upsamples the IMU stream to a stable sampling rate. It is important to note that this step alone is insufficient to fully address the unreliable sampling and missing data.

[0081]

[0066] FIGS. 8A and 8B show the SNR with and without SWT-based de-noising under various motion artifacts for the accelerometers (FIG. 8QA) and the gyroscopes (FIG. 8B, respectively. SWT-based denoising effectively reduces low-frequency motion interference, notably during subtle, slow motions like “Speak,” “Saccade” and “Type”. However, significant residual noise remains in axes with high-frequency interference, mostly due to fast motions like “Nod”, “Shake”, and “Walk” (see FIG. 9C).

[0082]

[0067] To reduce the residual noise, a denoising neural network has been designed that takes the SWT denoised IMU data as input, and fuses the multi-axis data based on the aforementioned two findings, namely, (1) in-ear BCG has a unique cardiac frequency pattern distinct from arbitrary body motion and (2) certain sensors and axes among the 6-axis IMU experience less motion interference during specific motions. FIG. 10 illustrates a neural network model architecture 1000 in accordance with embodiments. The model takes 5-second (window size), 6-axis IMU sensor data as input and outputs the corresponding denoised 5-second, 6-axis in-ear BCG signals. To preserve the critical timing information, the sampling rate for both input signals and output signals are consistently maintained at 25 Hz to match the IMU sampling rate. Thus, each sample set comprises 125 samples. The model consists of two components. The first component is a channel attention module 1010 that uses multi-head attention to dynamically leverage the channels and time periods with higher SNR to denoise the low-SNR channels. The second component is a denoising autoencoder 1012, which has proven effective in denoising time-series Attorney Docket No. 24636.777WO1 signals and reconstructing complex signal patterns. Specifically, the encoder 1012 compresses the input signal into a lowcr-dimcnsional representation, effectively filtering out noise while preserving essential features. A decoder 1014 then reconstructs the signal, focusing on the cardiac components while suppressing residual motion artifacts.

[0083]

[0068] Training a robust denoising model is challenging because it requires a massive amount of paired BCG data with and without motion artifacts. Even with extra sensors attached to the earbuds, acquiring ground truth in-ear BCG data under motion artifacts is difficult. The fact that motion artifacts can be considered independent and additive interference to in-ear BCG, as they have minimal impact on blood pulsation, can be leverages to address this challenge. Therefore, one can first collect clean in-ear BCG signals without motion artifacts, and separately simulate the IMU signals under motion. Then, the training dataset may be synthesized through linear combination.

[0084]

[0069] To collect clean in-ear BCG signals, the 100 subjects were re-used. Particularly, they were instructing them to remain seated and stationary. The resulting data include more than 145,000 heartbeats, totaling around 25 hours. As for the motion artifact, direct collection of IMU data will inadvertently mix the motion and BCG signals. Additionally, a massive training dataset is needed to cover a wide variety of motion types and ensure model generalization. Fortunately, existing research has developed IMU simulators capable of accurately converting human body motion data into accelerometer and gyroscope signals. The IMU simulator is adapted to synthesize motion artifact-induced earbud IMU signals. Specifically, the AMASS database, which is a large collection of human motion data that unifies 15 optical marker-based motion capture datasets, was used. The AMASS database provides 63 hours of high-quality, high-resolution 3D human meshes in Skinned Multi-Person Linear (SMPL) format, covering a wide variety of daily human motions, including locomotion, exercising, cooking, human object interaction, etc. First, the human mesh data is down-sampled from 60 FPS to 25 FPS, which is the sampling rate of TWS earbuds. Next, a virtual IMU is placed at specific vertices in the SMPL mesh model, i.e., at the ear site, and virtual orientations are obtained via forward kinematics, and accelerations are obtained via finite differences based on the TWS earbuds’ IMU coordinate systems. Finally, the training IMU data was synthesized by linearly combining the self-collected in-ear BCG signals with the simulated motion artifact signals. The final synthesized dataset contains 150 hours of data, over 300 different body shapes, and a wide Attorney Docket No. 24636.777WO1 variety of daily motions. This comprehensive dataset facilitates the training of a generalizable denoising neural network.

[0085]

[0070] A signal enhancement method is used to further address the challenges of cardiographic discontinuity caused by unreliable sampling and data loss. FIG. 11 shows a real- world example of an IMU stream received by a smartphone when a stationary subject wore the earbuds in ANC mode. Even after applying linear interpolation, the in-ear BCG signals remain significantly disrupted, particularly when critical peak information is lost, as highlighted in the bottom plot of FIG. 11 (y-axis accelerometer data feed).

[0086]

[0071] With respect to a model design, EarCardio leverages two principles to design the cardiographic continuity refinement solution. First, the peak positions in BCG signals vary across different axes due to the transmission of vibrations through blood vessels in multiple directions

[0049] , Therefore, even if a specific axis (e.g., FIG. 11, bottom plot) misses critical peak information, other axes can preserve their own (e.g., FIG. 11, middle plot). Second, consecutive heartbeats exhibit similar BCG signals and are unlikely to miss the same peak information simultaneously. Thus, the data from different axes can be combined to resolve cardiographic discontinuities. These two design principles align closely with the neural based multi-axis denoising model. Consequently, the same neural network can be repurposed to achieve dual objectives, namely, motion artifact denoising and cardiographic continuity refinement. To guide the model to focus on missing data refinement, the corresponding pattern of missing samples, extracted from IMU stream timestamps, is integrated into the model. As shown in FIG. 10, EarCardio employs the missing pattern 1016 as a binary mask, providing supplementary positional information for multi-head attention and guiding the autoencoder to recover the missing data.

[0087]

[0072] To train this model, it is necessary to collect in-ear BCG signals from TWS earbuds without data loss, which will serve as the ground truth. To achieve this, all audio functionalities are disabled to ensure that no additional BLE or Bluetooth devices are detected nearby. Data is collected while the associated smartphone receiver is placed on a table nearby, as shown in FIG. 13. After data collection, any received IMU streams with packet loss is manually removed, creating a clean dataset at a 25 Hz sampling rate. Then, a missing pattern with varying packet loss rates (0% to 50%) is randomly generated and applied to the clean data. Attorney Docket No. 24636.777WO1

[0088]

[0073] The cardiographic continuity refinement solution may not perform well when the packet loss rate is sufficiently high, c.g., due to a long BLE link distance. To address this, a packet loss rate threshold, r, is set to identify when the IMU stream is too severely corrupted. EarCardio can accordingly enter a fail-safe mode, warning the user to discard such data. It was empirically determined that threshold T should be set to approximately 24% by evaluating the performance of the cardiographic continuity refinement under different packet loss rates.

[0089]

[0074] Although this signal enhancement design improves the in-ear BCG quality, its low sampling rate and unreliable waveform limit its ability to provide detailed cardiac insights. Whereas the IMUs on COTS TWS earbuds can achieve 40 ms time resolution (at 25Hz), reliable HRV measurement requires a time resolution of at least 10 ms. Besides, multi-axis in-ear BCG provides inconsistent peak magnitude and waveform information, which varies depending on the subject’s ear shape, vessel locations, and wearing style. To overcome the challenge, a superresolution cardiogram reconstruction neural network 1200 such as illustrated by FIG. 12 was developed that translates multi-axis in-ear BCG into clinically reliable SCG signals with a high sampling rate.

[0090]

[0075] The model’s input 1210 is the output from the aforementioned denoising neural network, i.e., FIG. 10. The model begins with a feature extraction stage 1214, where 4-layer convolutional layers distill crucial spatial features from the multi-axis in-ear BCG signals. This is followed by a two-layer Transformer encoder 1216, which captures long-range dependencies and sequential patterns. The attention mechanism within the Transformer encoder allows the model to dynamically focus on key temporal features, particularly those associated with heartbeats. Finally, a fully connected layer 1218 decodes the processed features into singlechannel SCG signals with 100 Hz sampling rate. A fully connected layer 1218 was chosen instead of Transformer decoder to enforce super-resolution upsampling while maintaining the critical timing information of the cardiographic signals by fixing the output size. The model is trained using the self-collected 25 hours of clean in-ear BCG data and their corresponding SCG ground truth.

[0091]

[0076] The design of loss function is crucial to guide the neural network’s focus during training. Recognizing that SCG signals are dominated by the energy around their peaks, a heartbeat energy-weighted loss function was designed. This approach ensures that the network emphasizes the peak locations during the learning process. Specifically, the Hilbert transform Attorney Docket No. 24636.777WO1 was utilized to extract the heartbeat envelope of the target SCG signal. This envelope is then normalized and used to weight the Mean Absolute Error (MAE) loss, ensuring that regions with higher energy, corresponding to heartbeats, are prioritized during training. The weighted MAE loss is combined with a standard MAE loss, creating a dual-focus mechanism that encourages the network to minimize overall error while paying special attention to the heartbeat regions.

[0092]

[0077] The loss function is defined as: = a x s i lyi - yd) + P x Qs^ wdy - yd) Eq. 3 where N represents the number of samples, ytis the predicted value, ytis the ground truth value, and Wj denotes the normalized weight derived from the Hilbert-transformed heartbeat envelope. The weights a and / 3 are hyperparameters that balance the contribution of each loss term.

[0093]

[0078] In-ear BCG signals can be used, not only to reconstruct the SCG signals for measuring HR and HRV, but also for other diverse applications. However, each application requires collecting a large amount of paired in-ear BCG data and corresponding ground truth to train a specific model. To overcome this burden, the features learned from the aforementioned BCG-to- SCG super-resolution translation task may be leveraged as foundational embeddings. Instead of retraining an entire model, it is necessary only to replace the final fully connected layer and finetune the model for the specific task using few- shot learning.

[0094]

[0079] As depicted in FIG. 13, the experimental setup consists of a TWS earbud 1310 paired with a smartphone 1312. Apps were implemented to acquire 6-axis raw IMU data using native iOS API and eSense Android library for AirPods and eSense, respectively. Since these two achieve comparable performance, experiments used AirPods by default, unless noted otherwise.

[0095]

[0080] To collect clean in-ear BCG data in a controlled setting, participants 1314 were seated in a chair and instructed to wear the right earbud normally by default. All audio features of earbuds were disabled to prevent data loss. To simultaneously obtain ground truth SCG signals, a Polar H10 chest strap heart rate monitor device 1316 was employed, which provides 100Hz SCG data with nanosecond-level precision. Participants were instructed to wear the Polar H10 1316 according to the user manual. The Polar H10 was connected to another smartphone 1318 via the Polar Flow app for data acquisition. To ensure generalization of the trained models and evaluation results, it is essential to introduce variability in participants’ heart rates. Thus, following existing studies, heart rate fluctuations were stimulated by displaying participant- selected videos, including movies and news, on a monitor 1320 in front of them. Attorney Docket No. 24636.777WO1

[0096]

[0081] A total of 100 participants took part in the experiments, including 40 females and 60 males, aged 19 to 62 years, with body mass indices (BMI) ranging from 15.92 kg / m2to 33.74 kg / m2. Each participant underwent a 15-minute data collection session, resulting in a clean in-ear BCG dataset, totaling 25 hours and approximately 145, 000 heartbeats. This dataset is used for training / validating the EarCardio models and verifying the data synthesis method.

[0097]

[0082] For evaluation and case study purposes, an additional dataset was collected from 10 participants using TWS earbuds to cover various scenarios, amounting to 15 hours and approximately 87, 000 heartbeats. Specifically, data was collected using different earbud side and operation modes and during various motions and daily activities. Note that both AirPods and eSense only support one-ear IMU measurement streaming. To measure dual-ear IMU signals, two pairs of ear-buds and smartphones were used, connecting one specific earbud to a smartphone while disabling the other, and then repeating the process for the second earbud and smartphone pair. SCG signals were not considered as ground truth data in the testing dataset due to significant interference from motion artifacts. Instead, the Polar H10 heart monitor was used to collect 130Hz ECG signals and precise heartbeat rate (HR), interbeat interval (IB I) measurements calculated from ECG signals as the ground truth. All experimental procedures were approved by an Institutional Review Board (IRB).

[0098]

[0083] The 6-axis raw IMU data and corresponding ground truth SCG signals were segmented into 5-second intervals and preprocessed with a 0.5Hz high-pass filter to mitigate respiratory interference. Both the signal enhancement network (1000, FIG. 10) and cardiogram reconstruction model (1200, FIG. 12) were trained on a server equipped with an A40 GPU over 100 epochs. The initial learning rate was set at 0.001 and reduced by half every 10 epochs. Following the data synthesis methods discussed above in connection with FIGS. 10 and 12, we used the 25-hour clean in-ear BCG dataset from the 100 participants to synthesize a 150-hour dataset under 202 types of motion artifacts and with various packet loss rates from 0 - 40% for training and validation. The dataset was divided into 10 groups for cross-validation, with each fold involving training on 90 participants and validation on the remaining 10.

[0099]

[0084] The end-to-end system pipeline of EarCardio 700 as described in detail hereinabove is depicted at FIG. 7 as a single iOS APP on the iPhone 15 Pro running iOS 18. As shown, the IMU data stream is captured from the carbuds along with the IMU timestamps. At 710, the IMU timestamp stream is used to determine which IMU data was lost. At 712, it is determined is the Attorney Docket No. 24636.777WO1 packet loss threshold (e.g., 24% is exceeded for a period, in which case, the data is discarded for that period as being too unreliable.

[0100]

[0085] If the data is deemed sufficiently reliable, next, at 714, the missing IMU data stream data is filled in using simple interpolation between captured data samples. At 716, the data is denoised (e.g., using SWT per Eq. 2 and the related discussion above). Then, the denoised SWT data is applied to the signal enhancement neural network 1000. As previously noted, this same neural network 1000 may be used to both denoise motion artifacts (718), and refine cardiographic continuity (720). Also, as previously noted, the model may integrate the corresponding pattern of missing samples, extracted from IMU stream timestamps.

[0101]

[0086] The output of the signal enhancement stage, i.e., conditioned 6-axis in-ear BCG data, is fed to the cardiogram reconstruction neural network 1200. The data is processed by the transformer encoder 1216, which captures long-range dependencies and sequential patterns, and then decoded into a single-channel of reconstructed SCG signals.

[0102]

[0087] Neural network inference was implemented using Core ML, provided by Apple Developer. With Core ML, it was possible to switch the computational unit for neural network inference between the CPU, GPU, and Neural Engine, for system performance evaluation.

[0103]

[0088] The reconstructed SCG signals and the foundational cardiac-related embeddings are then fed to the downstream applications 722, which may use the reconstructed SCG data for various purposes, including, for instance, blood pressure estimation 722a, biometric authentication, 722b, ECG reconstruction 722c, and heart rate and / or heart rate variance estimation 722d.

[0104]

[0089] Each of these four applications were experimentally tested for robustness and accuracy. The experimental results are discussed further below.

[0105]

[0090] Five case studies were conducted to illustrate the utility of EarCardio for the applications 722 mentioned above as well as another potential application, detection of cardiac pathologies in patients.

[0106]

[0091] With regard to long-term HR and HRV monitoring (application 722d) , one participant was tracked using EarCardio over a 2-week period, measuring HR and HRV six times daily at 2- hour intervals, with each session lasting 5 minutes. For HRV evaluation, the Standard Deviation of Normal-to-Normal intervals (SDNN)

[0063] was used, which estimates HRV over a specific period, capturing the influence of both the sympathetic and parasympathetic branches of the autonomic nervous system. In the following discussion, HRV is used to refer to SDNN for Attorney Docket No. 24636.777WO1

[0107] HRV. FIG. 14 presents the average HR and HRV across 2 weeks. The results demonstrate that EarCardio maintains stable HR and HRV estimations over time with an average HR error of 1.68 beats per minute (BPM) and an HRV error of 3.68 milliseconds. The long-term fluctuations captured by EarCardio closely mirror those of the ground truth.

[0108]

[0092] With regard to detection of pathological cases, to demonstrate that EarCardio can preserve anomalous heart conditions, three patients were recruited known to have anomalous heart conditions, namely, tachycardia, bradycardia, and arrhythmia, corresponding to resting heart rate exceeding 100 BPM, below 60 BPM, and varying HR and HRV, respectively. FIG. 15 presents reconstructed SCG waveform for the first two cases demonstrating that EarCardio achieves robust Interbeat Interval (IB I) estimation with Mean Percentage Error (MPE) below 2% in 90% of cases, underscoring its effectiveness in accurately capturing anomalous cardiac signals.

[0109]

[0093] As shown in FIG. 15, EarCardio can detect the arrhythmia cases, which has significant long-term HR and HRV fluctuations. These results suggests that EarCardio can identify the pathological cases for early-stage CVD detection. Our future plan is to evaluate EarCardio across a broader cohort of CVD patients.

[0110]

[0094] With regard to use of EarCardio for biometric user authentication (application 722b), e.g., to enhance device security while enabling personalized TWS earbud experiences, Five percent of in-ear BCG data from 50 participants from the default dataset was used to fine-tune the pretrained cardiogram reconstruction model (FIG. 12) into an authentication model by adjusting the linear projection layers. The Area Under the Curve (AUC), a common metric for user authentication

[0064] , was used. An AUC > 0.9 is considered excellent for biometric authentication

[0064] , As shown in FIG. 16, EarCardio achieves AUCs of 0.92, 0.95 and 0.99 with 1.3 .md 5 heartbeat cycles as model input, respectively. This highlights the utility of EarCardio for quick and reliable earbud-based user authentication.

[0111]

[0095] With regard to blood pressure estimation (application 722a), previous study shows that blood pressure can be derived by analyzing the time differences in cardiac signal propagation between multiple measurement points, i.e., Pulse Transit Time (PTT)

[0032] . Inspired by this, it is proposed to leverage the dual-ear channels as natural reference points to capture PTT for BP estimation. Five participants were recruited to collect an additional 6-hour dual-ear in-ear BCG data, while measuring the ground truth BP using an FDA-certified Checkme BP2A Monitor over Attorney Docket No. 24636.777WO1 one week. The pre-trained cardiogram reconstruction network for BP estimation was fine-tuned using 4 hours of data for training and 2 hours for testing, achieving a correlation of 0.81 for Diastolic Blood Pressure (DBP) and 0.83 for Systolic Blood Pressure (SBP). FIGS. 17A and 17B show that the mean error for DBP and SBP is 1.93mmHg and 1.29mmHg, respectively, thereby meeting the clinical accuracy standards for BP estimation.

[0112] 6.5.5 Electrocardiogram reconstruction.

[0113]

[0096] With regard to electrocardiogram (ECG) reconstruction (application 722c), EarCardio also may be leveraged to reconstruct the in-ear BCG into ECG, the gold standard for CVD monitoring. An additional 2.5-hour dataset was collected from 10 participants over one week using a dual-ear setup along with Polar H1O ECG signals to be used as the ground truth. The model was fine-tuned from the pretrained cardiogram reconstruction network using a 2-hour training dataset and evaluated on 30-minute testing dataset. The ECG signals were resampled from 130 Hz to 100 Hz to align with the model output requirements. FIG. 18A showcases the reconstructed ECG waveform. FIG. 18B is a graph showing the Similarity and SOI for the experiments represented in FIG. 18 A. As can be seen in FIG. 18B, a single-ear setup achieves a Similarity of about 0.70 and an SOI of 0.80, while dual-ear setup further enhances performance, with Similarity of 0.87 and SOI of 0.91. respectively, thus demonstrating EarCardio's potential to reconstruct ECG signals.

[0114]

[0097] In some implementations, some of the aspects disclosed herein may be implemented in a computing system, such as a computing system 500 depicted at FIG. 19. The system may be comprised in a user equipment, such as a laptop, smartphone, tablet, and / or other device., The system may include a processor 510, a memory 520, a storage device 530, and input / output device 540. The processor 510, the memory 520, the storage device 530, and the input / output device 540 can be interconnected via a system bus 550. The processor 510 is capable of processing instructions for execution within the computing system 500. In some implementations of the current subject matter, the processor 510 can be a single-threaded processor. Alternately, the processor 510 can be a multi- threaded processor. Alternatively, or additionally, the processor may comprise a graphic processing unit, an Al chip, and / or the like. The processor 510 is capable of processing instructions stored in the memory 520 and / or on the storage device 530 to display graphical information for a user interface provided via the input / output device 540. The memory 520 is a computer readable medium such as volatile or Attorney Docket No. 24636.777WO1 non-volatile that stores information within the computing system 500. The storage device 530 is capable of providing persistent storage for the computing system 500. The storage device 530 can be a floppy disk device, a hard disk device, an optical disk device, or a tape device, or other suitable persistent storage means. The input / output device 540 provides input / output operations for the computing system 500. In some implementations of the current subject matter, the input / output device 540 includes a keyboard and / or pointing device. In various implementations, the input / output device 540 includes a display unit for displaying graphical user interfaces. According to some implementations of the current subject matter, the input / output device 540 can provide input / output operations for a network device. For example, the input / output device 540 can include Ethernet ports or other networking ports to communicate with one or more wired and / or wireless networks (e.g., a local area network (LAN), a wide area network (WAN), the Internet).

[0115]

[0098] FIG. 20 is a is a block diagram illustrating the basic components of a typical COTS TWS earbud 2000. It comprises a housing 2001 containing a speaker 2003 for reproducing audio signals for the listening pleasure of the wearer. It further comprises an input / output unit 2005. The input / output unit may receive audio signals from another device, such as a computer system such as illustrated in FIG. 19, which may, for instance, comprise a smartphone for playing audio content via the speaker 2003 among other things. The audio signal may be received via a Bluetooth radio channel. The input output unit 2005 may also output signals back to the smartphone (or other device) as well as other components of the earbud, such as lights for indicating various conditions of the earbuds to the wearer. It also includes a battery 2007 for powering the earbud and its various components.

[0116]

[0099] It may further include one of more sensors 2009, such as audio sensors which may be used for monitoring external noises (e.g., for noise cancellation purposes) and inertial sensors (accelerometers and / or gyroscopes) for sensing motion, orientation, and / or acceleration of the wearer. It further includes a controller 2013 for controlling operation of the earbud. It may further include a memory 2011 for storing the software for running the controller and / or storing useful information, such as a condition of the earbud (e.g., in ANC or transparency mode, etc.). In addition, the earbud may include one or more sensors 2015 specifically dedicated to sensing biometric information of the wearer. Attorney Docket No. 24636.777WO1

[0117]

[0100] In some implementations, some of the aspects disclosed herein may be implemented in a computing system, comprised as a device inscrtablc into the car such as the carbud depicted at FIG. 20. The earbud may include one or more of the following: sensors, such as the IMU, biometric sensors, memory, a controller (e.g., processor or PID controller), a battery, and an input / output device (which may provide for example a wireless Bluetooth Channel) to the user equipment of FIG. 19.

[0118]

[0101] One or more aspects or features of the subject matter described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs, field programmable gate arrays (FPGAs) computer hardware, firmware, software, and / or combinations thereof. These various aspects or features can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client- server relationship to each other.

[0119]

[0102] These computer programs, which can also be referred to as programs, software, software applications, applications, components, or code, include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object- oriented programming language, and / or in assembly / machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus and / or device, such as for example magnetic discs, optical disks, memory, and Programmable Logic Devices (PLDs), used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor. The machine -readable medium can store such machine instructions non-transitorily, such as, for example, as would a non-transient solid- state memory or a magnetic hard drive or any equivalent storage medium. The machine -readable medium can alternatively or additionally store such machine instructions in a transient manner, Attorney Docket No. 24636.777WO1 such as for example, as would a processor cache or other random access memory associated with one or more physical processor cores.

[0120]

[0103] To provide for interaction with a user, one or more aspects or features of the subject matter described herein can be implemented on a computer having a display device, such as for example a cathode ray tube (CRT) or a liquid crystal display (LCD) or a light emitting diode (LED) monitor for displaying information to the user and a keyboard and a pointing device, such as for example a mouse or a trackball, by which the user may provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, such as, for example, visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touch screens or other touch-sensitive devices such as single or multi-point resistive or capacitive track pads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and the like.

[0121] Conclusion

[0122]

[0104] In the descriptions above and in the claims, phrases such as “at least one of’ or “one or more of’ may occur followed by a conjunctive list of elements or features. The term “and / or” may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it is used, such a phrase is intended to mean any of the listed elements or features individually or any of the recited elements or features in combination with any of the other recited elements or features. For example, the phrases “at least one of A and “one or more of A and B;” and “A and / or B” are each intended to mean “A alone, B alone, or A and B together.” A similar interpretation is also intended for lists including three or more items. For example, the phrases “at least one of A, B, and C;” “one or more of A, B, and C;” and “A, B, and / or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” Use of the term “based on,” above and in the claims is intended to mean, “based at least in part on,” such that an unrecited feature or element is also permissible.

[0123]

[0105] The subject matter described herein can be embodied in systems, apparatus, methods, and / or articles depending on the desired configuration. The implementations set forth in the Attorney Docket No. 24636.777WO1 foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they arc merely some examples consistent with aspects related to the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of several further features disclosed above. In addition, the logic flows depicted in the accompanying figures and / or described herein do not necessarily require the particular order shown, or sequential order, to achieve desirable results. For example, the logic flows may include different and / or additional operations than shown without departing from the scope of the present disclosure. One or more operations of the logic flows may be repeated and / or omitted without departing from the scope of the present disclosure. Other implementations may be within the scope of the following claims.

[0124] REFERENCES

[0125] [1] Mariachiara Di Cesare, Pablo Perel, Sean Taylor, Chodziwadziwa Kabudula, Honor Bixby, Thomas A Gaziano, Diana Vaca McGhie, Jeremiah Mwangi, Botjana Pervau, Jagal Narula. et al. The heart of the world. Global heart, 2024.

[0126] [2] ResearchAndMarkets.com. Global true Wireless stereo earbuds market. 2022-2030: Health meets entertainment - the fusion of hearing aid functionalities in tws earbuds. Global TWS Market Report. 2023.

[0127] [3] Andrea Ferlini, Alessandro Montanari, Chulhong Min, Hongwei Li, Ugo Sassi, and Fahim Kawsar. In-ear ppg for vital signs. IEEE Pervasive Computing, 2021.

[0128] [4] Xiaoran Fan, Loagfei Shangguan, Siddharth Rupavmharam, Yanyong Zhang, Jie Xiong, Yunfei Ma, and Richard Howard. Headfi: bringing intelligence to all headphones. In Proceedings of ACM MobiCom, 2021.

[0129] [5] Xiaoran Fan, Da,~d Pearl, Richard Howard, Longfei Shangguan, and Trausti Thornundsson. Apg: Audioplethysmography for cardiac monitoring in hearables. In Proceedings of ACM MobiCom, 2023. Attorney Docket No. 24636.777WO1

[0130] [6] Tao Chea, Yongjie Yang, Xiaoran Fan, Xiuzhen Guo, Jie Xiong, and Longfei Shangguan. Exploring the fcasibilily of remote cardiac auscultation using earphones. In Proceedings of ACM MobiCum, 2024.

[0131] [7] Apple. Apple ai,pods. 2024.

[0132] [8] Fahim Kawsar, Chulhong Min, Akhil Mathur, and Alessandro Montamui. Earabies for personal-scale behavior analytics. IEEE Pervasive Computing, 2018.

[0133] [9] Lei Wang. Xingwei Wang, Dalin Zhang, Xiaolei Ma, Yong Zhang, Haipeng Dai, Chenren Xu. Zhijun Li, and Tao Gu. Knowing your heart condition anytime: User- independent ecg measurement using commercial mobile phones. Proceedings of ACM IMWUT (UbiComp). 2023.

[0134]

[0010] Apple. Apple watcll. -2024. https: / / www.apple.com / watch / .

[0135]

[0011] Yandao Huang, Minghui Qiu, Lin Chen, Zhencan Peng, Qian Zbang, and Kais bun Wu. Nf- heart: A near-field non-contact continuous user authentication system via ballistocardiogram. Proceedings of ACM IMWUT ( UbiComp ), 2023.

[0136]

[0012] Alessandro Montanan, Andrea Ferlini. Ananta Narayanan Balaji. Cecilia Mascolo, and Fahim Kawsar. Earset: A multi-modal dataset for studying the impact of head and facial movements on in-ear ppg signals. Scientific Data, 2023.

[0137]

[0013] Abdul Q Javaid, Hazar Ashouri, Alexis Dorier, Mozziyar Etemadi, J Alex Heller, Shuvo Roy, and Omer Tinan. Quantifying and reducing motion artifacts in wearable seismocardiogram measurements during walking to assess left ventricular health. IEEE Transactions in Biomedical Engineering, 20 I 6.

[0138]

[0014] Shubhojeet Chatterjee, Rini Smita Thakur, Ram Narayan Yadav, Lalita. Gupta, and Deepak Kumar Raghuvanshi. Review of noise removal techniques in ecg signals. IET Signal Processing. 14(9):569-590. 2020.

[0139]

[0015] Dongyeol Seok, Sanghyun Lee, Minjae Kim, Jaeou.k-Cho, and Chu] Kim. Motion artifact removal techniques for wearable eeg and ppg sensor systems. Fromiers in Electronics. 2021 .

[0140]

[0016] M Raghu Ram, K Venu Madhav, Ette Hari Krishna, Nagarjuna Reddy Komalla, Kosa raju Sivani, and K Ashoka Reddy. Ica-based improved dtewt technique for ma reduction in ppg Attorney Docket No. 24636.777WO1 signals with restored respiratory information. IEEE Transaction on Instrumentation and Measurement, 2013.

[0141]

[0017] Sayeed Shafayet Chowdhury, Rakib Hyder, Md Samzid Bin Hafiz, and Mohammad Ariful Haque. Real-time robust heart rate estimation from wrist-type ppg signals using multiple reference adaptive noise cancellation. IEEE journal of biomedical and health informatics, 2016.

[0142]

[0018] Remya Raj, J Selvakumar and Vivek Maik. Afodss: Heart rate estimation method from photoplethysmographic signals with motion artifacts using fourier sparse dual optimization. IEEE Sensor Journal. 201.9.

[0143]

[0019] Ke Xu, Xinyu Jiang, and Wei Chen. Photoplethysmography motion artifacts removal based on signal-noise interaction -modeling utilizing envelope filtering and time-delay neural network. IEEE Sensor Journal. 2019.

[0144]

[0020] Qiang Zhu, Xin Tian, Chau-Wai Wong, and Min Wu. Ecg reconstruction via ppg: A pilot study. In 2019 IEEE EMBS international conference on biomedical & health Informatics (BHI). IEEE, 2019.

[0145]

[0021] Pritam Sarkar and Aii Etemad.. Cardiogan: Attentive generative adversarial network with dual discriminators for synthesis of ecg from ppg. In Proceedings of AAAI. 2021.

[0146]

[0022] Debaditya Shome, Pritam Sarkar, and Ali Etemad. Region-disentangled diffusion model for high-fidelity ppg-to-eeg translation. In Proceedings of AAAI. 2024.

[0147]

[0023] Qiang Zhu, Xin Tian, Chau-Wai Wong, and Min Wu. Learning your heart actions from pulse: Ecg waveform reconstruction from ppg. IEEE Internet of Things Journal, 2021.

[0148]

[0024] Xin Tian, Qiang Zhu, Yuenan Li, and Min Wu. Cross-domain joint dictionary learning for ecg inference from ppg. IEEE Internet of Things, 2022.

[0149]

[0025] Ella Lan. Performer: A novel ppg-to-ecg reconstruction transformer for a digital biomarker of cardiovascular disease detection. In Proceedings of IEEE / CVF WACV, 2023.

[0150]

[0026] Unsoo Ha, Salah Assana, and Fadel Adib. Contactless seismocardiography via deep learning radars. In Proceedings of ACM MobiCom, 2020. Attorney Docket No. 24636.777WO1

[0151]

[0027] Lei Wang, Kang Huang, Ke Saa, Wei Wang, Chen Tian., Lei Xie, and Qing Gu. Unlock with your heart: Heartbeat-based authentication on commercial mobile phones. Proceedings of ACM IMWUT(UbiComp). 2018.

[0152]

[0028] Alistair EW Johnson, Tom J PoUard, Lu Shen, li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii. a freely accessible critical care database. Scientific data. 3(1): 1-9, 2016.

[0153]

[0029] Ary L Onlclherger, Luis AN Amaral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov. Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit and -physionet: components of a new research resource for complex physiologic signals. cir,-llfarioll, 2000.

[0154]

[0030] Lauren Sherman. Airpods as hearing aids: New features announced, 2024. https: / / www.ncoa.org / adviser / hearing-aids / airpods-heari ng- aids / .

[0155]

[0031] Julia Romero, AndreaFerlini, Dimitris Spatthis, Ting Dang, Katayoun Farrahi, Fahim Kawsar, and Alessandro Montanari. Optibreathe: An earable-based ppg system for continuous respiration rate, breathing phase, and tidal volume monitoring. In Proceedings of ACM HotMobile. 2024.

[0156]

[0032] Ananta Narayanan Balaji, Andren Ferlini, Fahim Kawsar, and Alessandro Montanari. Stereo-hp: Non-invasive blood pressure sensing with earables. In Proceedings of ACM HotMobile, 2023,

[0157]

[0033] Yetong Cao, Chao Cai, Fan Li, Zhe Chen, Md Jun Luo, Heartprint: Passive heart sounds authentication exploiting .in-ear microphones. In Proceedings of IEEE INFQCOM, 2023.

[0158]

[0034] Zhiyuan Zhao, Fan Li, Yadong Xie, Huanran Xie, Kerui Zhang, Li Zhang, and Yu Wang. Hearbp: Hear your blood pressure via in-ear acoustic sensing based on heart sounds. In Proceedings of IEEE INFOCOM. 2024.

[0159]

[0035] Tianfang Zhang, Cong Shi, Payton Walker, Zhengkun Ye, Yan Wang, Nitesh Saxena, and Yingying Chen. Passive vital sign monitoring via facial vibrations leveraging ar / vr headsets, In Proceedings of ACM MobiSys. 2023.

[0160]

[0036] Tianfang Zhang, Zhengkun Ye, Ahmed Tanvir Mahdad, Md Mojibur Rahman Redoy

[0161] Akanda, Cong Shi, Yan Wang, Nitesh Saxena, and Yiagying Chen. Facereader: Unobtrusively Attorney Docket No. 24636.777WO1 mining vital signs and vital sign embedded sensitive info via ar / vr motion sensors. In Proceedings of ACM ccs, 2023

[0162]

[0037] J Eric S Winokur, David Da He, and Charles G Sodini. A wearable vital signs monitor at the ear for continuous heart rate and pulse transit time measurements. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 2012.

[0163]

[0038] Chang-Sei Kim, Stephanie L Ober, M Sean McMurtry, Barry A Finegan, Omer T Inan, Ramakrishna Mukkamala, and lin-Oh Hahn. Ballistocardiogram: Mechanism and potential for unobtrusive cardiovascular health monitoring. Scientific reports, 2016.

[0164]

[0039] George A McKay. Signal-to-noise ratio in the bandpass output of a discriminator. IEEE Transactions on Aerospace and Electronic Systems. 1970.

[0165]

[0040] Apple. Choose your airpods pro ear tips and use the ear tip fit test. 2024. https : / / supporl .apple.co / en-us / 11 9849.

[0166]

[0041] Cambridge Nokia Bell Labs, esense user documentation. 2019. https: / / www.esense.io / share / eSense-User-Documentation.pdf.

[0167]

[0042] Ibrahim Sadek, Jit Biswas, and Bessam Abdulrazak. Ballistocardiogram signal processing: A review. Health information science and systems. 2019.

[0168]

[0043] VishaJ Varun Tipparaju, Kyle R Mallires, Di Wang, Francis Tsow, and Xiaojun Xian. Mitigation of data packet loss in bluetooth low energy based wearable healthcare ecosystem. Biosensors, 2021.

[0169]

[0044] Shuto Nagai, Daisuke Anzai, and Jianqing Waag. Motion artifact removals for wearable ecg using stationary wavelet transform. Healthcare technology letters, 2017.

[0170]

[0045] Emilio J Ochoa and Luis C ReviJJa. Signal lilleriog and peak analysis of ballistocardiography for heartbeat detection. In International Conference On Biomedical and Health informatics. Springer, 2022.

[0171]

[0046] Xugang Lu, Yu Tsao, Shigeki Matsuda, and Cbiori Hori. Speech enhancement based on deep denoising autoencoder. In Interspeech, 2013. Attorney Docket No. 24636.777WO1

[0172]

[0047] Yioghao Huang, Manuel Kaufmann, Emre Aksan, Michael J Black, Otmar Hilliges, and Gerard Pons-Moll. Deep inertial poser: Learning to reconstruct human pose from sparse inertial measurements in real time. ACM Transactions on Graphics (TOG), 2018.

[0173]

[0048] Naureen Mahmood, Nima Ghorhani, Nikolaus F Trroje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings ofICCV. 2019.

[0174]

[0049] Omer T Inan, Pierre-Francois Migeotte, Kwang-Suk Park, Mozziyar Etemadi, Kouhyar Tavakolian, Ramon Casanella, John Zanetti. Jens Tank, Irina Fumova. G Kim Prisk, et al. Balliscocardiography and seismocardiography: A review of recent advances. IEEE journal of biomedical and health informatics, 2014.

[0175]

[0050] Brian W Johnston, Richard Barrett-Jolley, Anton Krigc, and Ingcborg D Welters. Heart rate variability: Measurement and emerging use in critical care medicine. Journal of the Intensive Care Society’, 2020.

[0176]

[0051] A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems. 2017.

[0177]

[0052] Apple. Apple developer emheadphonemotionmanager. 2024. https: / / developer.apple.co,n / documenlatfon / coremotion / emheadphonemotiorunanager.

[0178]

[0053] eSense, esense android library, 2024. bltps: / / github.com / pervasive-systems / eSense- Android- Library .

[0179]

[0054] Polor. Polor h IO user manual. 2024. httnps: / / support.polar.com / e_manuals / hIO-heart-rate- sensor / polar-hlO-user-manunl-english / manual.pdf.

[0180]

[0055] Polor. Polor flow app, 2024. https: / / support.polar.com / en / support / Flow_app.

[0181]

[0056] Daniel C Richardson, Nicole K Griffin, Lara Zaki, Auburn Stephenson Jiachen Yan, Thomas Cuny, Richard Noble, John Hogan. Jeremy I Skipper, and Joseph T Devlin. Engagement in video and audio narratives: Contrasting self-report and physiological measures. Scientific Reports. 2020.

[0182]

[0057] Apple. Apple developer core ml. 2024. ht y; / / dey L&t / spgie.eom / documentation / coreml / . Attorney Docket No. 24636.777WO1

[0183]

[0058] Benjamin W Nelson and Nicholas B Allen. Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study. JMIR mHealth and uHealth, 2019.

[0184]

[0059] Mavuto M Mukaka. A guide to appropriate use of correlation coefficient in medical research. Malawi medical journal, 24(3):69-71, 2012.

[0185]

[0060] Rashmi Kumari, Surita Sarkar, Debeshi Dutta, Pabitru Das, and Amit Acharyya-. P2e-lgan: Ppg to ecg reconstruction methodology using Istm based generative adversarial network. In 2024 IEEE International Symposium on Circuits and Systems ( ISCAS). IEEE. 2024.

[0186]

[0061] lunichiro Hayano and Emi Yuda. Pitfalls of assessment of autonomic function by heart rate variability Journal of physiological anthropology 2019.

[0187]

[0062] Rahul Murmuria, Jeffrey Medsger, Angelos Stavrou, and Jeffrey M Voas. Mobile application and device power usage measurements. In 2012 IEEE sixth international conference on software security and reliability. 7EEE, 201 2.

[0188]

[0063] Fred Shaffer and Jay P Ginsberg. An overview of heart rate variability metrics and norms. Frontiers in public health, 2017.

[0189]

[0064] Chae Lin Seok, Young Do Song, Byeong Seon An, and Eui Chui Lee. Photoplethy smogram biometric authentication using a 1 d Siamese network. Sensors, 2023.

Claims

Attorney Docket No. 24636.777WO1CLAIMS1 . A method of determining biometric data about a wearer of wireless earbuds, the method comprising: receiving an inertial measurement data stream from an inertial measurement unit (IMLI) of an earbud worn by the wearer; denoising the inertial measurement data; applying the denoised inertial measurement data to a first neural network adapted to denoise motion artifacts from the data to generate a ballistocardiogram (BCG) signal from the denoised inertial measurement data; applying the BCG signal to a second neural network adapted to generate a single channel of reconstructed Seismocardiogram (SCG) signal from the BCG signal; and deriving from the SCG signal biometric data of the wearer of the earbud.

2. The method of claim 1 further comprising further using the first neural network to resolve cardiographic discontinuities in the denoised inertial measurement data in generating the SCG signal.

3. The method of claim 1 wherein the first neural network uses multi-head attention to dynamically leverage channels and time periods of the denoised inertial measurement data with higher Signal to Noise Ratio (SNR) to denoise channels and time periods of the denoised inertial measurement data with lower SNR channels.

4. The method of claim 1 wherein the data from the IMU comprises accelerometer data in three orthogonal axes and gyroscopic data in three orthogonal axes.

5. The method of claim 1 further comprising: receiving timestamp data from the IMU indicating the time of each piece of IMU data.

6. The method of claim 5 further comprising: using linear interpolation to fill in missing data in the inertial measurement data stream.Attorney Docket No. 24636.777WO17. The method of claim 1 wherein the denoising of the inertial measurement data is performed using a third neural network.

8. The method of claim 7 wherein the denoising of the inertial measurement data comprises stationary wavelet transform (SWT) denoising.

9. The method of claim 1 further comprising: discarding received inertial measurement data during periods when an amount of lost inertial measurement data exceeds a threshold.

10. The method of claim 8 when the threshold is 24%.11 . The method of claim 1 wherein the biometric data comprises at least one of blood pressure estimation, electrocardiogram reconstruction, heart rate estimation, and heart rate variance estimation.

12. The method of claim 1 further comprising: authenticating an identity of the wearer based on the biometric information.

13. The method of claim 1 wherein the earbud is a true stereo wireless (TWS) earbud.

14. A computer-readable device comprising non-transitory instructions, which, when executed by a processor, cause the processor to perform operations, the operations comprising: receiving an inertial measurement data stream from an inertial measurement unit (IMU) of an earbud; denoising the inertial measurement data; applying the denoised inertial measurement data to a first neural network adapted to denoise motion artifacts from the data to generate a ballistocardiogram (BCG) signal from the denoised inertial measurement data; applying the BCG signal to a second neural network adapted to generate a Seismocardiogram (SCG) signal from the BCG signal; and deriving from the SCG signal biometric data of the wearer of the earbud.Attorney Docket No. 24636.777WO115. The computer-readable device of claim 14 further comprising, in generating the SCG signal, using the first neural network to resolve cardiographic discontinuities in the denoised inertial measurement data.

16. The computer-readable device of claim 14 wherein the first neural network uses multihead attention to dynamically leverage channels and time periods of the denoised inertial measurement data with higher Signal to Noise Ratio (SNR) to denoise channels and time periods of the denoised inertial measurement data with lower SNR channels.

17. The computer-readable device of claim 14 wherein the data from the IMU comprises accelerometer data in three orthogonal axes and gyroscopic data in three orthogonal axes.

18. The computer-readable device of claim 14 further comprising : receiving timestamp data from the IMU indicating the time of each piece of inertial measurement data; discarding received inertial measurement data during periods when an amount of lost inertial measurement data exceeds a threshold; and using linear interpolation to fill in missing data in the inertial measurement data stream during periods when the threshold is not exceeded.

19. The computer-readable device of claim 14 wherein: the denoising of the inertial measurement data is performed by a third neural network and comprises stationary wavelet transform denoising.

20. The computer-readable device of claim 14 wherein the biometric data comprises at least one of blood pressure estimation, electrocardiogram reconstruction, heart rate estimation, and heart rate variance estimation.

Images