Audio plethysmography calibration

Abstract

Techniques and apparatus for performing audio plethysmography calibration are described. According to one or more preferred embodiments, a novel physiological monitoring process, called herein audio plethysmography (110), is provided that can perform an active acoustic method capable of sensing subtle physiologically relevant changes observable in a user's outer and middle ear (108). A wearable (102), such as an earphone, is provided that can utilize audio plethysmography (110) to monitor a user's biometrics, recognize facial behavior, and / or sense the environment using acoustic signals. Techniques for audio plethysmography calibration enable the wearable (102) to dynamically select frequencies that improve the performance of audio plethysmography (110). With audio plethysmography calibration, the wearable (102) can utilize different frequencies for different ears (108), and these frequencies can change over time.

Description

Background Art

[0001] Due to technological advancements in medicine and healthcare, people are able to live longer and healthier lives. To further achieve this, individuals are becoming interested in tracking their individual health. Health monitoring can encourage individuals to achieve specific fitness goals by tracking the gradual improvement of the performance of body functions. Additionally, individuals can monitor the impact of various chronic diseases on their bodies. Through active feedback from health monitoring, individuals can lead active and fulfilling lives despite having many chronic diseases and can quickly recognize situations where they need to seek medical treatment.

[0002] However, some devices that assist with health monitoring can be overly conspicuous and uncomfortable. Therefore, people may choose to refrain from health monitoring if the device negatively impacts their movements or causes inconvenience during daily activities. Thus, it is desirable for health monitoring devices to be reliable, portable, and inexpensive in order to encourage more users to utilize these features.

Summary of the Invention

[0003] Techniques and apparatuses for performing audio plethysmography calibration are described. Aspects described below include a method for audio plethysmography calibration. This method includes performing a calibration process to identify at least one acoustic frequency suitable for audio plethysmography using at least one speaker and at least one microphone. This method also includes using at least one frequency to perform audio plethysmography on a user's ear.

[0004] Some frequencies can have higher sensitivity for audio plethysmography than other frequencies. These frequencies can change over time based on the quality of at least a partially formed seal by the user's ear or a device around it. Also, these frequencies can vary according to different ears of the user due to differences in the shape of the ear canal. Therefore, the techniques for audio plethysmography calibration disclosed herein can enable dynamically selecting frequencies that improve the performance of audio plethysmography. Through audio plethysmography calibration, different frequencies for different ears can be utilized, and these frequencies can change over time.

[0005] After calibration, an acoustic transmission signal propagating inside at least a part of the user's ear canal can be transmitted by at least one speaker. Next, an acoustic reception signal can be received by at least one microphone, and the acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified by propagation inside the ear canal. At least one physiological metric of the user can be determined based on one or more modified waveform characteristics of the acoustic reception signal. Exemplary waveform characteristics include amplitude, phase, and / or frequency. Generally, the acoustic reception signal can result from the initially transmitted acoustic transmission signal being affected with respect to at least one of its amplitude, phase, and frequency when propagating inside the ear canal before being received via at least one microphone.

[0006] Aspects described below include a device including at least one transducer and at least one processor. The device is configured to perform any of the described methods.

[0007] Aspects described below also include a system having means for performing audio plethysmography calibration.

[0008] According to one or more preferred embodiments, there is provided a wearable such as an earphone that can perform an active acoustic method capable of sensing subtle physiologically related changes observable in a user's outer and middle ear, a novel physiological monitoring process called audio plethysmography herein. To perform audio plethysmography more appropriately, the wearable can form at least a partial seal inside or around the user's outer ear. This seal enables the formation of an acoustic circuit including the seal, at least one wearable, at least one ear canal, and at least one tympanic membrane of at least one ear. By transmitting and receiving acoustic signals, the wearable can recognize changes in the acoustic circuit, monitor the user's biometrics, recognize facial movements, and / or sense the environment. The wearable may be a stand-alone device or may be integrated inside another object or device such as glasses, a hat, earmuffs, or a helmet.

[0009] Apparatus and techniques for facilitating audio plethysmography calibration will be described with reference to the following drawings. The same numbers are used throughout the drawings to refer to like features and components.

Brief Description of the Drawings

[0010]

Figure 1-1

Figure 1-2

Figure 1-3

Figure 2

Figure 3

Figure 4-1

Figure 4-2

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

DETAILED DESCRIPTION OF THE INVENTION

[0011] Due to the technological advancements in medicine and healthcare, people are able to live longer and healthier lives. To further achieve this, individuals have become interested in tracking their personal health. Health monitoring can encourage individuals to achieve specific fitness goals by tracking the gradual improvement of the performance of bodily functions. Additionally, individuals can use health monitoring to observe the physical changes caused by chronic diseases. Through the active feedback provided by health monitoring, individuals can lead active and fulfilling lives despite having many chronic diseases and can recognize situations where they need to seek medical attention promptly.

[0012] However, some health monitoring devices can be overly conspicuous and uncomfortable. For example, to measure carbon dioxide levels, some devices collect blood samples from the user. Other devices may utilize auxiliary sensors, including optical or electronic sensors, which increase additional weight, cost, complexity, and / or bulk. Still other devices may require continuous battery recharging due to relatively high power consumption. Therefore, when health monitoring devices negatively impact one's actions or cause inconvenience during daily activities, people may choose to refrain from health monitoring. Thus, it is desirable for health monitoring devices to be reliable, portable, efficient, and inexpensive to expand access for more users.

[0013] To address this issue and provide new features, techniques for performing audio plethysmography calibration, particularly with respect to existing wearables, are described. According to one or more preferred embodiments, there is provided a wearable such as an earphone that can perform a novel physiological monitoring process called audio plethysmography herein, an active acoustic method capable of sensing subtle physiologically relevant changes observable in a user's outer and middle ear. To perform audio plethysmography more appropriately, the wearable may form at least a partial seal within or around the user's outer ear. This seal enables the formation of an acoustic circuit including the seal, at least one wearable, at least one ear canal, and at least one eardrum of at least one ear. By transmitting and receiving acoustic signals, the wearable can recognize changes in the acoustic circuit, monitor the user's biometrics, recognize facial movements, and / or sense the environment.

[0014] Some frequencies may be more sensitive to audio plethysmography than others. These frequencies can change over time, particularly based on the quality of the seal formed by the wearable in or around the user's ear. Also, these frequencies can vary depending on the shape differences of the ear canals for different users. Thus, techniques for audio plethysmography calibration enable a dynamic selection of frequencies that improve the performance of audio plethysmography. With audio plethysmography calibration, different frequencies can be utilized for different ears, and these frequencies can change over time.

[0015] Wireless technology has penetrated daily life, enabling users to easily access communication and data. One type of wireless technology is wireless wearables, examples of which include wireless earphones and wireless headphones. Wireless wearables have enabled users to move freely while listening to audio content from music, audiobooks, podcasts, and videos. The popularity of wireless wearables has created a market for adding additional features to existing wearables using current hardware (e.g., without introducing any new hardware). Therefore, the proposed solution can be implemented, in particular, by wireless wearables. The proposed solution can also be implemented by other objects or devices having one or more built-in wearables, such as glasses, hats, earmuffs, or helmets.

[0016] Operating Environment FIG. 1-1 is an illustration of an exemplary environment 100 that can implement audio plethysmography calibration. In the exemplary environment 100, a wearable 102 is connected to a smart device 104 using a physical interface or a wireless interface. The wearable 102 is a device that can play audible content provided by the smart device 104 and direct the audible content towards the ear 108 of the user 106. In this example, the wearable 102 operates with the smart device 104. In other examples, the wearable 102 can operate or be implemented as a stand-alone device. Although shown as a smartphone, the smart device 104 can include other types of devices including those described with respect to FIG. 2.

[0017] The hearable 102 can execute audio plethysmography 110, which is a method of acoustic perception occurring in the ear 108. The hearable 102 can perform this perception based only on the evaluation of the transmitted and received acoustic signals, thus without using other auxiliary sensors such as optical sensors or electrical sensors. By means of the audio plethysmography 110, the hearable 102 can perform biometric monitoring 112, facial behavior recognition 114, and / or environmental perception 116.

[0018] To use the audio plethysmography 110, the user 106 places the hearable 102 in such a way that at least a partial seal 118 is created around or in the ear 108. Some parts of the ear 108 are shown in FIG. 1 and include the ear canal 120 and the eardrum 122 (ear drum or tympanic membrane). By the seal 118, the hearable 102, the ear canal 120, and the eardrum 122 are coupled to each other to form an acoustic circuit. The audio plethysmography 110 includes measuring at least in part the characteristics associated with this acoustic circuit. The characteristics of the acoustic circuit can vary due to various different situations or actions.

[0019] For example, considering FIGS. 1-2, changes occur in the physical structure of the ear 108 therein. Exemplary changes to the physical structure include changes in the geometric shape of the ear canal 120 and / or changes in the volume of the ear canal 120. This change can be caused at least in part by a slight deformation of the blood vessels in the ear canal 120 caused by the heartbeat of the user 106. Other changes can also be caused by the movement of the eardrum 122 or the movement of the jaw of the user 106.

[0020] At 124, for example, the tissue around the external auditory canal 120 and the eardrum 122 itself are slightly "compressed" by the deformation of blood vessels. Due to this compression, the volume of the external auditory canal 120 is slightly reduced at 124. However, at 126, the compression is alleviated, and the volume of the external auditory canal 120 increases slightly compared to 124. The physical changes inside the ear 108 can modulate the amplitude and / or phase of the acoustic signal propagating through the external auditory canal 120, as will be further described below.

[0021] During audiopletismography 110, the acoustic signal propagates through at least a part of the external auditory canal 120. The wearable 102 can receive an acoustic signal representing the overlap of multiple acoustic signals propagating along different paths inside the external auditory canal 120. Each path is associated with a delay (τ) and an amplitude (a). The delay and amplitude can change over time due to subtle changes occurring in the physical structure of the external auditory canal 120. The received acoustic signal can be represented by Equation 1.

[0022] [Number]

[0023] Where S(t) represents the received acoustic signal, n represents noise, and φ ini represents the relative phase between the received acoustic signal and the transmitted acoustic signal, Ω fc represents the frequency of the transmitted acoustic signal, and t represents the time vector. Since the user 106's heart activity can modulate the amplitude and phase of the received acoustic signal, the received acoustic signal can also be described as further shown in Equation 2.

[0024] S(t)=n+(1+h amp (t))cos(h phase (t)+φ ini ++Ω fc (t)) Equation 2 Where h amp (t) represents the amplitude modulator, and h phase (t) represents the phase modulator. For example, two time-varying functions h amp(t) and h phase (t) can be influenced by the interaction between the hearable 102 and the ear 108, as well as the physiological activities of the user 106, especially the activities of the heart. When related to heartbeat-based modulation, for example, h amp (t) = k a sin(φ hr ++Ω hr (t) and h phase (t) = k p sin(φ hr ++Ω hr (t)) can be assumed. In the formula, k a and k p are modulation intensity coefficients, and Ω hr is the frequency of the user's heart rate. The interaction between the hearable 102 and the ear 108, and the physiological activities of the user 106 modulate the amplitude and phase of the received acoustic signal.

[0025] As another example, considering FIGS. 1 - 3, in which a change in the gas composition occurs in the external auditory canal 120. This change is caused at least partially through breathing. When the user 106 breathes, the skin of the user 106 can exchange gas with its surroundings. For example, at 128, inhalation 130 occurs, and the gas circulation system inside the external auditory canal 120 reduces the carbon dioxide concentration 132. At 134, exhalation 136 occurs, and the gas circulation system inside the external auditory canal 120 increases the carbon dioxide concentration 132. This change in the carbon dioxide concentration 132 affects the speed of sound, which in turn affects the speed at which the acoustic signal propagates through the external auditory canal 120.

[0026] Returning to FIG. 1-1, the hearable 102 can detect aspects associated with biometric monitoring 112, facial motion recognition 114, and / or environmental sensing 116 using an audio plethysmograph 110. Generally, biometric monitoring 112 can include measuring the user 106's heart rate, respiratory rate, blood pressure, body temperature, and / or carbon dioxide levels. Additionally, the physical structure of the ear canal 120 can be measured using biometric monitoring 112, and / or movements associated with impact forces can be detected. Biometric monitoring 112 can enable the hearable 102 to allow the user 106 to track fitness goals or monitor overall health. This can be particularly beneficial when caring for elderly patients or providing care for remote patients. Some types of biometric monitoring 112 may require various qualities of the seal 118. For example, heart rate can be measured with a relatively small seal 118, while respiratory rate may require a better seal 118.

[0027] The audio plethysmograph 110 can also be used for face motion recognition 114 that may include detecting jaw clenching, recognizing the start of speech, and / or recognizing specific activities involving the jaw (e.g., speaking or eating). Other types of face motion recognition 114 include recognizing facial expressions, tracking the user 106's line of sight or head pose, and / or recognizing face-touch gestures. To provide some of these features, the audio plethysmograph 110 can analyze the acoustic channel formed between the left and right ears 108. This acoustic channel can be modified by the user 106's facial expression, line of sight, head pose, or touch. With face motion recognition 114, the hearable 102 can facilitate communication with speech-impaired and hearing-impaired individuals and / or improve automatic speech recognition. Face motion recognition 114 also enables a less burdensome user experience since the user 106 can control features of the hearable 102 and / or the smart device 104 without touching the hearable 102.

[0028] The hearable 102 can also support environmental sensing 116 that may include detecting sports activities (e.g., walking or running). By detecting sports activities, the hearable 102 can automatically increase the volume of the user 106's audible content or play audible content from a playlist associated with a workout routine. As another example, the hearable 102 can also automatically detect when the user 106 positions the hearable 102 close to their ear 108 to form a seal 118. Thus, the hearable 102 can automatically determine when to play or pause audible content for the user 106 or when to perform biometric monitoring 112 or face motion recognition 114. The technology of the audio plethysmograph 110 can be executed while the hearable 102 is playing audible content to the user 106. The smart device 104 is further described with respect to FIG. 2.

[0029] FIG. 2 shows an exemplary smart device 104. Smart device 104 is shown with respect to various non-limiting exemplary devices including desktop computer 104-1, tablet 104-2, laptop 104-3, television 104-4, computing watch 104-5, computing glasses 104-6, game system 104-7, microwave oven 104-8, and vehicle 104-9. Other devices such as home service devices, smart speakers, smart thermostats, baby monitors, Wi-Fi® routers, drones, track pads, drawing pads, netbooks, e-book readers, home automation and control systems, wall displays, and other household appliances may also be used. Note that smart device 104 can be wearable, non-wearable but portable, or relatively stationary (e.g., desktop and appliances).

[0030] Smart device 104 includes one or more computer processors 202 and at least one computer-readable medium 204 including a memory medium and a storage medium. An application and / or an operating system (not shown) embodied as computer-readable instructions to the computer-readable medium 204 can be executed by computer processor 202 to provide some of the functions described herein. Computer-readable medium 204 also includes an audio plethysmography-based application 206 that performs an action using information provided by wearable 102. Exemplary actions can include displaying biometric data to user 106 based on biometric monitoring 112, providing touch-free control of smart device 104 based on face gesture recognition 114, or changing the presentation of audible content based on environmental sensing 116.

[0031] The smart device 104 can also include a network interface 208 for communicating data over a wired, wireless, or optical network. For example, the network interface 208 can communicate data over a local area network (LAN), wireless local area network (WLAN), personal area network (PAN), wide area network (WAN), intranet, Internet, peer-to-peer network, point-to-point network, mesh network, Bluetooth®, etc. The smart device 104 can also include a display 210. Although not explicitly shown, the wearable 102 may be integrated within the smart device 104 or physically or wirelessly connected to the smart device 104. The wearable 102 is further described with respect to FIG. 3.

[0032] FIG. 3 shows an exemplary wearable 102. The wearable 102 is shown with various non-limiting exemplary devices including wireless earphones 302-1, wired earphones 302-2, and headphones 302-3. Earphones 302-1 and 302-2 are of the in-ear device type that fits into the ear canal 120. Each earphone 302-1 or 302-2 can represent the wearable 102. Headphones 302-3 can be placed on top of or over the ear 108. Headphones 302-3 can represent closed-back headphones, open-back headphones, on-ear headphones, or over-ear headphones. Some headphones 302-3 include two wearables 102 physically packaged together. In this case, there is one wearable 102 for each ear 108. Other headphones 302-2, such as single-ear headphones 302-2, include one wearable 102. In some embodiments, one or more wearables 102 are implemented within (or as part of) another device such as glasses, a hat, earmuffs, or a helmet.

[0033] The hearable 102 includes a communication interface 304 for communicating with the smart device 104, although this need not be used when the hearable 102 is integrated inside the smart device 104. The communication interface 304 may be a wired or wireless interface, and audio content is passed from the smart device 104 to the hearable 102. The hearable 102 may also use the communication interface 304 to pass data measured using the audio plethysmograph 110 to the smart device 104. Generally, the data provided by the communication interface 304 is in a format usable by the audio plethysmography-based application 206. The communication interface 304 also enables the hearable 102 to communicate with another hearable 102. During binaural sensing, for example, the hearable 102 can use the communication interface 304 to coordinate with another hearable 102 to assist with the binaural audio plethysmograph 110, as further described with respect to FIG. 4-2. Specifically, the transmitting hearable 102 can communicate timing and waveform information to the receiving hearable 102 to enable the receiving hearable 102 to properly decode the acoustic signals it receives.

[0034] The hearable 102 includes at least one speaker and at least one microphone, for example, as part of at least one transducer 306 that can convert an electrical signal into a sound wave. The same transducer 306 or a further transducer of the hearable 102 can also detect a sound wave and convert it into an electrical signal. These sound waves can include ultrasonic frequencies and / or audible frequencies, either of which can be used for audio plethysmography 110. Specifically, the frequency spectrum (e.g., range of frequencies) used by the transducer 306 to generate an acoustic signal can include frequencies from the lower end of the audible region to the upper end of the ultrasonic region, for example, frequencies from 20 Hertz (Hz) to 2 Megahertz (MHz). Other exemplary frequency spectra for audio plethysmography 110 can include frequencies of 20 Hz to 20 kilohertz (kHz), 20 kHz to 2 MHz, 20 to 60 kHz, or 30 to 40 kHz.

[0035] In an exemplary embodiment, the transducer 306 has a monostatic topology. With this topology, the transducer 306 can convert an electrical signal into a sound wave and convert a sound wave into an electrical signal (e.g., can transmit or receive an acoustic signal). Exemplary monostatic transducers can include piezoelectric transducers, capacitive transducers, and micromachine ultrasonic transducers (MUTs) using microelectromechanical systems (MEMS) technology.

[0036] Alternatively, transducer 306 can be implemented using a bistatic topology that includes a plurality of physically separated transducers. In this case, a first transducer converts an electrical signal into a sound wave (e.g., transmits an acoustic signal), and a second transducer converts the sound wave into an electrical signal (e.g., receives an acoustic signal). An exemplary bistatic topology can be implemented using at least one speaker 308 and at least one microphone 310. The speaker 308 and the microphone 310 may be dedicated to the audio plethysmograph 110, or may be used for other functions of the audio plethysmograph 110 and the smart device 104 (e.g., presenting audible content to the user 106, making a phone call, or capturing the voice of the user 106 for voice control).

[0037] Generally, the speaker 308 and the microphone 310 are directed (e.g., oriented) towards the ear canal 120. Accordingly, the speaker 308 can direct an acoustic signal towards the ear canal 120, and the microphone 310 responds to receive an acoustic signal from a direction associated with the ear canal 120.

[0038] The wearable 102 includes at least one analog circuit 312, which includes circuitry and logic for conditioning electrical signals in the analog domain. The analog circuit 312 can include analog-to-digital converters, digital-to-analog converters, amplifiers, filters, mixers, and switches for generating and modifying electrical signals. In some embodiments, the analog circuit 312 includes other hardware circuitry associated with the speaker 308 or the microphone 310.

[0039] The wearable 102 also includes at least one system processor 314 and at least one system medium 316 (e.g., one or more computer-readable storage media). In the illustrated configuration, the system medium 316 includes an audio plethysmography measurement module 318 (APG measurement module 318) and optionally includes an audio plethysmography calibration module 320 (APG calibration module 320). The audio plethysmography measurement module 318 and the audio plethysmography calibration module 320 can be implemented using hardware, software, firmware, or a combination thereof. In this example, the system processor 216 implements the audio plethysmography measurement module 318 and the audio plethysmography calibration module 320. In an alternative example, the computer processor 202 of the smart device 104 can implement at least a portion of the audio plethysmography measurement module 318 and / or at least a portion of the audio plethysmography calibration module 320. In this case, the wearable 102 can communicate digital samples of the acoustic signal to the smart device 104 using the communication interface 304.

[0040] The audio plethysmography measurement module 318 analyzes the received acoustic signal to measure data related to the audio plethysmography 110. The audio plethysmography measurement module 318 can be implemented using at least one biometric monitor 322 for biometric monitoring 112, at least one facial behavior detector 324 for facial behavior recognition 114, and / or at least one environmental detector 326 for environmental sensing 116. An exemplary audio plethysmography measurement module 318 is further described with respect to FIGS. 5 and 7.

[0041] The audio plethysmography calibration module 320 can determine appropriate waveform characteristics for transmitting acoustic signals to improve the performance of the audio plethysmography 110. For example, the audio plethysmography calibration module 320 can determine the transmission frequency that can enable the wearable 102 to detect the user's 106 heart rate and / or respiratory rate with an accuracy of 5% or less, taking into account the quality of the seal 118 and the physical structure of the ear canal 120. Using the audio plethysmography calibration module 320, the wearable 102 can dynamically adjust the transmission frequency based on the unique physical structure of each ear 108 each time the seal 118 is formed. Through this calibration process, the wearables 102 in different ears can operate at one or more different acoustic frequencies. Exemplary embodiments of the audio plethysmography calibration module 320 are further described with respect to FIG. 6.

[0042] Some of the hearables 102 include an active noise cancellation circuit 328, which enables the hearable 102 to reduce background noise or environmental noise. In this case, the microphone 310 used for audio prethysmography 110 can be implemented using the feedback microphone 330 of the active noise cancellation circuit 328. During active noise cancellation, the feedback microphone 330 provides feedback information regarding the performance of the active noise cancellation. During audio prethysmography 110, the feedback microphone 330 receives an acoustic signal, which is provided to the audio prethysmography measurement module 318 and / or the audio prethysmography calibration module 320. In some situations, active noise cancellation and audio prethysmography 110 are performed simultaneously using the feedback microphone 330. In this case, the acoustic signal received by the feedback microphone 330 can be provided to at least one of the audio prethysmography modules 318 or 320 and can be provided to the active noise cancellation circuit 328. Different types of audio prethysmography 110 are further described with respect to FIGS. 4-1 and 4-2.

[0043] Audio prethysmography FIG. 4-1 shows an exemplary operation of two earables 102-1 and 102-2 that perform monaural audio plethysmography 110. In environment 400-1, earables 102-1 and 102-2 independently perform audio plethysmography 110 on different ears 108 of user 106. In this case, the first earable 102-1 is proximate to the right ear 108 of user 106, and the second earable 102-2 is proximate to the left ear 108 of user 106. Each earable 102-1 and 102-2 includes a speaker 308 and a microphone 310. Earables 102-1 and 102-2 can operate monostatically during the same period or different periods. In other words, each earable 102-1 and 102-2 can independently transmit and receive acoustic signals.

[0044] For example, the first earable 102-1 transmits a first acoustic transmission 402-1 using speaker 308, which propagates at least partially inside at least a portion of the right ear canal 120 of user 106. The first earable 102-1 receives a first acoustic reception signal 404-1 using microphone 310, which may be a feedback microphone 330. In this example, an acoustic circuit is formed that includes seal 118, earable 102-1, right ear canal 120, and tympanic membrane 122 of right ear 108. The first acoustic reception signal 404-1 represents a version of the first acoustic transmission signal 402-1 that has been at least partially modified by the acoustic circuit associated with the right ear canal 120. This modification can change the amplitude, phase, and / or frequency of the first acoustic reception signal 404-1 relative to the first acoustic transmission signal 402-1.

[0045] Similarly, the second hearable 102-2 transmits a second acoustic transmission signal 402-2 using the speaker 308, which propagates at least partially inside the left ear canal 120 of the user 106. The second hearable 102-2 receives a second acoustic reception signal 404-2 using the microphone 310, which can be the feedback microphone 330. The second acoustic reception signal 404-2 represents a version of the second acoustic transmission signal 402-2 that has been at least partially modified by an acoustic circuit associated with the left ear canal 120. This modification can change the amplitude, phase, and / or frequency of the second acoustic reception signal 404-2 relative to the second acoustic transmission signal 402-2.

[0046] In this example, both hearables 102-1 and 102-2 operate as both transmitters and receivers. More specifically, hearable 102-1 represents the transmitter (or source) of the acoustic transmission signal 402-1 and also represents the receiver (or destination) of the acoustic reception signal 404-1. Similarly, hearable 102-2 represents the transmitter (or source) of the acoustic transmission signal 402-2 and also represents the receiver (or destination) of the acoustic reception signal 404-2.

[0047] The technique of monaural audio prethysmography 110 can be particularly beneficial in at least some aspects of biometric monitoring 112, environmental sensing 116, and facial gesture recognition 114. Thereby, the smart device 104 can also compile information from both hearables 102-1 and 102-2, thereby further improving the reliability of the measurements. In some aspects of audio prethysmography 110, it may be beneficial to analyze the acoustic channels between the two ears 108, as will be further described with respect to FIG. 4-2.

[0048] FIG. 4-2 shows an exemplary co-operation of two earables 102-1 and 102-2 that perform binaural audio plethysmography 110. In environment 400-2, earables 102-1 and 102-2 together perform audio plethysmography 110 across two ears 108 of user 106. In this case, at least one of the earables 102 (e.g., the first earable 102-1) includes a speaker 308 and at least one of the other earables 102 (e.g., the second earable 102-2) includes a microphone 310. Earables 102-1 and 102-2 operate together bistatically during the same period.

[0049] During operation, the first hearable 102-1 transmits a first acoustic transmission 402 using the speaker 308. The acoustic transmission signal 402 propagates through the right ear canal 120 of the user 106. The acoustic transmission signal 402 also propagates through the acoustic channel existing between the right and left ears 108. In the left ear 108, the acoustic transmission signal 402 propagates through the left ear canal 120 of the user 106 and is represented as an acoustic reception signal 404. The second hearable 102-2 receives the acoustic reception signal 404 using the microphone 310. In this example, an acoustic circuit is formed that includes the seals 118 associated with the hearables 102-1 and 102-2, the hearable 102-1, the right ear canal 120, the eardrum 122 of the right ear 108, the acoustic channel between the right and left ears 108, the eardrum 122 of the left ear 108, the left ear canal 120, and the hearable 102-2. The acoustic reception signal 404 represents a version of the acoustic transmission signal 402 that has been modified by the acoustic circuit associated with the right ear canal 120, by the acoustic channel associated with the user 106's face, and by the acoustic circuit associated with the left ear canal 120. This modification can change the amplitude, phase, and / or frequency of the acoustic reception signal 404 relative to the acoustic transmission signal 402. In some cases, the hearable 102-2 measures the time of flight (ToF) associated with the propagation from the first hearable 102-1 to the second hearable 102-2. In some cases, a combination of monaural and binaural audio plethysmography 110 is applied to further improve the reliability of the measurement. The monaural and binaural audio plethysmography 110 can occur during the same period or different periods.

[0050] In this example, the hearable 102-1 operates as a transmitter and the hearable 102-2 operates as a receiver. More specifically, the hearable 102-1 represents the transmitter (or source) of the acoustic transmission signal 402. In contrast, the hearable 102-2 represents the receiver (or destination) of the acoustic reception signal 404.

[0051] The acoustic transmission signals 402 in FIGS. 4-1 and 4-2 can represent various different types of signals. As previously described with respect to FIG. 3, the acoustic transmission signal 402 can be an ultrasonic signal and / or an audible signal. Also, the acoustic transmission signal 402 can be a continuous wave signal or a pulse signal. Some acoustic transmission signals 402 can have a specific tone or frequency. Other acoustic transmission signals 402 can have multiple tones or multiple frequencies. To generate the acoustic transmission signal 402, various modulations can be applied. Exemplary modulations include linear frequency modulation, triangular frequency modulation, stepped frequency modulation, phase modulation, or amplitude modulation. The acoustic transmission signal 402 can be transmitted during an operation mode or a mission mode, as further described with respect to FIGS. 5 and 7. Also, the acoustic transmission signal 402 can be transmitted during a calibration mode, as further described with respect to FIG. 6. An exemplary audio plethysmography measurement module 318 will be further described with reference to FIG. 5.

[0052] FIG. 5 shows an exemplary scheme implemented by the audio plethysmography measurement module 318. In the illustrated configuration, the audio plethysmography measurement module 318 includes at least one audio plethysmography preprocessing pipeline 502 and at least one biometric monitor 322. The audio plethysmography preprocessing pipeline 502 processes digital samples of the acoustic reception signal 404 and outputs the data in a format usable by the biometric monitor 322. The biometric monitor 322 determines one or more physiological metrics (e.g., one or more biometrics) of the user 106 for biometric monitoring 112. In this example, the biometric monitor 322 includes a heart rate detector 504 and / or a respiration rate detector 506. The heart rate detector 504 measures the heart rate of the user 106. The respiration rate detector 506 measures the respiration rate of the user 106.

[0053] Other embodiments are possible where the audio plethysmography measurement module 318 includes a facial behavior detector 324 and / or an environmental detector 326 coupled to the output of the audio plethysmography preprocessing pipeline 502. Generally, the audio plethysmography measurement module 318 can include any combination of the biometric monitor 322, the facial behavior detector 324, and / or the environmental detector 326.

[0054] The audio plethysmography preprocessing pipeline 502 includes at least one demodulator 508, at least one filter 510, and at least one autocorrelation module 512. The demodulator 508 operates as a mixer and can perform the operation of multiplication. The filter 510, which can be implemented as a low-pass filter, is designed to attenuate spurious or unwanted frequencies. Exemplary spurious frequencies include harmonic frequencies generated through the operation of the demodulator 508. The audio plethysmography preprocessing pipeline 502 can optionally include a clutter cancellation module 514. The clutter cancellation module 514 can attenuate other unwanted frequencies passing through the filter 510.

[0055] During audio plethysmography 110, the audio plethysmography preprocessing pipeline 502 receives a digital transmission signal 516 representing a version of the acoustic transmission signal 402. In some embodiments, the system processor 314 generates the digital transmission signal 516 in the digital domain and passes the digital transmission signal 516 to the analog circuitry 312 to enable transmission of the acoustic transmission signal 402 via the transducer 306. The audio plethysmography preprocessing pipeline 502 also receives a digital reception signal 518 from the analog circuitry 312. The digital reception signal 518 represents a digital version of the acoustic reception signal 404.

[0056] Demodulator 508 uses the digital transmission signal 516 to decode the digital reception signal 518 and generate a mixed signal 520. As an example, demodulator 508 can multiply or perform a beat operation to combine the digital transmission signal 516 with the digital reception signal 518. For example, demodulator 508 can apply in-phase and quadrature (IQ) mixing to the digital reception signal 518 using the digital transmission signal 516. Referring to Equation 2 above, the in-phase digital transmission signal 516 can be given by S I (t)=cos(Ω fc (t)), and demodulator 508 can then perform the multiplication of S(t) and S I (t). Filter 510 filters the mixed signal 520 to generate a filtered signal 522. By the operation of filter 510, some of the higher frequency components of the filtered signal 522 can be attenuated with respect to the mixed signal 520. Based on the filtering, for example when applying IQ mixing to the digital reception signal 518, the in-phase part I(t) and the quadrature phase part Q(t) can be determined together with the amplitude R(t)=√(I(t) 2 +Q(t) 2 ) or the phase Φ(t)=arctan(Q(t) / I(t)) of the digital reception signal 518.

[0057] In a first exemplary embodiment, the autocorrelation module 512 receives the filtered signal 522 and applies an autocorrelation function to generate an autocorrelation 524. The biometric monitor 322 analyzes the autocorrelation 524 to measure a physiological metric of the user 106. For example, the heart rate detector 504 detects the peaks 526 of the autocorrelation 524 and measures the time intervals between the peaks 526. This time interval, i.e., the period of the autocorrelation 524, represents the heart beat. At 528, a graph of an exemplary autocorrelation 524 is shown having peaks 526-1 and 526-2, which can be used to determine the heart rate. A similar process can be performed to measure the respiratory rate using the respiratory rate detector 506.

[0058] In some cases, frequencies associated with other physiological metrics or noise can make it more difficult to accurately measure the desired physiological metric. To address this, the audio plethysmography preprocessing pipeline 502 can apply a clutter cancellation module 514. Instead of sending the filtered signal 522 directly to the autocorrelation module 512, the clutter cancellation module 514 operates on the filtered signal 522 to generate a modified filtered signal 526. For example, the clutter cancellation module 514 can attenuate frequencies outside the range associated with the heart rate. These can include slower frequencies associated with the breathing rate of the user 106 and / or frequencies associated with the movement of the hearable 102.

[0059] In an exemplary embodiment, the clutter cancellation module 514 applies curve fitting (e.g., 5th degree polynomial curve fitting) to the filtered signal 522 to generate a fitted curve. The fitted curve has frequencies that at least partially incorporate frequencies associated with unwanted noise or other physiological metrics. Next, the clutter cancellation module 514 subtracts the fitted curve from the filtered signal 522 to generate a modified filtered signal 526. The modified filtered signal 526 is passed to the autocorrelation module 512, and the measurement process can continue as described above.

[0060] The frequencies of some transmissions may be more suitable for the audio plethysmography 110 than other frequencies. The desired frequencies can depend at least in part on the quality of the seal 118 and the physical structure of the ear canal 120. To determine the desired frequencies, the hearable 102 can optionally perform a calibration process using the audio plethysmography calibration module 320, which is further described with respect to FIG. 6.

[0061] FIG. 6 shows an exemplary scheme implemented by the audio plethysmography calibration module 320. In the illustrated configuration, the audio plethysmography calibration module 320 includes a demodulator 508, a filter 510, and at least one frequency selector 602. The frequency selector 602 selects one or more acoustic frequencies for the audio plethysmography 110. In an exemplary embodiment, the frequency selector 602 includes a differentiation module 604, a zero-crossing detector 606, and an evaluator 608. The operation of these components will be further described below.

[0062] During the calibration mode, the hirable 102 transmits an acoustic transmission signal 402 and receives an acoustic reception signal 404. The acoustic transmission signal 402 may have a specific bandwidth on the order of several kilohertz. For example, the acoustic transmission signal 402 can have a bandwidth of about 4, 6, 8, 10, 16, or 20 kilohertz. The audio plethysmography calibration module 320 receives a digital transmission signal 516 representing a version of the acoustic transmission signal 402. The audio plethysmography calibration module 320 also receives a digital reception signal 518 representing a digital version of the acoustic reception signal 404.

[0063] Using the digital transmission signal 516, the demodulator 508 demodulates the digital reception signal 518 to generate a mixed signal 520 as described above with respect to FIG. 5. The filter 510 filters the mixed signal 520 to attenuate unwanted or undesirable frequencies and generates a filtered signal 522.

[0064] The differentiation module 604 calculates the second derivative of the frequency response of the filtered signal 522 to generate a differentiation 610. The zero-crossing detector 606 identifies the frequencies within the differentiation 610 associated with zero crossings. These zero-crossing frequencies 612 represent frequencies that are particularly sensitive to changes in the acoustic channel or acoustic circuit. The zero-crossing frequencies 612 are passed to the evaluator 608.

[0065] Evaluator 608 identifies one or more zero-crossing frequencies 612 for audio plethysmography 110 represented by the selected frequency 614. To determine the selected frequency 614, evaluator 608 can take into account the difference between adjacent zero-crossing frequencies 612 and / or the amount of energy in the filtered signal 522 at the zero-crossing frequencies 612. Generally, evaluator 608 selects a frequency that is far enough apart to reduce interference and has sufficient energy to perform audio plethysmography 110. Using the resulting selected frequency 614 (or frequencies 614), accurate results for audio plethysmography 110 can be achieved. As an example, evaluator 608 can select 1, 2, 3, 4, 6, or 10 different frequencies.

[0066] In some cases, evaluator 608 may apply an autocorrelation function to evaluate the performance of each selected frequency 614. A selected frequency 614 that produces an autocorrelation function with a peak-to-average ratio greater than a predetermined threshold can be a candidate for selection.

[0067] The hearable 102 can transmit a subsequent acoustic transmission signal 402 for the audio plethysmograph 110 using at least one of the selected frequencies 614. This calibration process can be performed at the desired frequency to account for changes in the seal 118 and / or changes in the physical structure of the ear canal 120. In some embodiments, the hearable 102 detects the formation of the seal 118 and performs a calibration process based on this detection. The hearable 102 can detect the formation of the seal 118 using the audio plethysmograph 110 or another sensor that performs on-head (or in-ear) detection. Also, the calibration process can be performed for each ear 108. In some cases, the hearable 102 uses a plurality of selected frequencies 614 to transmit the subsequent acoustic transmission signal 402. In this case, the audio plethysmograph measurement module 318 can execute a plurality of audio plethysmograph preprocessing pipelines 502 as further described with respect to FIG. 7.

[0068] FIG. 7 shows another exemplary scheme implemented by the audio plethysmograph measurement module 318. In this case, the hearable 102 transmits an acoustic transmission signal 402 having a plurality of tones or frequencies, which can be based on the selected frequencies 614 determined during the calibration mode. As shown in FIG. 7, the audio plethysmograph measurement module 318 includes a plurality of audio plethysmograph preprocessing pipelines 502-1 to 502-N. Each of the audio plethysmograph preprocessing pipelines 502-1 to 502-N is designed to process information associated with one of the selected frequencies 614 and generate corresponding autocorrelations 524-1 to 524-N.

[0069] The audio plethysmography measurement module 318 also includes a rank selector 702 that evaluates autocorrelations 524-1 through 524-N and selects the autocorrelation of the highest quality factor. For example, the rank selector 702 can select one of the autocorrelations 524-1 through 524-N that has the highest peak-to-average ratio in the frequency domain of the autocorrelation. This selected autocorrelation 704 is passed to other modules such as the biometric monitor 322, the face behavior detector 324, or the environment detector 326 for further processing. This selection process enables the audio plethysmography measurement module 318 to achieve a higher level of accuracy in performing audio plethysmography 110, including measuring at least one physiological metric as part of biometric monitoring 112. FIGS. 8-11 further graphically illustrate exemplary signals related to the calibration process performed by the audio plethysmography calibration module 320 and described with respect to FIG. 6.

[0070] FIG. 8 shows graphs 800 and 802 of an exemplary mixed signal 520 and an exemplary filtered signal 522. Graphs 800 and 802 show the amplitude versus frequency. Graph 802 represents an enlarged view of a portion of graph 800. As shown in 802, the mixed signal 520 has at least some noise. The filtered signal 522 represents a smoother version of the mixed signal 520.

[0071] FIG. 9 shows a graph 900 of an exemplary differentiation 610 of the filtered signal 522 of FIG. 8. In this example, the differentiation 610 represents a second derivative calculated by the differentiation module 604. The dashed line 902 represents an amplitude of zero. The zero crossing detector 606 calculates and identifies the frequencies at which the differentiation 610 crosses the zero amplitude represented by 902. Based on these zero crossings, several frequencies are identified. These frequencies may be particularly susceptible to the effects of changes in the acoustic channel or acoustic circuit. The frequencies are further described in connection with FIG. 10.

[0072] FIG. 10 shows a graph 1000 in which frequencies 1002-1 to 1002-7 associated with the zero crossings of FIG. 9 are shown for the mixed signal 520 and the filtered signal 522 of FIG. 8. Evaluator 608 evaluates the zero crossing frequencies 1002-1 to 1002-7 and (pre-)selects a subset of the frequencies 1002, taking into account the difference between adjacent zero crossing frequencies and / or the amount of energy within the filtered signal 522 at the zero crossing frequency 1002. As a result, frequencies 1002-1, 1002-3, 1002-6 shown by the solid line may be (pre-)selected, and frequencies 1002-2, 1002-4, 1002-5, and 1002-7 shown by the dashed line may not be selected. By this operation, for example, different frequencies may be (pre-)selected for each ear 108, including the zero crossing frequency 1002 having the highest amplitude. The autocorrelation 524 applied by the evaluator 608 to evaluate the performance of each of the frequencies 1002 selected for the audio plethysmograph 110 will be further described with respect to FIG. 11.

[0073] FIG. 11 shows a graph 1100 showing exemplary autocorrelations 524-1 and 524-2. Autocorrelations 524-1 and 524-2 can be associated with different ones of the frequencies 1002 shown in FIG. 10. As can be seen from the corresponding plots of 524-1 and 524-2, the calculated autocorrelations 524-1 and 524-2 may indicate that at the (pre-)selected frequencies, physiological metrics such as the heart rate of user 106 cannot be determined. Thus, evaluator 608 (finally) selects the frequencies 1002 that generate an autocorrelation 524 with a peak-to-average ratio greater than a predetermined threshold to determine the frequencies 1002 to be used for the audio plethysmograph 110. In this regard, autocorrelation 524-1 can have a sufficiently high peak-to-average ratio, whereby the frequency 1002 associated with it is selected. However, autocorrelation 524-2 has too low a peak-to-average ratio, and the frequency 1002 associated with it is not selected.

[0074] Exemplary method Figures 12 - 14 illustrate exemplary methods 1200, 1300, and 1400 for implementing aspects of the audio plethysmograph 110. Methods 1200, 1300, and 1400 are shown as a set of operations (or actions) to be performed, but the operations are not necessarily limited to the order or combination shown herein. Further, one or more of the operations may be repeated, combined, rearranged, or joined to provide a wide array of additional methods and / or alternative methods. In some portions of the following discussion, reference may be made to the environment 100 of FIG. 1 and the entities detailed in FIGS. 2 and 3, but these references are for illustration only. These techniques are not limited to being performed by one or more entities operating on one device.

[0075] In 1202 of FIG. 12, an acoustic transmission signal is transmitted. The acoustic transmission signal propagates within at least a portion of the user's external auditory canal. For example, at least one speaker 308 transmits the acoustic transmission signal 402. The at least one speaker 308 can represent the speaker of the wearable 102 - 1, the speaker of the wearable 102 - 2, or both. The acoustic transmission signal 402 propagates within at least a portion of the external auditory canal 120 of the user 106 as described with respect to FIGS. 4 - 1 or 4 - 2.

[0076] At 1204, an acoustic reception signal is received. The acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified by propagation inside the external auditory canal. For example, as described with respect to FIGS. 4-1 or 4-2, at least one microphone 310 receives the acoustic reception signal 404. The at least one microphone 310 may represent the microphone 310 of the wearable 102-1, the microphone of the wearable 102-2, or both. The acoustic reception signal 404 represents a version of the acoustic transmission signal 402 having one or more waveform characteristics modified for propagation inside the external auditory canal 120. The waveform characteristics may also be modified, at least in part, by the user's biometrics, by the user's facial behavior, or by the environment around the external auditory canal 120. Exemplary waveform characteristics include amplitude, phase, and / or frequency. In some embodiments, the feedback microphone 330 of the active noise cancellation circuit 328 can receive the acoustic reception signal 404.

[0077] At 1206, at least one physiological metric of the user is determined based on one or more modified waveform characteristics of the acoustic reception signal. For example, the wearable 102 determines at least one physiological metric of the user 106 according to the biometric monitoring 112. Exemplary physiological metrics include heart rate, respiratory rate, blood pressure, body temperature, and carbon dioxide level.

[0078] At 1302 in FIG. 13, an acoustic transmission signal is transmitted. The acoustic transmission signal propagates inside at least a part of the user's external auditory canal. For example, at least one speaker 308 transmits the acoustic transmission signal 402. The at least one speaker 308 can represent the speaker of the wearable 102-1, the speaker of the wearable 102-2, or both. The acoustic transmission signal 402 propagates inside at least a part of the external auditory canal 120 of the user 106 as described with respect to FIGS. 4-1 and 4-2.

[0079] At 1304, an acoustic reception signal is received. The acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified for propagation inside the ear canal. For example, as described with respect to FIGS. 4-1 or 4-2, at least one microphone 310 receives the acoustic reception signal 404. The at least one microphone 310 may represent the microphone 310 of the wearable 102-1, the microphone 310 of the wearable 102-2, or both. The acoustic reception signal 404 represents a version of the acoustic transmission signal 402 having one or more waveform characteristics modified for propagation inside the ear canal 120. When the user 106 breathes, the composition of the gas inside the ear canal 120 changes as shown in FIG. 1-3. In particular, the carbon dioxide concentration changes, which affects the speed of sound inside the ear canal 120. Exemplary waveform characteristics may include amplitude, phase, and / or frequency. In some embodiments, the feedback microphone 330 of the active noise cancellation circuit 328 can receive the acoustic reception signal 404.

[0080] At 1306, the user's breathing rate is determined by analyzing one or more waveform characteristics of the acoustic reception signal. For example, as described with respect to FIG. 5, the wearable 102 uses the audio plethysmography measurement module 318 and the breathing rate detector 506 to determine the breathing rate based on one or more waveform characteristics of the acoustic reception signal 404.

[0081] Optionally, at 1308, the breathing rate is communicated to the smart device, enabling the smart device to display the breathing rate to the user. For example, the wearable 102 transmits the breathing rate to the smart device 104, enabling the smart device 104 to transmit (e.g., display) the breathing rate to the user 106.

[0082] In 1402 of FIG. 14, a calibration process is performed to identify at least one acoustic frequency suitable for audio prethysmography having at least one speaker and at least one microphone. For example, the wearable 102 uses at least one speaker 308, at least one microphone 310, and an audio prethysmography calibration module 320 to perform a calibration process to identify at least one acoustic frequency suitable for audio prethysmography 110 as described with respect to FIG. 6.

[0083] In 1404, the audio prethysmography is performed using at least one acoustic frequency at the user's ear. For example, the wearable 102 performs the audio prethysmography 110 using the selected frequency 614. Specifically, the wearable 102 uses at least one acoustic frequency (e.g., transmits an acoustic transmission signal 402 using the selected frequency 614) to perform the audio prethysmography at the user 106's ear 108 (e.g., at one or both ears 108). The wearable 102 uses the audio prethysmography measurement module 318 to analyze the received acoustic reception signal 404.

[0084] In some situations, the methods 1200, 1300, and / or 1400 are performed using one wearable 102 for monaural audio prethysmography 110 as described with respect to FIG. 4-1. In other situations, the methods 1200, 1300, and / or 1400 are performed using two wearables 102 for binaural audio prethysmography 110 as described with respect to FIG. 4-2.

[0085] Exemplary computing system FIG. 15 shows various components of an exemplary computing system 1500 that can be implemented as any type of client, server, and / or computing device as described with reference to FIGS. 2 and 3 above for implementing aspects of audio prethysmography calibration.

[0086] Computing system 1500 includes a communication device 1502 that enables wired and / or wireless communication of device data 1504 (e.g., received data, data being received, data scheduled for broadcast, or data packets of data). The communication device 1502 or the computing system 1500 can include one or more wearables 102. The device data 1504 or other device content can include settings for device configuration, media content stored on the device, and / or information related to the user of the device. The media content stored in the computing system 1500 can include any type of audio, video, and / or image data. The computing system 1500 includes one or more data inputs 1506 through which any kind of data, media content, and / or input can be received, such as human speech, user-selectable inputs (explicit or implicit), messages, music, television media content, recorded video content, and any other kind of audio, video, and / or image data received from any content and / or data source that one wishes to receive.

[0087] The computing system 1500 also includes a communication interface 1508, which can be implemented as any one or more of a serial interface and / or a parallel interface, a wireless interface, any kind of network interface, a modem, and any other kind of communication interface. The communication interface 1508 provides a connection and / or communication link between the computing system 1500 and a communication network, through which other electronic devices, computing devices, and communication devices communicate data with the computing system 1500.

[0088] Computing system 1500 includes one or more processors 1510 (e.g., any of a microprocessor, a controller, etc.) that process various computer-executable instructions to control the operation of computing system 1500. Alternatively or additionally, computing system 1500 can be implemented using any one or combination of hardware, firmware, or fixed logic circuitry implemented in connection with processing and control circuitry generally designated as 1512. Although not shown, computing system 1500 can include a system bus or data transfer system that couples various components within the device. The system bus can include any one or combination of various bus structures, examples of which include a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor bus or local bus that utilizes any of various bus architectures.

[0089] Computing system 1500 also includes computer-readable media 1514, such as one or more memory devices that enable persistent and / or non-transitory data storage (i.e., as opposed to merely transmitting a signal), examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and disk storage devices. The disk storage device can be implemented as any type of magnetic or optical storage device, for example, a hard disk drive, a recordable and / or rewritable compact disk (CD), any type of digital versatile disk (DVD), etc. Computing system 1500 can also include a mass storage media device (storage media) 1516.

[0090] The computer-readable medium 1514 provides a data storage mechanism for storing device data 1504, as well as various device applications 1518 and any other kind of information and / or data related to the operational aspects of the computing system 1500. For example, the operating system 1520 is maintained as a computer application using the computer-readable medium 1514 and can be executed by the processor 1510. The device applications 1518 can include any form of control application, software application, signal processing and control module, code specific to a particular device, a device manager such as a hardware abstraction layer for a particular device, etc.

[0091] The device applications 1518 also include any system component, engine, or manager for performing audiopretismography 110. In this example, the device applications 1518 include the audiopretismography-based application 206 (APG-based application 206) of FIG. 2, the audiopretismography measurement module 318 of FIG. 3, and optionally, the audiopretismography calibration module 320 of FIG. 3.

[0092] Several examples will be described below. Example 1: A method comprising: performing a calibration process to identify at least one acoustic frequency suitable for audiopretismography using at least one speaker and at least one microphone; and using the at least one acoustic frequency to perform the audiopretismography in the user's ear.

[0093] Example 2: Detecting whether the user's external ear canal is at least partially sealed by a device including the at least one speaker and / or the at least one microphone; and In response to said detecting, starting to execute said calibration process; The method according to Example 1, further comprising.

[0094] Example 3: Executing the calibration process includes executing a first calibration process in a first ear using a first speaker of the at least one speaker and a first microphone of the at least one microphone; Said using the at least one acoustic frequency includes using the at least one acoustic frequency to perform the audio plethysmography in the first ear; The method comprises Executing a second calibration process to identify at least one second acoustic frequency suitable for the audio plethysmography using a second speaker of the at least one speaker and a second microphone of the at least one microphone in a second ear of the user, and Using the at least one second acoustic frequency to perform the audio plethysmography in the second ear; The method according to Example 1 or 2, further comprising.

[0095] Example 4: Further comprising transmitting audible content during at least a part of the time when the calibration process is executed or during at least a part of the time when the audio plethysmography is executed, the method according to any one of the preceding examples.

[0096] Example 5: Executing the calibration process includes Transmitting a first acoustic transmission signal that propagates inside at least a part of the external auditory canal of the user and has a plurality of frequencies; Receiving a first acoustic reception signal having one or more waveform characteristics modified based on the propagation inside the external auditory canal, and Selecting the at least one acoustic frequency from the plurality of frequencies based on the one or more waveform characteristics. The method according to any one of the preceding embodiments, comprising

[0097] Example 6: said selecting said at least one acoustic frequency comprises demodulating said first acoustic reception signal by mixing a digital version of said first acoustic reception signal with a digital version of said first acoustic transmission signal to generate a first mixed signal; passing said first mixed signal through a low-pass filter to generate a first filtered signal; determining a second derivative of said first filtered signal; identifying a zero-crossing frequency associated with said second derivative of said first filtered signal; and selecting said at least one acoustic frequency from said zero-crossing frequency. The method according to Example 5, comprising

[0098] Example 7: said transmitting said first acoustic transmission signal comprises transmitting said first acoustic transmission signal having a bandwidth of at least 4 kilohertz, the method according to Example 5 or 6.

[0099] Example 8: said first acoustic transmission signal is at least one of an ultrasonic signal having a frequency between about 20 kilohertz and 2 megahertz, or an audible signal having a frequency between about 20 hertz and 20 kilohertz, the method according to any one of Examples 5 to 7.

[0100] Example 9: said using said at least one acoustic frequency comprises transmitting a second acoustic transmission signal having said at least one acoustic frequency, said second acoustic transmission signal propagating within at least a portion of the external auditory canal of said user, said method comprising Receiving a second acoustic reception signal representing a version of the second acoustic transmission signal having one or more waveform characteristics modified based on the propagation within the ear canal, and Determining at least one physiological metric of the user based on the one or more modified waveform characteristics of the second acoustic reception signal, The method according to any one of the preceding embodiments, further comprising.

[0101] Example 10: Determining the at least one physiological metric of the user comprises Demodulating the second acoustic reception signal by mixing a digital version of the second acoustic reception signal with a digital version of the second acoustic transmission signal to generate a second mixed signal, Passing the second mixed signal through a low-pass filter to generate a second filtered signal, Generating an autocorrelation of the second filtered signal, and Determining a period of the autocorrelation of the second filtered signal to determine the at least one physiological metric, The method according to Example 9, comprising.

[0102] Example 11: The at least one physiological metric is one of the following The heart rate of the user, or The respiratory rate of the user, The method according to Example 9 or 10, comprising at least one of.

[0103] Example 12: A device comprising At least one speaker, At least one microphone, and At least one processor, The device is configured to perform any one of the methods of Examples 1 to 11 using the at least one speaker, the at least one microphone, and the at least one processor.

[0104] Example 13: The device according to Example 12, further comprising an active noise cancellation circuit including the at least one microphone.

[0105] Example 14: The device according to Example 13, wherein the at least one speaker and the at least one microphone are configured to be disposed proximate to one ear of the user.

[0106] Example 15: The at least one speaker is configured to be disposed proximate to the first ear of the user, and the at least one microphone is configured to be disposed near the second ear, of the device according to Example 12.

[0107] Example 16: The device according to any one of Examples 12 to 15, wherein the at least one speaker and / or the at least one microphone is a part of at least one transducer of the device.

[0108] Example 17: The device according to any one of Examples 12 to 16, wherein the device is configured to at least partially seal one or more ears of the user.

[0109] Example 18: The device includes at least one earphone, or headphones, of the device according to any one of Examples 12 to 17.

[0110] Conclusion Techniques for facilitating audiopretysmography calibration and devices including the same have been described in terms specific to features and / or methods, but it will be understood that the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary embodiments of facilitating audiopretysmography calibration.

Claims

1. A method comprising: performing a calibration process to identify at least one acoustic frequency suitable for audio pre-stimography using at least one speaker and at least one microphone; and using the at least one acoustic frequency to perform the audio pre-stimography on the user's ear.

2. detecting whether the user's external auditory canal is at least partially sealed by a device including the at least one speaker and / or the at least one microphone; and in response to the detecting, starting to perform the calibration process.

3. Performing the calibration process includes performing a first calibration process on a first ear using a first speaker of the at least one speaker and a first microphone of the at least one microphone, using the at least one acoustic frequency includes using the at least one acoustic frequency to perform the audio pre-stimography on the first ear, The method further includes: performing a second calibration process on a second ear of the user to identify at least one second acoustic frequency suitable for the audio pre-stimography using a second speaker of the at least one speaker and a second microphone of the at least one microphone; and using the at least one second acoustic frequency to perform the audio pre-stimography on the second ear.

4. The method according to any one of the preceding claims, further comprising transmitting audible content during at least a portion of the time the calibration process is performed or during at least a portion of the time the audio pre-stimography is performed.

5. Performing the calibration process includes: transmitting a first acoustic transmission signal having a plurality of frequencies and propagating at least partially inside at least a portion of the user's external auditory canal; receiving a first acoustic reception signal representing a version of the first acoustic transmission signal having one or more waveform characteristics modified based on the propagation inside the external auditory canal; and Selecting the at least one acoustic frequency from the plurality of frequencies based on the one or more waveform characteristics, comprising the method according to any one of the preceding claims.

6. Said selecting the at least one acoustic frequency comprises demodulating the first acoustic reception signal by mixing a digital version of the first acoustic reception signal with a digital version of the first acoustic transmission signal to generate a first mixed signal; passing the first mixed signal through a low-pass filter to generate a first filtered signal; determining a second derivative of the first filtered signal; identifying a zero-crossing frequency associated with the second derivative of the first filtered signal; and selecting the at least one acoustic frequency from the zero-crossing frequency, comprising the method according to claim 5.

7. Said transmitting the first acoustic transmission signal comprises transmitting the first acoustic transmission signal having a bandwidth of at least 4 kilohertz, comprising the method according to claim 5 or 6.

8. The first acoustic transmission signal is one of the following an ultrasonic signal having a frequency between about 20 kilohertz and 2 megahertz, or an audible signal having a frequency between about 20 hertz and 20 kilohertz, comprising the method according to any one of claims 5 to 7.

9. Said using the at least one acoustic frequency comprises transmitting a second acoustic transmission signal having the at least one acoustic frequency, the second acoustic transmission signal propagating within at least a portion of the user's external auditory canal, The method further comprises receiving a second acoustic reception signal representative of a version of the second acoustic transmission signal having one or more waveform characteristics modified based on the propagation within the external auditory canal; and determining at least one physiological metric of the user based on the modified one or more waveform characteristics of the second acoustic reception signal, comprising the method according to any one of the preceding claims.

10. Said determining the at least one physiological metric of the user comprises demodulating the second acoustic reception signal by mixing a digital version of the second acoustic reception signal with a digital version of the second acoustic transmission signal to generate a second mixed signal; Passing the second mixed signal through a low-pass filter to generate a second filtered signal; Generating an autocorrelation of the second filtered signal; and Determining a period of the autocorrelation of the second filtered signal to determine the at least one physiological metric, The method according to claim 9, comprising.

11. The at least one physiological metric includes at least one of the following: The heart rate of the user, or The respiratory rate of the user, The method according to claim 9 or 10, comprising at least one of.

12. A device, comprising: At least one speaker, At least one microphone, and At least one processor, The device is configured to perform any one of the methods according to claims 1 to 11 using the at least one speaker, the at least one microphone, and the at least one processor.

13. The device according to claim 12, further comprising an active noise cancellation circuit including the at least one microphone.

14. The device according to claim 13, wherein the at least one speaker and the at least one microphone are configured to be disposed proximate to one ear of a user.

15. The at least one speaker is configured to be disposed proximate to a first ear of the user, The device according to claim 12, wherein the at least one microphone is configured to be disposed near a second ear.

16. The device according to any one of claims 12 to 15, wherein the at least one speaker and / or the at least one microphone is part of at least one transducer of the device.

17. The device according to any one of claims 12 to 16, wherein the device is configured to at least partially seal one or more ears of a user.

18. The device is At least one earphone, or The device according to any one of claims 12 to 17, comprising headphones.

Images