Detection of Respiratory Rate

Abstract

Techniques and devices for performing respiration rate sensing are described. According to one or more preferred embodiments, a novel physiological monitoring process, called audio plethysmography (110) herein, is provided that can perform an active acoustic method capable of sensing subtle physiologically relevant changes observable in a user's outer and middle ear (108), such as a wearable (102) like an earphone. Instead of relying on other auxiliary sensors such as optical or electrical sensors, audio plethysmography (110) includes transmitting and receiving acoustic signals to monitor a user's respiration rate. Some wearables (102) can be configured to support audio plethysmography (110) without the need for additional hardware, in addition to being relatively unobtrusive. Doing so may enable the size, cost, and power consumption of the wearable (102) to be such that a larger population of people can utilize health monitoring, promoting an improvement in the user experience with the wearable (102).

Description

Background Art

[0001] Due to technological advancements in medicine and healthcare, people are able to live longer and healthier lives. To further achieve this, individuals are becoming interested in tracking their individual health. Health monitoring can encourage individuals to achieve specific fitness goals by tracking the gradual improvement of the performance of bodily functions. Additionally, individuals can monitor the impact of various chronic diseases on their bodies. Through active feedback from health monitoring, individuals can lead active and fulfilling lives while having many chronic diseases and can recognize situations where they need to seek medical attention promptly.

[0002] However, some devices that support health monitoring can be overly conspicuous and uncomfortable. Therefore, people may choose to refrain from health monitoring if the device negatively affects their movements or causes inconvenience during daily activities. Thus, it is desirable for health monitoring devices to be reliable, portable, and inexpensive in order to encourage more users to utilize these features.

Summary of the Invention

[0003] Technologies and apparatuses for implementing respiration rate sensing are described. The aspects described below in this context include a method for sensing respiratory rate. The method includes transmitting an acoustic transmission signal that propagates within at least a portion of the user's external ear canal. The method also includes receiving an acoustic reception signal. The acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified by propagation within the ear canal. The method further includes determining the user's respiratory rate by analyzing one or more waveform characteristics of the acoustic reception signal. Exemplary waveform characteristics include amplitude, phase, and / or frequency. Generally, the acoustic reception signal can result from the initially transmitted acoustic transmission signal being affected with respect to at least one of its amplitude, phase, and frequency when propagating within the ear canal before being received via at least one microphone.

[0004] Accordingly, the proposed method may be related to a novel physiological monitoring process, called audio plethysmography herein, an active acoustic method capable of sensing subtle physiologically related changes observable in the user's outer and middle ear. Instead of relying on other auxiliary sensors such as optical or electrical sensors, audio plethysmography involves transmitting and receiving acoustic signals that at least partially propagate within the user's external ear canal. By transmitting and receiving acoustic signals that have propagated within the ear canal, the user's respiratory rate can be determined.

[0005] The aspects described below include a device comprising at least one speaker, at least one microphone, and at least one processor. The device is configured to perform any of the methods described.

[0006] The aspects described below also include a system comprising means for performing sensing of respiratory rate. Generally, proposed solutions can involve using a wearable, or another object or device incorporating a wearable (e.g., glasses, hat, earphones, or helmet) to determine a user's respiratory rate via audio plethysmography. According to one or more preferred embodiments, a wearable such as earphones capable of performing the proposed method is provided. To perform audio plethysmography more effectively, the wearable can form at least a partial seal within or around the user's outer ear. This seal enables the formation of an acoustic circuit including the seal, at least one wearable, at least one ear canal, and at least one eardrum of at least one ear. By transmitting and receiving acoustic signals, the wearable can recognize changes in the acoustic circuit and monitor the user's respiratory rate. In addition to being relatively unobtrusive, some wearables can be configured to support audio plethysmography without requiring additional hardware. Doing so may enable the size, cost, and power consumption of wearables to be such that a larger population of people can utilize health monitoring, promoting an improved user experience with wearables.

[0007] Apparatus and techniques for facilitating the sensing of respiratory rate will be described with reference to the following drawings. The same numbers are used throughout the drawings to refer to like features and components.

Brief Description of the Drawings

[0008]

Figure 1-1

Figure 1-2

Figure 1-3

Figure 2

Figure 3

Figure 4-1

Figure 4-2

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Best Mode for Carrying Out the Invention

[0009] Due to technological advancements in medicine and healthcare, people are able to live longer and healthier lives. To further achieve this, individuals are becoming interested in tracking their individual health. Health monitoring can encourage individuals to achieve specific fitness goals by tracking the gradual improvement of the performance of bodily functions. Additionally, individuals can use health monitoring to observe the physical changes caused by chronic diseases. Through active feedback from health monitoring, individuals can lead active and fulfilling lives while having many chronic diseases and can recognize situations where they need to seek medical attention promptly.

[0010] However, some health monitoring devices can be overly conspicuous and uncomfortable. For example, to measure carbon dioxide levels, some devices collect blood samples from users. Other devices may utilize auxiliary sensors, including optical or electronic sensors, which increase additional weight, cost, complexity, and / or bulk. Still other devices may require continuous battery recharging due to relatively high power consumption. Therefore, when health monitoring devices negatively impact their own operation or cause inconvenience when performing daily activities, people may choose to refrain from health monitoring. Thus, it is desirable for health monitoring devices to be reliable, portable, efficient, and inexpensive to expand access for more users.

[0011] To address this issue and provide new functionality for tracking an individual's health, for example, by using an existing wearable, a technique for implementing respiration rate sensing is described. According to one or more preferred embodiments, there is provided a wearable such as an earphone that can perform a novel physiological monitoring process called audio plethysmography herein, an active acoustic method capable of sensing subtle physiologically relevant changes observable in a user's outer and middle ear. Instead of relying on other auxiliary sensors such as optical or electrical sensors, audio plethysmography involves transmitting and receiving acoustic signals that at least partially propagate inside the user's ear canal. To more appropriately perform audio plethysmography, the wearable may form at least a partial seal within or around the user's outer ear. This seal enables the formation of an acoustic circuit that includes the seal, at least one wearable, at least one ear canal, and at least one tympanic membrane of at least one ear. By transmitting and receiving acoustic signals, the wearable can recognize changes in the acoustic circuit and monitor the user's respiration rate. In addition to being relatively unobtrusive, the proposed solution can be configured to support audio plethysmography without the need for additional hardware, despite being a wearable for placement, for example, near the user's ear. Doing so may encourage the size, cost, and power consumption of the wearable to enable a larger population of people to utilize health monitoring and improve the user experience with the wearable.

[0012] Wireless technology has penetrated into daily life, enabling users to easily access communication and data. One type of wireless technology is wireless wearables, examples of which include wireless earphones and wireless headphones. Wireless wearables have enabled users to move freely while listening to audio content from music, audiobooks, podcasts, and videos. The popularity of wireless wearables has created a market for adding additional features to existing wearables using current hardware (e.g., without introducing any new hardware). Therefore, the proposed solution can be implemented, in particular, by wireless wearables. The wearables can be stand-alone devices or integrated into another object or device such as glasses, hats, earmuffs, or helmets.

[0013] Operating Environment Figure 1-1 is an illustration of an exemplary environment 100 that can implement the sensing of the respiratory rate. In the exemplary environment 100, a wearable 102 is connected to a smart device 104 using a physical interface or a wireless interface. The wearable 102 is a device that can play the audible content provided by the smart device 104 and direct the audible content towards the ear 108 of the user 106. In this example, the wearable 102 operates with the smart device 104. In other examples, the wearable 102 can operate or be implemented as a stand-alone device. Although shown as a smartphone, the smart device 104 can include other types of devices including those described with respect to Figure 2.

[0014] The hearable 102 can execute audio plethysmography 110, which is an acoustic perception method that occurs in the ear 108. The hearable 102 can perform this perception based only on the evaluation of the transmitted and received acoustic signals, thus without using other auxiliary sensors such as optical sensors or electrical sensors. By means of the audio plethysmography 110, the hearable 102 can perform biometric monitoring 112, face movement recognition 114, and / or environmental perception 116.

[0015] To use the audio plethysmography 110, the user 106 places the hearable 102 in such a way that at least a partial seal 118 is created around or in the ear 108. Some parts of the ear 108 are shown in FIG. 1 and include the ear canal 120 and the eardrum 122 (ear drum or tympanic membrane). By means of the seal 118, the hearable 102, the ear canal 120, and the eardrum 122 are coupled to each other to form an acoustic circuit. The audio plethysmography 110 includes at least partially measuring the characteristics associated with this acoustic circuit. The characteristics of the acoustic circuit can vary due to various different situations or actions.

[0016] For example, considering FIGS. 1-2, in which changes occur in the physical structure of the ear 108. Exemplary changes to the physical structure include changes in the geometric shape of the ear canal 120 and / or changes in the volume of the ear canal 120. This change can be caused, at least in part, by a slight deformation of the blood vessels in the ear canal 120 caused by the pumping of the user 106's heart. Other changes can also be caused by the movement of the eardrum 122 or the movement of the user 106's jaw.

[0017] At , for example, the tissue around the ear canal 120 and the eardrum 122 itself are slightly "compressed" by the deformation of blood vessels. Due to this compression, the volume of the ear canal 120 is slightly reduced at . However, at , the compression is relieved, and the volume of the ear canal 120 increases slightly compared to . The physical changes inside the ear 108 can modulate the amplitude and / or phase of the acoustic signal propagating through the ear canal 120, as will be further described below.

[0018] During audiopletismography 110, the acoustic signal propagates through at least a part of the ear canal 120. The wearable 102 can receive an acoustic signal representing the overlap of a plurality of acoustic signals propagating along different paths inside the ear canal 120. Each path is associated with a delay (τ) and an amplitude (a). The delay and amplitude can change over time due to subtle changes occurring in the physical structure of the ear canal 120. The received acoustic signal can be represented by Equation 1.

[0019] [Number]

[0020] Wherein, S(t) represents the received acoustic signal, n represents noise, and φ ini represents the relative phase between the received acoustic signal and the transmitted acoustic signal, Ω fc represents the frequency of the transmitted acoustic signal, and t represents the time vector. Since the activity of the user 106's heart can modulate the amplitude and phase of the received acoustic signal, the received acoustic signal can also be described as further shown in Equation 2.

[0021] S(t)=n+(1+h amp (t))cos(h phase (t)+φ ini ++Ω fc (t)) Equation 2 Wherein, h amp (t) represents the amplitude modulator, and h phase (t) represents the phase modulator. For example, two time-varying functions h amp(t) and h phase (t) can be affected by the interaction between the hearable 102 and the ear 108, as well as the physiological activities of the user 106, particularly the activities of the heart. When related to heartbeat-based modulation, for example, h amp (t) = k a sin(φ hr ++Ω hr (t)), and h phase (t) = k p sin(φ hr ++Ω hr (t)) can be assumed. In the formula, k a and k p are modulation intensity coefficients, and Ω hr is the frequency of the user's heart rate. The interaction between the hearable 102 and the ear 108, and the physiological activities of the user 106 modulate the amplitude and phase of the received acoustic signal.

[0022] As another example, considering FIGS. 1 - 3, in which a change in gas composition occurs in the external auditory canal 120. This change is caused at least in part via breathing. When the user 106 breathes, the skin of the user 106 can exchange gas with its surroundings. For example, at 128, an inhalation 130 occurs, and the gas circulation system inside the external auditory canal 120 reduces the carbon dioxide concentration 132. At 134, an exhalation 136 occurs, and the gas circulation system inside the external auditory canal 120 increases the carbon dioxide concentration 132. This change in the carbon dioxide concentration 132 affects the speed of sound, which in turn affects the speed at which the acoustic signal propagates through the external auditory canal 120.

[0023] Returning to FIG. 1-1, the hearable 102 can detect aspects associated with biometric monitoring 112, facial motion recognition 114, and / or environmental sensing 116 using an audio plethysmograph 110. Generally, biometric monitoring 112 can include measuring the user 106's heart rate, respiratory rate, blood pressure, body temperature, and / or carbon dioxide levels. Additionally, the physical structure of the ear canal 120 can be measured using biometric monitoring 112, and / or movements related to impact forces can be detected. Biometric monitoring 112 can enable the hearable 102 to allow the user 106 to track fitness goals or monitor overall health. This can be particularly beneficial when caring for elderly patients or providing care for remote patients. Some types of biometric monitoring 112 may require various qualities of the seal 118. For example, heart rate can be measured with a relatively small seal 118, while respiratory rate may require a better seal 118.

[0024] The audio plethysmograph 110 can also be used for facial motion recognition 114 which may include detecting jaw clenching, recognizing the start of speech, and / or recognizing certain activities involving the jaw (e.g., speaking or eating). Other types of facial motion recognition 114 include recognizing facial expressions, tracking the user 106's gaze or head pose, and / or recognizing face-touch gestures. To provide some of these functions, the audio plethysmograph 110 can analyze an acoustic channel formed between the left and right ears 108. This acoustic channel can be modified by the user 106's facial expression, gaze, head pose, or touch. Through facial motion recognition 114, the hearable 102 can facilitate communication with speech-impaired and hearing-impaired individuals and / or improve automatic speech recognition. Facial motion recognition 114 also enables a less burdensome user experience since the user 106 can control features of the hearable 102 and / or the smart device 104 without touching the hearable 102.

[0025] The hearable 102 can also support environmental sensing 116 which may include detecting sports activities (e.g., walking or running). By detecting sports activities, the hearable 102 can automatically increase the volume of the user 106's audible content or play audible content from a playlist associated with a workout routine. As another example, the hearable 102 can also automatically detect when the user 106 places the hearable 102 in proximity to their ear 108 to form a seal 118. Thus, the hearable 102 can automatically determine when to play or pause audible content for the user 106 or when to perform biometric monitoring 112 or facial motion recognition 114. The technology of the audio plethysmograph 110 can be executed while the hearable 102 is playing audible content to the user 106. The smart device 104 is further described with respect to FIG. 2.

[0026] Figure 2 shows an exemplary smart device 104. Smart device 104 is shown with respect to various non-limiting exemplary devices including desktop computer 104-1, tablet 104-2, laptop 104-3, television 104-4, computing watch 104-5, computing glasses 104-6, game system 104-7, microwave oven 104-8, and vehicle 104-9. Other devices such as home service devices, smart speakers, smart thermostats, baby monitors, Wi-Fi (registered trademark) routers, drones, trackpads, drawing pads, netbooks, e-book readers, home automation and control systems, wall displays, and other household appliances may also be used. Note that smart device 104 can be wearable, non-wearable but portable, or relatively fixed (e.g., desktop and appliances).

[0027] Smart device 104 includes one or more computer processors 202 and at least one computer-readable medium 204 including a memory medium and a storage medium. Applications and / or operating systems (not shown) embodied as computer-readable instructions to the computer-readable medium 204 can be executed by computer processor 202 to provide some of the functions described herein. Computer-readable medium 204 also includes an audio plethysmography-based application 206 that performs actions using information provided by wearable 102. Exemplary actions can include displaying biometric data to user 106 based on biometric monitoring 112, providing touch-free control of smart device 104 based on face gesture recognition 114, or changing the presentation of audible content based on environmental sensing 116.

[0028] The smart device 104 can also include a network interface 208 for communicating data via a wired, wireless, or optical network. For example, the network interface 208 can communicate data via a local area network (LAN), wireless local area network (WLAN), personal area network (PAN), wide area network (WAN), intranet, Internet, peer-to-peer network, point-to-point network, mesh network, Bluetooth (registered trademark), etc. The smart device 104 can also include a display 210. Although not explicitly shown, the wearable 102 may be integrated inside the smart device 104, or physically or wirelessly connected to the smart device 104. The wearable 102 will be further described with respect to FIG. 3.

[0029] FIG. 3 shows an exemplary wearable 102. The wearable 102 is shown with various non-limiting exemplary devices, including wireless earphones 302-1, wired earphones 302-2, and headphones 302-3. Earphones 302-1 and 302-2 are of the in-ear device type that fits into the ear canal 120. Each earphone 302-1 or 302-2 can represent the wearable 102. Headphones 302-3 can be placed on top of or over the ear 108. Headphones 302-3 can represent closed-back headphones, open-back headphones, on-ear headphones, or over-ear headphones. Some headphones 302-3 include two wearables 102 physically packaged together. In this case, there is one wearable 102 for each ear 108. Other headphones 302-2, such as single-ear headphones 302-2, include one wearable 102. In some embodiments, one or more wearables 102 are implemented inside (or as part of) another device such as glasses, a hat, earmuffs, or a helmet.

[0030] The hearable 102 includes a communication interface 304 for communicating with the smart device 104, although this need not be used when the hearable 102 is integrated inside the smart device 104. The communication interface 304 may be a wired or wireless interface, and audio content is passed from the smart device 104 to the hearable 102. The hearable 102 may also use the communication interface 304 to pass data measured using the audio plethysmograph 110 to the smart device 104. Generally, the data provided by the communication interface 304 is in a format usable by the audio plethysmography-based application 206. The communication interface 304 also enables the hearable 102 to communicate with another hearable 102. During binaural sensing, for example, the hearable 102 can use the communication interface 304 to coordinate with another hearable 102 to support the binaural audio plethysmograph 110, as further described with respect to FIG. 4-2. Specifically, the transmitting hearable 102 can communicate timing and waveform information to the receiving hearable 102 to enable the receiving hearable 102 to properly decode the acoustic signals received.

[0031] The hearable 102 includes at least one speaker and at least one microphone, for example, as part of at least one transducer 306 that can convert an electrical signal into a sound wave. The same transducer 306 or a further transducer of the hearable 102 can also detect a sound wave and convert it into an electrical signal. These sound waves can include ultrasonic frequencies and / or audible frequencies, either of which can be used for audio plethysmography 110. Specifically, the frequency spectrum (e.g., range of frequencies) used by the transducer 306 to generate an acoustic signal can include frequencies from the lower end of the audible region to the upper end of the ultrasonic region, for example, frequencies from 20 Hertz (Hz) to 2 Megahertz (MHz). Other exemplary frequency spectra for audio plethysmography 110 can include frequencies of 20 Hz to 20 kilohertz (kHz), 20 kHz to 2 MHz, 20 to 60 kHz, or 30 to 40 kHz.

[0032] In an exemplary embodiment, the transducer 306 has a monostatic topology. With this topology, the transducer 306 can convert an electrical signal into a sound wave and convert a sound wave into an electrical signal (e.g., can transmit or receive an acoustic signal). Exemplary monostatic transducers can include piezoelectric transducers, capacitive transducers, and micromachine ultrasonic transducers (MUTs) that use microelectromechanical systems (MEMS) technology.

[0033] Alternatively, transducer 306 can be implemented using a bistatic topology that includes a plurality of physically separated transducers. In this case, a first transducer converts an electrical signal into a sound wave (e.g., transmits an acoustic signal), and a second transducer converts the sound wave into an electrical signal (e.g., receives an acoustic signal). An exemplary bistatic topology can be implemented using at least one speaker 308 and at least one microphone 310. The speaker 308 and the microphone 310 may be dedicated to the audio plethysmograph 110, or can be used for other functions of the audio plethysmograph 110 and the smart device 104 (e.g., presenting audible content to the user 106, making a phone call, or capturing the voice of the user 106 for voice control).

[0034] Generally, the speaker 308 and the microphone 310 are directed (e.g., oriented) toward the ear canal 120. Accordingly, the speaker 308 can direct an acoustic signal toward the ear canal 120, and the microphone 310 responds to receiving an acoustic signal from a direction associated with the ear canal 120.

[0035] The wearable 102 includes at least one analog circuit 312, and the analog circuit 312 includes circuits and logic for conditioning electrical signals in the analog domain. The analog circuit 312 can include analog-to-digital converters, digital-to-analog converters, amplifiers, filters, mixers, and switches for generating and modifying electrical signals. In some embodiments, the analog circuit 312 includes other hardware circuits associated with the speaker 308 or the microphone 310.

[0036] The wearable 102 also includes at least one system processor 314 and at least one system medium 316 (e.g., one or more computer-readable storage media). In the illustrated configuration, the system medium 316 includes an audio plethysmography measurement module 318 (APG measurement module 318) and optionally includes an audio plethysmography calibration module 320 (APG calibration module 320). The audio plethysmography measurement module 318 and the audio plethysmography calibration module 320 can be implemented using hardware, software, firmware, or a combination thereof. In this example, the system processor 216 implements the audio plethysmography measurement module 318 and the audio plethysmography calibration module 320. In an alternative example, the computer processor 202 of the smart device 104 can implement at least a portion of the audio plethysmography measurement module 318 and / or at least a portion of the audio plethysmography calibration module 320. In this case, the wearable 102 can communicate digital samples of the acoustic signal to the smart device 104 using the communication interface 304.

[0037] The audio plethysmography measurement module 318 analyzes the received acoustic signal to measure data related to the audio plethysmography 110. The audio plethysmography measurement module 318 can be implemented using at least one biometric monitor 322 for biometric monitoring 112, at least one face behavior detector 324 for face behavior recognition 114, and / or at least one environment detector 326 for environment sensing 116. An exemplary audio plethysmography measurement module 318 is further described with respect to FIGS. 5 and 7.

[0038] The audio plethysmography calibration module 320 can determine appropriate waveform characteristics for transmitting acoustic signals to improve the performance of the audio plethysmography 110. For example, the audio plethysmography calibration module 320 can determine a transmission frequency that may enable the wearable 102 to detect the user's 106 heart rate and / or respiratory rate with an accuracy of 5% or less, taking into account the quality of the seal 118 and the physical structure of the ear canal 120. Using the audio plethysmography calibration module 320, the wearable 102 can dynamically adjust the transmission frequency based on the unique physical structure of each ear 108 each time the seal 118 is formed. Through this calibration process, the wearables 102 in different ears can operate at one or more different acoustic frequencies. Exemplary embodiments of the audio plethysmography calibration module 320 are further described with respect to FIG. 6.

[0039] Some of the hearables 102 include an active noise cancellation circuit 328, which enables the hearable 102 to reduce background noise or environmental noise. In this case, the microphone 310 used for audio prethysmography 110 can be implemented using the feedback microphone 330 of the active noise cancellation circuit 328. During active noise cancellation, the feedback microphone 330 provides feedback information regarding the performance of the active noise cancellation. During audio prethysmography 110, the feedback microphone 330 receives an acoustic signal, which is provided to the audio prethysmography measurement module 318 and / or the audio prethysmography calibration module 320. In some situations, active noise cancellation and audio prethysmography 110 are performed simultaneously using the feedback microphone 330. In this case, the acoustic signal received by the feedback microphone 330 can be provided to at least one of the audio prethysmography modules 318 or 320 and can be provided to the active noise cancellation circuit 328. Different types of audio prethysmography 110 are further described with respect to FIGS. 4-1 and 4-2.

[0040] Audio prethysmography FIG. 4-1 shows an exemplary operation of two earables 102-1 and 102-2 that perform monaural audio plethysmography 110. In environment 400-1, earables 102-1 and 102-2 independently perform audio plethysmography 110 on different ears 108 of user 106. In this case, the first earable 102-1 is proximate to the right ear 108 of user 106, and the second earable 102-2 is proximate to the left ear 108 of user 106. Each earable 102-1 and 102-2 includes a speaker 308 and a microphone 310. Earables 102-1 and 102-2 can operate monostatically during the same period or different periods. In other words, each earable 102-1 and 102-2 can independently transmit and receive acoustic signals.

[0041] For example, the first earable 102-1 transmits a first acoustic transmission 402-1 using speaker 308, which propagates at least partially inside at least a portion of the right ear canal 120 of user 106. The first earable 102-1 receives a first acoustic reception signal 404-1 using microphone 310, which may be a feedback microphone 330. In this example, an acoustic circuit is formed that includes seal 118, earable 102-1, right ear canal 120, and tympanic membrane 122 of right ear 108. The first acoustic reception signal 404-1 represents a version of the first acoustic transmission signal 402-1 that has been at least partially modified by the acoustic circuit associated with the right ear canal 120. This modification can change the amplitude, phase, and / or frequency of the first acoustic reception signal 404-1 relative to the first acoustic transmission signal 402-1.

[0042] Similarly, the second hearable 102-2 transmits a second acoustic transmission signal 402-2 using the speaker 308, which propagates inside at least a part of the left ear canal 120 of the user 106. The second hearable 102-2 receives a second acoustic reception signal 404-2 using the microphone 310, which can be the feedback microphone 330. The second acoustic reception signal 404-2 represents a version of the second acoustic transmission signal 402-2 that has been at least partially modified by an acoustic circuit associated with the left ear canal 120. This modification can change the amplitude, phase, and / or frequency of the second acoustic reception signal 404-2 relative to the second acoustic transmission signal 402-2.

[0043] In this example, both hearables 102-1 and 102-2 operate as transmitters and receivers. More specifically, the hearable 102-1 represents the transmitter (or source) of the acoustic transmission signal 402-1 and also represents the receiver (or destination) of the acoustic reception signal 404-1. Similarly, the hearable 102-2 represents the transmitter (or source) of the acoustic transmission signal 402-2 and also represents the receiver (or destination) of the acoustic reception signal 404-2.

[0044] The technique of monaural audio plethysmography 110 can be particularly beneficial in at least some aspects of biometric monitoring 112, environmental sensing 116, and facial gesture recognition 114. Thereby, the smart device 104 can also compile information from both the hearables 102-1 and 102-2, which can further improve the reliability of the measurements. In some aspects of the audio plethysmography 110, it may be beneficial to analyze the acoustic channel between the two ears 108, as will be further described with respect to FIG. 4-2.

[0045] FIG. 4-2 shows an exemplary co-operation of two earables 102-1 and 102-2 performing binaural audio plethysmography 110. In environment 400-2, earables 102-1 and 102-2 perform binaural audio plethysmography 110 together across two ears 108 of user 106. In this case, at least one of the earables 102 (e.g., the first earable 102-1) includes a speaker 308 and at least one of the other earables 102 (e.g., the second earable 102-2) includes a microphone 310. Earables 102-1 and 102-2 operate together bistatically during the same period.

[0046] During operation, the first wearable 102-1 transmits a first acoustic transmission 402 using the speaker 308. The acoustic transmission signal 402 propagates through the right ear canal 120 of the user 106. The acoustic transmission signal 402 also propagates through the acoustic channel existing between the right and left ears 108. At the left ear 108, the acoustic transmission signal 402 propagates through the left ear canal 120 of the user 106 and is represented as an acoustic reception signal 404. The second wearable 102-2 receives the acoustic reception signal 404 using the microphone 310. In this example, an acoustic circuit is formed that includes the seals 118 associated with the wearables 102-1 and 102-2, the wearable 102-1, the right ear canal 120, the eardrum 122 of the right ear 108, the acoustic channel between the right and left ears 108, the eardrum 122 of the left ear 108, the left ear canal 120, and the wearable 102-2. The acoustic reception signal 404 represents a version of the acoustic transmission signal 402 that has been modified by the acoustic circuit associated with the right ear canal 120, by the acoustic channel associated with the user 106's face, and by the acoustic circuit associated with the left ear canal 120. This modification can change the amplitude, phase, and / or frequency of the acoustic reception signal 404 relative to the acoustic transmission signal 402. In some cases, the wearable 102-2 measures the time of flight (ToF) associated with the propagation from the first wearable 102-1 to the second wearable 102-2. In some cases, a combination of monaural and binaural audio plethysmography 110 is applied to further improve the reliability of the measurement. The monaural and binaural audio plethysmography 110 can occur during the same period or different periods.

[0047] In this example, the wearable 102-1 operates as a transmitter and the wearable 102-2 operates as a receiver. More specifically, the wearable 102-1 represents the transmitter (or source) of the acoustic transmission signal 402. In contrast, the wearable 102-2 represents the receiver (or destination) of the acoustic reception signal 404.

[0048] The acoustic transmission signals 402 in FIGS. 4-1 and 4-2 can represent various different types of signals. As previously described with respect to FIG. 3, the acoustic transmission signal 402 can be an ultrasonic signal and / or an audible signal. Also, the acoustic transmission signal 402 can be a continuous wave signal or a pulse signal. Some acoustic transmission signals 402 can have a specific tone or frequency. Other acoustic transmission signals 402 can have multiple tones or multiple frequencies. Various modulations can be applied to generate the acoustic transmission signal 402. Exemplary modulations include linear frequency modulation, triangular frequency modulation, stepped frequency modulation, phase modulation, or amplitude modulation. The acoustic transmission signal 402 can be transmitted during an operation mode or a mission mode, as further described with respect to FIGS. 5 and 7. Also, the acoustic transmission signal 402 can be transmitted during a calibration mode, as further described with respect to FIG. 6. An exemplary audio plethysmography measurement module 318 is further described with reference to FIG. 5.

[0049] FIG. 5 shows an exemplary scheme implemented by the audio plethysmography measurement module 318. In the illustrated configuration, the audio plethysmography measurement module 318 includes at least one audio plethysmography preprocessing pipeline 502 and at least one biometric monitor 322. The audio plethysmography preprocessing pipeline 502 processes digital samples of the acoustic reception signal 404 and outputs the data in a format usable by the biometric monitor 322. The biometric monitor 322 determines one or more physiological metrics (e.g., one or more biometrics) of the user 106 for biometric monitoring 112. In this example, the biometric monitor 322 includes a heart rate detector 504 and / or a respiration rate detector 506. The heart rate detector 504 measures the heart rate of the user 106. The respiration rate detector 506 measures the respiration rate of the user 106.

[0050] Other embodiments are possible where the audio plethysmography measurement module 318 includes a facial behavior detector 324 and / or an environmental detector 326 coupled to the output of the audio plethysmography preprocessing pipeline 502. In general, the audio plethysmography measurement module 318 can include any combination of the biometric monitor 322, the facial behavior detector 324, and / or the environmental detector 326.

[0051] The audio plethysmography preprocessing pipeline 502 includes at least one demodulator 508, at least one filter 510, and at least one autocorrelation module 512. The demodulator 508 operates as a mixer and can perform the operation of multiplication. The filter 510, which can be implemented as a low-pass filter, is designed to attenuate spurious frequencies or unwanted frequencies. Exemplary spurious frequencies include the harmonic frequencies generated through the operation of the demodulator 508. The audio plethysmography preprocessing pipeline 502 can optionally include a clutter cancellation module 514. The clutter cancellation module 514 can attenuate other unwanted frequencies passing through the filter 510.

[0052] During audio plethysmography 110, the audio plethysmography preprocessing pipeline 502 receives a digital transmission signal 516 representing a version of the acoustic transmission signal 402. In some embodiments, the system processor 314 generates the digital transmission signal 516 in the digital domain and passes the digital transmission signal 516 to the analog circuitry 312 to enable transmission of the acoustic transmission signal 402 via the transducer 306. The audio plethysmography preprocessing pipeline 502 also receives a digital reception signal 518 from the analog circuitry 312. The digital reception signal 518 represents a digital version of the acoustic reception signal 404.

[0053] Demodulator 508 uses the digital transmission signal 516 to decode the digital received signal 518 and generate a mixed signal 520. As an example, the demodulator 508 can multiply or perform a beat operation to combine the digital transmission signal 516 with the digital received signal 518. For example, the demodulator 508 can apply in-phase and quadrature (IQ) mixing to the digital received signal 518 using the digital transmission signal 516. Referring to Equation 2 above, the in-phase digital transmission signal 516 can be given by S I (t)=cos(Ω fc (t)), and the demodulator 508 can then perform the multiplication of S(t) and S I (t). Filter 510 filters the mixed signal 520 to generate a filtered signal 522. By the operation of filter 510, some of the higher frequency components of the filtered signal 522 can be attenuated with respect to the mixed signal 520. Based on the filtering, for example when applying IQ mixing to the digital received signal 518, the in-phase part I(t) and the quadrature phase part Q(t) can be determined along with the amplitude R(t)=√(I(t) 2 +Q(t) 2 ) or the phase Φ(t)=arctan(Q(t) / I(t)) of the digital received signal 518.

[0054] In a first exemplary embodiment, the autocorrelation module 512 receives the filtered signal 522 and applies an autocorrelation function to generate an autocorrelation 524. The biometric monitor 322 analyzes the autocorrelation 524 to measure a physiological metric of the user 106. For example, the heart rate detector 504 detects the peaks 526 of the autocorrelation 524 and measures the time intervals between the peaks 526. This time interval, i.e., the period of the autocorrelation 524, represents the heart beat. At 528, a graph of an exemplary autocorrelation 524 is shown having peaks 526-1 and 526-2, which can be used to determine the heart rate. A similar process can be performed to measure the respiratory rate using the respiratory rate detector 506.

[0055] In some cases, frequencies associated with other physiological metrics or noise can make it more difficult to accurately measure the desired physiological metric. To address this, the audio plethysmography preprocessing pipeline 502 can apply a clutter cancellation module 514. Instead of sending the filtered signal 522 directly to the autocorrelation module 512, the clutter cancellation module 514 operates on the filtered signal 522 to generate a modified filtered signal 526. For example, the clutter cancellation module 514 can attenuate frequencies that are not in the range associated with the heart rate. These can include slower frequencies associated with the respiration rate of the user 106 and / or frequencies associated with the movement of the hearable 102.

[0056] In an exemplary embodiment, the clutter cancellation module 514 applies curve fitting (e.g., fifth-degree polynomial curve fitting) to the filtered signal 522 to generate a fitted curve. The fitted curve has frequencies that at least partially incorporate frequencies associated with unwanted noise or other physiological metrics. Next, the clutter cancellation module 514 subtracts the fitted curve from the filtered signal 522 to generate a modified filtered signal 526. The modified filtered signal 526 is passed to the autocorrelation module 512, and the measurement process can continue as described above.

[0057] The frequencies of some transmissions may be more suitable for the audio plethysmography 110 than other frequencies. The desired frequencies can depend at least in part on the quality of the seal 118 and the physical structure of the ear canal 120. To determine the desired frequencies, the hearable 102 can optionally perform a calibration process using the audio plethysmography calibration module 320, which is further described with respect to FIG. 6.

[0058] FIG. 6 shows an exemplary scheme implemented by the audio plethysmography calibration module 320. In the illustrated configuration, the audio plethysmography calibration module 320 includes a demodulator 508, a filter 510, and at least one frequency selector 602. The frequency selector 602 selects one or more acoustic frequencies for the audio plethysmography 110. In an exemplary embodiment, the frequency selector 602 includes a differentiation module 604, a zero-crossing detector 606, and an evaluator 608. The operation of these components will be further described below.

[0059] During the calibration mode, the hearable 102 transmits the acoustic transmission signal 402 and receives the acoustic reception signal 404. The acoustic transmission signal 402 may have a specific bandwidth of several kilohertz. For example, the acoustic transmission signal 402 can have a bandwidth of about 4, 6, 8, 10, 16, or 20 kilohertz. The audio plethysmography calibration module 320 receives a digital transmission signal 516 representing a version of the acoustic transmission signal 402. Also, the audio plethysmography calibration module 320 receives a digital reception signal 518 representing a digital version of the acoustic reception signal 404.

[0060] Using the digital transmission signal 516, the demodulator 508 demodulates the digital reception signal 518 to generate a mixed signal 520 as described above with respect to FIG. 5. The filter 510 filters the mixed signal 520 to attenuate unwanted or undesirable frequencies and generates a filtered signal 522.

[0061] The differentiation module 604 calculates the second derivative of the frequency response of the filtered signal 522 to generate a differentiation 610. The zero-crossing detector 606 identifies the frequencies within the differentiation 610 associated with the zero-crossings. These zero-crossing frequencies 612 represent frequencies that are particularly sensitive to changes in the acoustic channel or acoustic circuit. The zero-crossing frequencies 612 are passed to the evaluator 608.

[0062] Evaluator 608 identifies one or more zero-crossing frequencies 612 for audioplethysmograph 110 represented by the selected frequency 614. To determine the selected frequency 614, evaluator 608 can take into account the difference between adjacent zero-crossing frequencies 612 and / or the amount of energy in the filtered signal 522 at the zero-crossing frequencies 612. Generally, evaluator 608 selects a frequency that is far enough apart to reduce interference and has sufficient energy to perform audioplethysmograph 110. The resulting selected frequency 614 (or frequencies 614) can be used to achieve accurate results for audioplethysmograph 110. As an example, evaluator 608 can select 1, 2, 3, 4, 6, or 10 different frequencies.

[0063] In some cases, evaluator 608 may apply an autocorrelation function to evaluate the performance of each selected frequency 614. A selected frequency 614 that generates an autocorrelation function with a peak-to-average ratio greater than a predetermined threshold can be a candidate for selection.

[0064] The hearable 102 can transmit a subsequent acoustic transmission signal 402 for the audio plethysmograph 110 using at least one of the selected frequencies 614. This calibration process can be performed at the desired frequency to account for changes in the seal 118 and / or changes in the physical structure of the ear canal 120. In some embodiments, the hearable 102 detects the formation of the seal 118 and performs a calibration process based on this detection. The hearable 102 can detect the formation of the seal 118 using the audio plethysmograph 110 or another sensor that performs on-head (or in-ear) detection. Also, the calibration process can be performed for each ear 108. In some cases, the hearable 102 uses a plurality of selected frequencies 614 to transmit the subsequent acoustic transmission signal 402. In this case, the audio plethysmograph measurement module 318 can execute a plurality of audio plethysmograph preprocessing pipelines 502, as further described with respect to FIG. 7.

[0065] FIG. 7 shows another exemplary scheme implemented by the audio plethysmograph measurement module 318. In this case, the hearable 102 transmits an acoustic transmission signal 402 having a plurality of tones or frequencies, and the acoustic transmission signal 402 can be based on the selected frequencies 614 determined during the calibration mode. As shown in FIG. 7, the audio plethysmograph measurement module 318 includes a plurality of audio plethysmograph preprocessing pipelines 502-1 to 502-N. Each of the audio plethysmograph preprocessing pipelines 502-1 to 502-N is designed to process information associated with one of the selected frequencies 614 and generate corresponding autocorrelations 524-1 to 524-N.

[0066] The audio plethysmography measurement module 318 also includes a rank selector 702 that evaluates the autocorrelations 524-1 through 524-N and selects the autocorrelation of the highest quality factor. For example, the rank selector 702 can select one of the autocorrelations 524-1 through 524-N that has the highest peak-to-average ratio in the frequency domain of the autocorrelation. This selected autocorrelation 704 is passed to other modules such as the biometric monitor 322, the face behavior detector 324, or the environment detector 326 for further processing. This selection process enables the audio plethysmography measurement module 318 to achieve a higher level of accuracy in performing audio plethysmography 110, including measuring at least one physiological metric as part of biometric monitoring 112. FIGS. 8-11 further graphically illustrate exemplary signals related to the calibration process performed by the audio plethysmography calibration module 320 and described with respect to FIG. 6.

[0067] FIG. 8 shows graphs 800 and 802 of an exemplary mixed signal 520 and an exemplary filtered signal 522. Graphs 800 and 802 show the amplitude versus frequency. Graph 802 represents an enlarged view of a portion of graph 800. As shown in 802, the mixed signal 520 has at least some noise. The filtered signal 522 represents a smoother version of the mixed signal 520.

[0068] FIG. 9 shows a graph 900 of an exemplary differentiation 610 of the filtered signal 522 of FIG. 8. In this example, the differentiation 610 represents a second derivative calculated by the differentiation module 604. The dashed line 902 represents zero amplitude. The zero-crossing detector 606 calculates and identifies the frequencies at which the differentiation 610 crosses the zero amplitude represented by 902. Based on these zero crossings, several frequencies are identified. These frequencies may be particularly susceptible to changes in the acoustic channel or acoustic circuit. The frequencies are further described in connection with FIG. 10.

[0069] Figure 10 shows a graph 1000 in which the frequencies 1002-1 to 1002-7 associated with the zero crossings of Figure 9 are shown for the mixed signal 520 and the filtered signal 522 of Figure 8. The evaluator 608 evaluates the zero crossing frequencies 1002-1 to 1002-7 and (pre-)selects a subset of the frequencies 1002 taking into account the difference between adjacent zero crossing frequencies and / or the amount of energy within the filtered signal 522 at the zero crossing frequency 1002. As a result, the frequencies 1002-1, 1002-3, 1002-6 shown by the solid line may (pre-)be selected, and the frequencies 1002-2, 1002-4, 1002-5, and 1002-7 shown by the dashed line may not be selected. By this operation, for example, different frequencies may (pre-)be selected for each ear 108, including the zero crossing frequency 1002 having the highest amplitude. The autocorrelation 524 applied by the evaluator 608 to evaluate the performance of each of the frequencies 1002 selected for the audio plethysmography 110 is further described with respect to Figure 11.

[0070] Figure 11 shows a graph 1100 showing exemplary autocorrelations 524-1 and 524-2. The autocorrelations 524-1 and 524-2 can be associated with different ones of the frequencies 1002 shown in Figure 10. As can be seen from the corresponding plots of 524-1 and 524-2, the calculated autocorrelations 524-1 and 524-2 may indicate that at the (pre-)selected frequencies, physiological metrics such as the heart rate of the user 106 cannot be determined. Thus, the evaluator 608 (finally) selects the frequencies 1002 that generate an autocorrelation 524 with a peak-to-average ratio greater than a predetermined threshold to determine the frequencies 1002 to be used for the audio plethysmography 110. In this regard, the autocorrelation 524-1 can have a sufficiently high peak-to-average ratio, whereby the frequency 1002 associated with it is selected. However, the autocorrelation 524-2 has too low a peak-to-average ratio so that the frequency 1002 associated with it is not selected.

[0071] Exemplary method Figures 12 - 14 illustrate exemplary methods 1200, 1300, and 1400 for implementing aspects of the audio plethysmograph 110. Methods 1200, 1300, and 1400 are shown as a set of operations (or actions) to be performed, but the operations are not necessarily limited to the order or combination shown herein. Further, any one or more of the operations may be repeated, combined, rearranged, or joined to provide a wide array of additional methods and / or alternative methods. In some portions of the following discussion, reference may be made to the environment 100 of FIG. 1 and the entities detailed in FIGS. 2 and 3, but these references are for illustration only. These techniques are not limited to being performed by one entity or multiple entities operating on one device.

[0072] In 1202 of FIG. 12, an acoustic transmission signal is transmitted. The acoustic transmission signal propagates within at least a portion of the user's ear canal. For example, at least one speaker 308 transmits the acoustic transmission signal 402. The at least one speaker 308 can represent the speaker of the wearable 102 - 1, the speaker of the wearable 102 - 2, or both. The acoustic transmission signal 402 propagates within at least a portion of the ear canal 120 of the user 106 as described with respect to FIGS. 4 - 1 or 4 - 2.

[0073] At 1204, an acoustic reception signal is received. The acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified by propagation within the ear canal. For example, as described with respect to FIGS. 4-1 or 4-2, at least one microphone 310 receives the acoustic reception signal 404. The at least one microphone 310 may represent the microphone 310 of the hearable 102-1, the microphone of the hearable 102-2, or both. The acoustic reception signal 404 represents a version of the acoustic transmission signal 402 having one or more waveform characteristics modified due to propagation within the ear canal 120. The waveform characteristics may also be modified at least in part by the user's biometrics, by the user's facial behavior, or by the environment around the ear canal 120. Exemplary waveform characteristics include amplitude, phase, and / or frequency. In some embodiments, the feedback microphone 330 of the active noise cancellation circuit 328 can receive the acoustic reception signal 404.

[0074] At 1206, at least one physiological metric of the user is determined based on one or more modified waveform characteristics of the acoustic reception signal. For example, the hearable 102 determines at least one physiological metric of the user 106 according to the biometric monitoring 112. Exemplary physiological metrics include heart rate, respiratory rate, blood pressure, body temperature, and carbon dioxide level.

[0075] At 1302 of FIG. 13, an acoustic transmission signal is transmitted. The acoustic transmission signal propagates within at least a portion of the user's ear canal. For example, at least one speaker 308 transmits the acoustic transmission signal 402. The at least one speaker 308 can represent the speaker of the hearable 102-1, the speaker of the hearable 102-2, or both. The acoustic transmission signal 402 propagates within at least a portion of the ear canal 120 of the user 106 as described with respect to FIGS. 4-1 and 4-2.

[0076] At 1304, an acoustic reception signal is received. The acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified for propagation inside the ear canal. For example, as described with respect to FIGS. 4-1 or 4-2, at least one microphone 310 receives the acoustic reception signal 404. The at least one microphone 310 may represent the microphone 310 of the hearable 102-1, the microphone 310 of the hearable 102-2, or both. The acoustic reception signal 404 represents a version of the acoustic transmission signal 402 having one or more waveform characteristics modified for propagation inside the ear canal 120. When the user 106 breathes, the composition of the gas inside the ear canal 120 changes as shown in FIG. 1-3. In particular, the carbon dioxide concentration changes, which affects the speed of sound inside the ear canal 120. Exemplary waveform characteristics may include amplitude, phase, and / or frequency. In some embodiments, the feedback microphone 330 of the active noise cancellation circuit 328 can receive the acoustic reception signal 404.

[0077] At 1306, the user's respiratory rate is determined by analyzing one or more waveform characteristics of the acoustic reception signal. For example, as described with respect to FIG. 5, the hearable 102 uses the audio plethysmography measurement module 318 and the respiratory rate detector 506 to determine the respiratory rate based on one or more waveform characteristics of the acoustic reception signal 404.

[0078] Optionally, at 1308, the respiratory rate is communicated to the smart device, enabling the smart device to display the respiratory rate to the user. For example, the hearable 102 transmits the respiratory rate to the smart device 104, enabling the smart device 104 to transmit (e.g., display) the respiratory rate to the user 106.

[0079] In 1402 of FIG. 14, a calibration process is performed to identify at least one acoustic frequency suitable for audio prethysmography using at least one speaker and at least one microphone. For example, the wearable 102 uses at least one speaker 308, at least one microphone 310, and an audio prethysmography calibration module 320 to perform a calibration process to identify at least one acoustic frequency suitable for audio prethysmography 110, as described with respect to FIG. 6.

[0080] In 1404, the audio prethysmography is performed using at least one acoustic frequency in the user's ear. For example, the wearable 102 uses the selected frequency 614 to perform the audio prethysmography 110. Specifically, the wearable 102 uses at least one acoustic frequency (e.g., uses the selected frequency 614 to transmit an acoustic transmission signal 402) to perform audio prethysmography in the user 106's ear 108 (e.g., in one or both ears 108). The wearable 102 uses an audio prethysmography measurement module 318 to analyze the received acoustic reception signal 404.

[0081] In some situations, the methods 1200, 1300, and / or 1400 are performed using one wearable 102 for monaural audio prethysmography 110, as described with respect to FIG. 4-1. In other situations, the methods 1200, 1300, and / or 1400 are performed using two wearables 102 for binaural audio prethysmography 110, as described with respect to FIG. 4-2.

[0082] Exemplary computing system FIG. 15 shows various components of an exemplary computing system 1500 that can be implemented as any type of client, server, and / or computing device, as described with reference to FIGS. 2 and 3 above, to implement aspects of respiratory rate sensing.

[0083] The computing system 1500 includes a communication device 1502 that enables wired and / or wireless communication of device data 1504 (e.g., received data, data being received, data scheduled for broadcast, or data packets of data). The communication device 1502 or computing system 1500 may include one or more hearables 102. The device data 1504 or other device content may include configuration settings of the device, media content stored on the device, and / or information related to a user of the device. The media content stored on the computing system 1500 may include any type of audio, video, and / or image data. The computing system 1500 includes one or more data inputs 1506 via which it may receive any type of data, media content, and / or input, such as, for example, human speech, user selectable input (explicit or implicit), messages, music, television media content, recorded video content, and any other type of audio, video, and / or image data received from any content and / or data source.

[0084] Computing system 1500 also includes a communications interface 1508, which may be implemented as any one or more of a serial and / or parallel interface, a wireless interface, any type of network interface, a modem, and any other type of communications interface. Communications interface 1508 provides a connection and / or communications link between computing system 1500 and a communications network through which other electronic, computing, and communications devices communicate data with computing system 1500.

[0085] Computing system 1500 includes one or more processors 1510 (e.g., any of a microprocessor, a controller, etc.) that process various computer-executable instructions to control the operation of computing system 1500. Alternatively or additionally, computing system 1500 can be implemented using any one or a combination of hardware, firmware, or fixed logic circuitry implemented in relation to processing and control circuitry generally identified as 1512. Although not shown, computing system 1500 can include a system bus or data transfer system that couples various components within the device. The system bus can include any one or a combination of various bus structures, examples of which include a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor bus or local bus that utilizes any of various bus architectures.

[0086] Computing system 1500 also includes computer-readable media 1514, such as one or more memory devices that enable persistent and / or non-transitory data storage (i.e., as opposed to merely transmitting a signal), examples of which include random access memory (RAM), non-volatile memory (e.g., any one or more of read-only memory (ROM), flash memory, EPROM, EEPROM, etc.), and disk storage devices. The disk storage device can be implemented as any type of magnetic or optical storage device, such as, for example, a hard disk drive, a recordable and / or rewritable compact disk (CD), any type of digital versatile disk (DVD), etc. Computing system 1500 can also include a mass storage media device (storage media) 1516.

[0087] The computer-readable medium 1514 provides a data storage mechanism for storing device data 1504, as well as various device applications 1518 and any other kind of information and / or data related to the operational aspects of the computing system 1500. For example, the operating system 1520 is maintained as a computer application using the computer-readable medium 1514 and can be executed by the processor 1510. The device applications 1518 can include any form of control application, software application, signal processing and control module, code specific to a particular device, a device manager such as a hardware abstraction layer for a particular device, and the like.

[0088] The device applications 1518 also include any system component, engine, or manager for performing audiopletismography 110. In this example, the device applications 1518 include the audiopletismography-based application 206 (APG-based application 206) of FIG. 2, the audiopletismography measurement module 318 of FIG. 3, and optionally, the audiopletismography calibration module 320 of FIG. 3.

[0089] Several examples will be described below. Example 1: A method comprising: transmitting an acoustic transmission signal that propagates inside at least a part of the user's external auditory canal by at least one speaker; receiving an acoustic reception signal by at least one microphone, wherein the acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified for the propagation inside the external auditory canal, said receiving, and determining the user's respiratory rate by analyzing the one or more waveform characteristics of the acoustic reception signal.

[0090] Example 2: The method according to Example 1, further comprising communicating the respiratory rate to a smart device so that the smart device can display the respiratory rate to the user.

[0091] Example 3: The method according to Example 1 or 2, wherein determining the respiratory rate is based only on the acoustic transmission signal and the acoustic reception signal.

[0092] Example 4: The method according to any one of the preceding examples, further comprising performing active noise cancellation using the at least one microphone.

[0093] Example 5: Transmitting the acoustic transmission signal includes transmitting the acoustic transmission signal having a plurality of frequencies, Determining the respiratory rate includes determining the respiratory rate based on an autocorrelation associated with one of the plurality of frequencies of the acoustic reception signal that has the highest peak-to-average ratio compared to the autocorrelation of other frequencies of the plurality of frequencies of the acoustic reception signal. The method according to any one of the preceding examples.

[0094] Example 6: The method according to any one of the preceding examples, wherein the modification to the one or more waveform characteristics is based on a change in the carbon dioxide concentration inside the user's external auditory canal.

[0095] Example 7: Determining the respiratory rate of the user includes demodulating the acoustic reception signal by mixing a digital version of the acoustic reception signal with a digital version of the acoustic transmission signal to generate a mixed signal, passing the mixed signal through a low-pass filter to generate a filtered signal, generating an autocorrelation of the filtered signal, and determining the period of the autocorrelation of the filtered signal to determine the respiratory rate. The method according to any one of the preceding examples.

[0096] Example 8: The method according to any one of the preceding examples, further comprising determining the heart rate of the user based on the acoustic reception signal.

[0097] Example 9: The acoustic transmission signal is one of the following an ultrasonic signal having a frequency between about 20 kilohertz and 2 megahertz, or an audible signal having a frequency between about 20 hertz and 20 kilohertz, the method according to any one of the preceding examples, comprising at least one of them.

[0098] Example 10: The method according to any one of the preceding examples, further comprising transmitting audible content to the ear during at least a part of the time when the acoustic transmission signal is transmitted.

[0099] Example 11: Further comprising performing a calibration process for identifying at least one acoustic frequency of audio plethysmography, wherein transmitting the acoustic transmission signal includes transmitting the acoustic transmission signal having the at least one acoustic frequency, the method according to any one of the preceding examples.

[0100] Example 12: A device, at least one speaker, at least one microphone, and at least one processor, comprising the device is configured to perform any one of the methods described in Examples 1 to 11 using the at least one speaker, the at least one microphone, and the at least one processor.

[0101] Example 13: The device according to Example 12, further comprising an active noise cancellation circuit including the at least one microphone.

[0102] Example 14: The device according to Example 13, wherein the at least one speaker and the at least one microphone are configured to be disposed proximate to one ear of the user.

[0103] Example 15: The device according to Example 12, wherein the at least one speaker is configured to be disposed proximate to the first ear of the user, and the at least one microphone is configured to be disposed proximate to the second ear.

[0104] Example 16: The device according to any one of Examples 12 - 15, wherein the at least one speaker and / or the at least one microphone is part of at least one transducer of the device.

[0105] Example 17: The device according to any one of Examples 12 - 15, wherein the device is configured to at least partially seal one or more ears of the user.

[0106] Example 18: The device comprises at least one earphone or headphones, and is the device according to any one of Examples 12 - 17.

[0107] Conclusion Techniques for use in facilitating the sensing of respiratory rate and devices including those that ease the sensing of respiratory rate have been described in terms specific to the features and / or methods. It will be understood, however, that the subject matter of the appended claims is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as exemplary embodiments of facilitating the sensing of respiratory rate.

Claims

1. A method comprising: transmitting, by at least one speaker, an acoustic transmission signal that propagates inside at least a part of a user's external auditory canal; receiving, by at least one microphone, an acoustic reception signal, wherein the acoustic reception signal represents a version of the acoustic transmission signal having one or more waveform characteristics modified for the propagation inside the external auditory canal, and said receiving; and determining a respiration rate of the user by analyzing the one or more waveform characteristics of the acoustic reception signal.

2. The method according to claim 1, further comprising communicating the respiration rate to a smart device so that the smart device can display the respiration rate to the user.

3. The method according to claim 1 or 2, wherein said determining the respiration rate is based only on the acoustic transmission signal and the acoustic reception signal.

4. The method according to any one of the preceding claims, further comprising performing active noise cancellation using the at least one microphone.

5. Said transmitting the acoustic transmission signal includes transmitting the acoustic transmission signal having a plurality of frequencies, and said determining the respiration rate includes determining the respiration rate based on an autocorrelation associated with one frequency having the highest peak-to-average ratio compared to the autocorrelations of other frequencies among the plurality of frequencies of the acoustic reception signal. The method according to any one of the preceding claims.

6. The method according to any one of the preceding claims, wherein the modification to the one or more waveform characteristics is based on a change in the carbon dioxide concentration inside the user's external auditory canal.

7. Said determining the respiration rate of the user includes demodulating the acoustic reception signal by mixing a digital version of the acoustic reception signal with a digital version of the acoustic transmission signal to generate a mixed signal; passing the mixed signal through a low-pass filter to generate a filtered signal; generating an autocorrelation of the filtered signal; and determining a period of the autocorrelation of the filtered signal to determine the respiration rate. The method according to any one of the preceding claims.

8. The method according to any one of the preceding claims, further comprising determining a heart rate of the user based on the acoustic reception signal.

9. The acoustic transmission signal is the following, an ultrasonic signal having a frequency between about 20 kilohertz and 2 megahertz, or an audible signal having a frequency between about 20 hertz and 20 kilohertz, the method according to any one of the preceding claims, comprising at least one of.

10. The method according to any one of the preceding claims, further comprising transmitting audible content to the ear during at least a part of the time when the acoustic transmission signal is transmitted.

11. Further comprising performing a calibration process for identifying at least one acoustic frequency for audio plethysmography, transmitting the acoustic transmission signal includes transmitting the acoustic transmission signal having the at least one acoustic frequency, the method according to any one of the preceding claims.

12. A device, at least one speaker, at least one microphone, and at least one processor, and the device is configured to perform any one of the methods according to claims 1 to 11 using the at least one speaker, the at least one microphone, and the at least one processor.

13. The device according to claim 12, further comprising an active noise cancellation circuit including the at least one microphone.

14. The device according to claim 13, wherein the at least one speaker and the at least one microphone are configured to be disposed close to one ear of the user.

15. The at least one speaker is configured to be disposed close to a first ear of the user, the at least one microphone is configured to be disposed close to a second ear, the device according to claim 12.

16. The device according to any one of claims 12 to 15, wherein the at least one speaker and / or the at least one microphone is part of at least one transducer of the device.

17. The device according to any one of claims 12 to 15, wherein the device is configured to at least partially seal one or more ears of the user.

18. The device is, at least one earphone, or a headphone, and is the device according to any one of claims 12 to 17.

Images