Passive respiratory rate determination

Through the head-mounted device combining motion and audio sensors, using sliding windows and multiple breathing rate algorithms to process data, the accuracy and convenience of passive monitoring of breathing rate in the prior art is solved, and efficient and low-interference breathing rate is achieved monitor.

CN118973478APending Publication Date: 2024-11-15SAMSUNG ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380031318.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-18
Filing Date
2023-05-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing technology is difficult to passively monitor the user's breathing rate accurately and conveniently. Active methods require users to actively participate and actively determine the accuracy of the possible interference of the breathing rate measurement results. Smart watches and PPG sensors are inaccurate when other movements occur. The device is difficult to distinguish breath-related signals under background noise.

Method used

Through a head-mounted device, the user's motion data is accessed, the activity level is determined, the activity-based technology is selected to estimate the respiration rate, the data is processed using sliding windows and filtering technology, and the interpolation is combined with multiple respiration rate algorithms to activate the microphone The sensor is used to identify respiratory noise and clustering, establish respiratory rate mass and activate the second sensor.

Benefits of technology

It realizes accurate and effective monitoring of user's breathing rate without the active participation of users, reducing interference and cost, improving data accuracy and reliability, and reducing power consumption.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118973478A_ABST
    Figure CN118973478A_ABST
Patent Text Reader

Abstract

In an embodiment, a method includes accessing motion data representing motion of a user from a first sensor of a wearable device, and determining an activity level of the user from the motion data. The method includes selecting an activity-based technique for estimating a respiratory rate of the user based on the activity level, where the activity-based technique is used to determine the respiratory rate of the user. The method further includes determining a quality associated with the respiratory rate and comparing the determined quality to a threshold. If the determined quality is not less than the threshold, the method includes using the determined respiration rate as a final respiration rate determination for the user. If the determined quality is less than the threshold, the method includes activating a second sensor of the wearable device and determining a breathing rate of the user based on data from the second sensor.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application generally relates to passively determining a user's breathing rate. Background Art

[0002] Breathing is the basis of human health and wellness. Respiratory rate is the number of breaths a person takes in a period of time (e.g., one minute). It is a recognized vital sign related to human health and is usually closely related to a person's respiratory health, stress, and fitness level. In addition, abnormalities in respiratory rate can usually indicate a medical condition. For example, an abnormal respiratory rate (which can be a respiratory rate higher than 27 breaths per minute (27bpm) or a respiratory rate less than 10bpm) is associated with pneumonia, cardiac arrest, drug overdose, and respiratory distress. Summary of the invention

[0003] Technical Solution

[0004] A system includes: one or more non-transitory computer-readable storage media storing instructions; and one or more processors coupled to the non-transitory computer-readable storage media, wherein the one or more processors are capable of operating to execute the instructions to perform the following operations: access motion data obtained by a first sensor of a wearable device from the movement of a user wearing the wearable device; determine the activity level of the user from the motion data; based on the determined activity level, select an activity-based technique for estimating the user's breathing rate; determine the user's breathing rate using the selected activity-based technique and the accessed motion data; determine a quality associated with the determined breathing rate; compare the determined quality associated with the determined breathing rate to a threshold quality; and in response to determining that the determined quality is not less than the threshold quality, use the determined breathing rate as a final breathing rate determination of the user; and in response to determining that the determined quality is less than the threshold quality, perform the following operations: activate a second sensor of the wearable device for a period of time; and determine the user's breathing rate based on data from the second sensor.

[0005] The wearable device includes a head-mounted device.

[0006] The activity level is an activity level selected from a set of activity levels including a resting activity level and a moving activity level.

[0007] The one or more processors are operable to execute the instructions to determine the user's breathing rate using a resting activity-based technique associated with the resting activity level by: applying a sliding window to the accessed data; selecting motion data corresponding to a particular axis from the accessed motion data; filtering the selected motion data corresponding to the particular axis; and estimating the user's breathing rate based on the filtered motion data.

[0008] The one or more processors are operable to execute the instructions to: generate a plurality of breathing rate estimates based on a plurality of different breathing rate algorithms; and estimate a breathing rate of the user by interpolating the plurality of breathing rate estimates.

[0009] The one or more processors are operable to execute the instructions to determine a breathing rate of the user using a movement activity based technique associated with the movement activity level by segmenting the accessed movement data into movement segments, wherein each movement segment corresponds to a breathing cycle of the user.

[0010] The second sensor includes a microphone.

[0011] The one or more processors are operable to execute the instructions to determine a quality of a breathing rate determined based on data from the microphone, wherein the quality is based at least on (1) a noise-to-respiration ratio associated with the data from the microphone, and (2) a number of breathing clusters identified in the data from the microphone.

[0012] The one or more processors are operable to execute the instructions to: classify data from the microphone using a set of categories including a breathing category and a transition category; cluster the classified data into a plurality of clusters; and determine the user's breathing rate based on the number of clusters having the transition category label.

[0013] The one or more processors are operable to execute the instructions to perform the following operations: determine whether there are any activity-based techniques for estimating the user's breathing rate that correspond to the determined activity level; and when no activity-based techniques for estimating the user's breathing rate correspond to the determined activity level, activate a second sensor of the wearable device.

[0014] The one or more processors are operable to execute the instructions to perform the following operations: in response to determining that the determined quality is less than the threshold quality, determine whether to activate the second sensor based on one or more of the following items: the amount of time that has passed since the second sensor was last activated; or the amount of time that has passed since the user's breathing rate was last effectively determined.

[0015] The value of the threshold quality depends on one or more of: a user preference; a user-specific breathing quality score associated with the first sensor; or a user-specific breathing quality score associated with the second sensor.

[0016] The second sensor is one of a plurality of second sensors, wherein each second sensor of the plurality of second sensors is ranked based at least on power consumption and detection accuracy of the sensor.

[0017] The one or more processors are operable to execute the instructions to periodically repeat the above process.

[0018] One or more non-transitory computer-readable storage media store instructions and are coupled to one or more processors operable to execute the instructions to: access motion data obtained by a first sensor of a wearable device from motion of a user wearing the wearable device; determine an activity level of the user from the motion data; based on the determined activity level, select an activity-based technique for estimating the user's breathing rate; determine the user's breathing rate using the selected activity-based technique and the accessed motion data; determine a quality associated with the determined breathing rate; compare the determined quality associated with the determined breathing rate to a threshold quality; and in response to determining that the determined quality is not less than the threshold quality, use the determined breathing rate as a final breathing rate determination for the user; and in response to determining that the determined quality is less than the threshold quality, perform the following operations: activate a second sensor of the wearable device for a period of time; and determine the user's breathing rate based on data from the second sensor.

[0019] The wearable device includes a head-mounted device.

[0020] The activity level is selected from a set of activity levels including a resting activity level and a moving activity level.

[0021] The medium is also coupled to one or more processors operable to execute the instructions to determine the user's breathing rate using a resting activity-based technique associated with the resting activity level by: applying a sliding window to the accessed data; selecting motion data corresponding to a particular axis from the accessed motion data; filtering the selected motion data corresponding to the particular axis; and estimating the user's breathing rate based on the filtered motion data.

[0022] The medium is also coupled to one or more processors operable to execute the instructions to: generate a plurality of breathing rate estimates based on a plurality of different breathing rate algorithms; and estimate the user's breathing rate by interpolating the plurality of breathing rate estimates.

[0023] The medium is also coupled to one or more processors operable to execute the instructions to determine the user's breathing rate using a movement activity-based technique associated with the movement activity level by segmenting the accessed motion data into motion segments, wherein each motion segment corresponds to a breathing cycle of the user.

[0024] The second sensor includes a microphone.

[0025] The method is also coupled to one or more processors operable to execute the instructions to determine a quality of a breathing rate determined based on data from the microphone, wherein the quality is based at least on (1) a noise-to-respiration ratio associated with the data from the microphone, and (2) a number of breathing clusters identified in the data from the microphone.

[0026] The method is also coupled to one or more processors operable to execute the instructions to: classify data from the microphone using a set of categories including a breathing category and a transition category; cluster the classified data into a plurality of clusters; and determine the user's breathing rate based on the number of clusters having the transition category label.

[0027] The medium is also coupled to one or more processors operable to execute the instructions to: determine whether any activity-based techniques for estimating the user's breathing rate correspond to the determined activity level; and when no activity-based techniques for estimating the user's breathing rate correspond to the determined activity level, activate a second sensor of the wearable device.

[0028] The medium is also coupled to one or more processors operable to execute the instructions to perform the following operations: in response to determining that the determined quality is less than the threshold quality, determine whether to activate the second sensor based on one or more of the following items: the amount of time that has passed since the second sensor was last activated; or the amount of time that has passed since the user's breathing rate was last effectively determined.

[0029] The value of the threshold quality depends on one or more of: a user preference; a user-specific breathing quality score associated with the first sensor; or a user-specific breathing quality score associated with the second sensor.

[0030] The second sensor is one of a plurality of second sensors, wherein each second sensor of the plurality of second sensors is ranked based at least on power consumption and detection accuracy of the sensor.

[0031] The medium is also coupled to one or more processors operable to execute the instructions to periodically repeat the above process.

[0032] A method includes: accessing motion data obtained by a first sensor of a wearable device from motion of a user wearing the wearable device; determining an activity level of the user from the motion data; based on the determined activity level, selecting an activity-based technique for estimating the user's breathing rate; determining the user's breathing rate using the selected activity-based technique and the accessed motion data; determining a quality associated with the determined breathing rate; comparing the determined quality associated with the determined breathing rate to a threshold quality; and in response to determining that the determined quality is not less than the threshold quality, using the determined breathing rate as a final breathing rate determination of the user; or in response to determining that the determined quality is less than the threshold quality, performing the following steps: activating a second sensor of the wearable device for a period of time; and determining the user's breathing rate based on data from the second sensor.

[0033] The wearable device includes a head-mounted device.

[0034] The activity level is selected from a set of activity levels including a resting activity level and a moving activity level. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Example recorded motion signals and audio signals from a head mounted device are shown.

[0036] Figure 2 An example method for monitoring a user's breathing rate is shown.

[0037] Figure 3 Show Figure 2 An example implementation of the method.

[0038] Figure 4 An example approach for determining a user's activity level is shown.

[0039] Figure 5 Show Figure 2 An example of step 220 and step 225 of the method.

[0040] Fig. 6A and Figure 6B Shown from Figure 5 Example motion data of the processing.

[0041] Fig. 7A Show Figure 2 An example of steps 220 and 225 of the method.

[0042] Figure 7B Show Fig. 7A Example results of the process.

[0043] Figure 8 Show Figure 2 An example of step 245 of the method.

[0044] Fig. 9 Example audio data is shown.

[0045] Fig.10 An example set of classifications output by the prediction step is shown.

[0046] Fig.11 An example computing system is shown.

[0047] Best Mode for Carrying Out the Invention DETAILED DESCRIPTION

[0049] Many current methods for determining the breathing rate of a user are active methods, because they require some purposeful activities of the user or another person (e.g., a doctor) in order to implement the method. For example, some methods require the user to place a device with a microphone at a specific position on the user's chest or abdomen to capture breathing movements. However, such methods require the active participation of the user, which is inconvenient, and in addition, when the user is aware of their breathing and / or the user is aware that their breathing rate is being determined, the user's breathing rate measurements (measurements) may be affected, generating data that cannot truly represent the user's breathing rate. Therefore, passive breathing determination that determines the user's breathing rate without the conscious awareness or effort of the user (or any other person) may be more accurate than active determination. In addition, for example, because the user must stop what they are doing to perform the test or visit a clinical facility in order to perform a specialized test, active determination may be disruptive, cost-intensive and labor-intensive.

[0050] Respiration belts can perform passive respiration rate determination, but because such belts are uncommon, off-the-shelf devices with a dedicated purpose, they are often uncomfortable and expensive. Some smart watches attempt to capture respiration rate, for example based on motion sensors or PPG sensors. However, because motion data is discarded when there are other movements (e.g., arm movements), motion data from smart watches may be inaccurate for determining respiration rate. In practice, a very low percentage (e.g., 3% to 17%) of motion data obtained by a smart watch may actually be retained for respiration rate determination. In addition, PPG sensors may be inaccurate in the presence of excessive motion, and individual user differences (e.g., skin color, wearing habits) may also affect PPG data.

[0051] Head-mounted devices (such as earbuds / headphones, glasses, etc.) can incorporate multiple types of sensors that can detect data related to breathing rate. For example, a pair of headphones may have motion sensors (e.g., accelerometers and / or gyroscopes) and audio sensors (e.g., microphones) to capture head movements related to breathing (e.g., certain vertical head movements that occur when breathing) and breathing sounds generated by the nose and mouth, respectively. However, subtle head movements that indicate breathing can easily disappear in the sensor data when other movements are present. For example, Figure 1 Graph 110 in shows an example motion signal recorded by a sensor (e.g., an inertial measurement unit or IMU) integrated into a head mounted device, where the sensor captures periodic motion associated with breathing. In contrast, graph 115 shows a motion signal from the same sensor but in the presence of other motions. As shown in graph 115, it is difficult to discern the periodic motion signal associated with breathing due to the presence of other motions that dominate the motion signal. Likewise, breathing related noise can be drowned out in the sensor data by background noise. For example, Figure 1 The graph 120 in FIG. 1 shows an example recorded audio signal from a sensor integrated in a head mounted device, wherein the sensor captures a periodic audio signal associated with breathing. In contrast, Figure 1 Graph 125 in FIG. 1 shows an audio signal from the same sensor but with background noise dominating the captured data in graph 125 , making it difficult to discern the breathing related audio data.

[0052] Figure 2 An example method is shown for accurately, efficiently, and passively monitoring a user's breathing rate using a default motion sensor and intelligent activation of another sensor, such as an audio sensor. Figure 3 FIG. 1 shows a head mounted device (such as a pair of headphones) including a six-axis IMU and a microphone. Figure 2 An example implementation of the method.

[0053] Figure 2 Step 205 of the example method includes accessing motion data obtained by a first sensor of the wearable device from the motion of a user wearing the wearable device. For example, the first sensor may be an IMU, and the wearable device may be a pair of headphones. In certain embodiments, the step 206 may be performed by the wearable device or by another device (e.g., by a smart phone or server device connected to the wearable device directly or via an intermediate device). Figure 2 Some or all of the steps (including step 205) of the example method.

[0054] Figure 2Step 210 of the example method includes determining the user's activity level from the motion data. The determined activity level can be selected from a set of predetermined activity levels (such as resting, low activity, moderate activity, high activity, etc.). Figure 4 An example method for determining a user's activity level in the context of a predetermined "resting" activity level and a predetermined "heavy" activity level is shown. In certain embodiments, "heavy" activity may include, but is not limited to, physical activity such as high-intensity interval training (HIIT) (such as a 3-Minute Step Test, etc.) or walking (such as a 6-Minute Walk Test, etc.). In certain embodiments, step 210 may be performed by a machine learning architecture (such as a lightweight machine learning-based approach) that identifies the user's head movement from motion sensor data. Figure 4 In the example of , this could be a three-category classification that predicts whether an activity is classified as "resting", "heavy", or "other". The "other" category could represent, among other events, random movements made by the user, and because breathing-related movements are not periodic during many random movements, it is difficult to reliably determine the breathing rate from the corresponding motion data. For example, during talking, eating, or drinking, breathing occurs randomly (i.e., not in the typical periodic pattern of a person's normal breathing), and thus the motion signals during these activities are unlikely to generate recognizable breathing data.

[0055] exist Figure 4 In the example of , a sliding window is applied to the motion data accessed in step 205. In certain embodiments, steps 205 and 210 may occur substantially continuously and in real time. Thus, the sliding window may be continuously updated and evaluated as new motion data segments are obtained from the sensor. Figure 4 In the example of , features are extracted from the raw motion data within the window. In a specific embodiment, several (e.g., 10) statistical features may be extracted, for example, in order to keep the method lightweight. In a specific embodiment, a correlation-based feature selection (CFS) algorithm for more than 200 commonly used time-domain features for activity recognition may be used to identify features for motion activity classification. The present disclosure contemplates that any suitable features may be used to classify activity levels based on motion data (such as acceleration, signal amplitude area, average amplitude, etc.). The present disclosure contemplates that any suitable one or more classifiers may be used to classify activity levels based on features extracted from a window of motion data. For example, a random forest and / or a logistic regression classifier may be used, wherein, in a specific embodiment, the logistic regression classifier is particularly suitable for lightweight classification.

[0056] Figure 2Step 215 of the example method includes selecting an activity-based technique for estimating the user's breathing rate based on the determined activity level. Figure 4 As shown in the example of , if the user's activity level is determined to be "resting," a corresponding resting-based technique (explained more fully below) is used to determine the user's breathing rate. Similarly, if the user's activity level is classified as "heavy," a corresponding heavy activity-based technique (also explained more fully below) is used to determine the user's breathing rate. As explained more fully herein, in particular embodiments, if the determined activity is not associated with any activity-based technique for estimating the user's breathing rate (e.g., the user's activity is classified as "other"), a second sensor (e.g., a microphone) may be activated. In particular embodiments, as discussed more fully herein, activation of the second sensor may depend on the output of the activation pipeline.

[0057] Figure 2 Step 220 of the example method includes determining a breathing rate of a user using the selected activity-based technique and the accessed motion data. Step 225 includes determining a quality associated with the determined breathing rate, and step 230 includes comparing the quality of the determination associated with the determined breathing rate to a threshold quality. As explained more fully herein, the quality is used to determine whether the breathing rate determination is sufficiently reliable for use as an estimate of the user's actual breathing rate.

[0058] Figure 5 An example of steps 220 and 225 is shown in the context of a particular technique associated with a user's "resting" activity level. Figure 5 The example of is relatively lightweight, and the input data can be, for example, 3-axis accelerometer data, 3-axis gyroscope data, or full 6-axis data from an IMU. A sliding window is applied to the input IMU data. For example, a 30 second sliding window can be used, where the 30 second sliding window balances accuracy and performance requirements in some embodiments. The data within the sliding window can be divided into multiple steps. For example, a 30 second window can be divided into 2 15 second steps.

[0059] Figure 5An example includes selecting an axis from the 6-axis IMU data. The motion data relative to the selected axis will then be used to determine the user's breathing rate (e.g., data relative to the other 5 axes does not need to be considered in subsequent steps in the process). In certain embodiments, because respiration at rest tends to provide a periodic signal, the axis selection process selects the axis containing the most periodic motion signal. As an example of periodic axis selection, a certain embodiment may perform the following steps for each axis: 1) apply a median filter and then normalize the signal, 2) apply a Butterworth Bandpass filter with, for example, a 0.13 Hz low cut frequency and a 0.66 Hz high cut frequency, and 3) perform a Fast Fourier Transform (FFT) on the filtered signal. In this example, the axis with the largest frequency value produces the most periodic axis with the largest signal variation caused by respiration. Subsequent steps in the process then use the data corresponding to the selected axis for preliminary breathing rate estimation. Although Figure 5 The examples of select a single axis from the IMU data, but the present disclosure contemplates that any suitable number of axes may be selected.

[0060] Figure 5 Examples include filtering the data about the selected axis. For example, certain embodiments may perform normalization on the filtered signal and apply a 3rd order Butterworth bandpass filter with a low cut frequency of 0.13 Hz and a high cut frequency of 0.66 Hz. These cutoff frequencies correspond to respiration rates between 8 BPM and 40 BPM, but the present disclosure contemplates that other cutoff values ​​may be used. Certain embodiments then apply a second order Savitzky-Golay filter to further smooth the signal.

[0061] Figure 5 Examples include estimating the respiration rate based on the filtered data signal. In certain embodiments, the filtered signal may resemble a breathing cycle (typically inhale-pause-exhale), for example due to the step size used. Certain embodiments may use one or more respiration rate algorithms to arrive at a final respiration rate determination. For example, certain embodiments may use an algorithm based on zero crossing (Zero-Crossing) (BR ZCR ), where the algorithm identifies zero crossings in the filtered signal and calculates the respiration rate based on the time between the two zero crossings. Certain embodiments may use the median time of the zero crossing duration to estimate the respiration rate, and then the respiration rate may be calculated as

[0062]

[0063] Certain embodiments may be based on an FFT-based algorithm (BR FFT) to determine the respiratory rate, wherein the algorithm applies a fast Fourier transform algorithm to the filtered signal to calculate a frequency domain representation of the respiratory signal. Figure 5 In the example of , the breathing rate can be calculated by selecting the frequency with the highest amplitude (peak) within the frequency range of 0.13 Hz and 0.66 Hz (7.8 BPM to 40 BPM). In this way, the rate is calculated as BR FFT =60×argmaxFFT(x).

[0064] Certain embodiments may be based on a peak-based algorithm (BR peak ) is used to determine the respiratory rate, wherein the algorithm uses a peak detection algorithm to find peaks and valleys in the filtered signal. Each valley to peak indicates an inhalation cycle, and the peak to valley indicates an expiratory cycle. In an ideal breathing cycle, the peaks and valleys must appear in an alternating order. However, due to noise in the derived breathing cycle, false peaks or valleys may be detected. Therefore, when there are multiple peaks between two valleys or multiple valleys between two peaks, specific embodiments remove false peaks and valleys. When there are multiple peaks between valleys, or vice versa, specific embodiments select the peak closest to the nearest valley or the valley closest to the nearest peak, respectively. Then, specific embodiments can estimate the respiratory rate by taking the median peak-to-peak distance. In this manner, the rate is calculated as

[0065]

[0066] Certain embodiments may use a plurality of breathing rate algorithms to determine the final breathing rate. For example, certain embodiments may be based on BR ZCR , BR FFT and BR peak The breathing rate may be determined by taking the median of the three estimates of the user's breathing rate. Other approaches may also be used, such as by taking a weighted sum of the individual determinations, etc.

[0067] In certain embodiments, the quality determination for the breathing rate estimate may be based on statistical properties of a collection of one or more breathing rate determinations. For example, in an example where three breathing rate determinations are used to arrive at a final breathing rate, Figure 2 Step 225 of the exemplary method may include determining the quality Q by taking the three determined standard deviations m ,Right now:

[0068] Q m =σ(BR zcr ,BR fft ,BR peak )

[0069] Fig. 6A and Figure 6BShown by Figure 5 Example data obtained by the example implementation of the example. Fig. 6A As shown in , the original signal is obtained. Fig. 6A Graph 610 shows the result of the axis selection, for example, the signal about the x-axis has been selected for further processing. Graph 620 shows the resulting signal after applying a median filter and normalization. Graph 630 shows the resulting signal after a bandpass filter, and graph 640 shows the resulting signal after applying a Savitzy-Golay filter. Figure 6B Graph 650 in shows the zero crossing estimation. Graph 660 shows the peak-to-peak estimation method, and graph 670 shows the FFT respiration rate estimation method after converting the signal (ie, the signal shown in graph 640) to the frequency domain.

[0070] Fig. 7A An example of steps 220 and 225 is shown in the context of a particular technique associated with an activity level of a user corresponding to an "active" classification. In a particular embodiment, the input data may be a 3-axis IMU signal, such as sensor data from an accelerometer or from a gyroscope, which may reduce the computational resources required to estimate the respiration rate relative to using a 6-axis IMU signal. In a particular embodiment, Fig. 7A The data smoothing step may include transforming x 2 +y 2 +z 2 The square root of is calculated as the breathing estimation signal, where x, y, and z are the three axis signal values ​​from the corresponding IMU sensor. Certain embodiments may then use a sliding window of, for example, 4 seconds to smooth the 3-axis IMU data.

[0071] In certain embodiments, the wearable device may be a pair of headphones, and the user's breathing signals collected from the ear canal may be correlated to small variations in the user's breathing intervals. Therefore, in order to extract the breathing rate, certain embodiments extract these intervals from the relatively large, noisy audible IMU signal. Certain embodiments may perform this extraction using the following formula:

[0072]

[0073] Here, f i refers to the value of the amplitude time series at a distance of i sampling points, and h is the average time interval between consecutive sampling points. Then, a differential filter with four steps can be used to obtain a signal proportional to the acceleration, which is effective for extracting relatively small motion signals caused by breathing. In certain embodiments, noise can be removed from the data in a denoising step (e.g., by applying one or both of outlier detection and Gaussian filtering).

[0074] like Fig. 7A As shown, certain embodiments may segment the denoised data into segments corresponding to the respiratory cycle. For example, certain embodiments may first initialize the template vector to 0, and then search each point in the time series to find the most similar segment during the possible range. Certain embodiments may set the batch size, for example, to balance between computational cost and accuracy. The similarity between segments may be calculated by dynamic time warping (DTW) distance. For example, the example method may take a sequence x as input and output a segment by repeating the following operation:

[0075] S l+1 = DistDTW(x,μ l )

[0076] μ 1+1 -DistDTw(x,S l+1 )

[0077] Where S is the set of segments and μ is the template segment. Figure 7B Show Fig. 7A The example results of the processing of Figure 7B 7. An example of a segment 710 defined by vertical bars superimposed on an IMU signal in FIG. 7, wherein the IMU signal includes both activity motion and respiratory motion.

[0078] In a specific embodiment, assuming that 60 seconds of data are being analyzed, the Fig. 7A The respiratory rate prediction makes BR activity = length(segs) (lengh(segs)). Since the user will breathe at a slightly higher rate, we can expect more breathing activity segments, so in a specific embodiment, if length(segs) is less than 1, then Q a is set to 0, otherwise Q a is set to 1. In this example, the corresponding quality threshold may have a value of 1.

[0079] exist Figure 2 In an example method of, in response to determining that the determined quality is not less than a threshold quality, step 235 includes using the determined breathing rate as a final breathing rate determination for the user. The breathing rate may be stored, for example, with a timestamp. In particular embodiments, the breathing rate may be displayed and updated to the user in real time or near real time, for example, on a wearable device or on a connected device (such as, the user's smart phone) The UI of a health-related application running on the wearable device and connected The breathing rate is displayed and updated to the user in real time or near real time. As explained below, the data may be sent to medical personnel and / or the data may be used to alert the user or medical personnel in the event of a health-related emergency.

[0080] exist Figure 2 In the example method of , in response to determining that the determined quality is less than the threshold quality, step 240 includes activating a second sensor of the wearable device for a period of time, and step 245 includes determining the user's breathing rate based on data from the second sensor. As explained herein, in certain embodiments, step 240 may occur when no motion-based technique for determining the user's breathing rate is detected (e.g., when the user's motion is classified as "other" in the example above).

[0081] Prior to activating the second sensor in step 240, certain embodiments may first use a second sensor activation process to determine whether to activate the second sensor, even when the quality of the motion-based respiration rate is less than a threshold. For example, the activation process may check whether the duration since the last valid respiration rate determination is greater than a threshold duration D. b If not, the process may not activate the second sensor (e.g., not perform step 240), and the process may loop back to step 205. If the time since the last valid respiration rate determination is greater than the threshold, step 240 may be performed. In certain embodiments, the activation process may check whether the duration since the last activation of the second sensor is greater than a threshold duration D. a If not, the process may not activate the second sensor (eg, not perform step 240), and the process may loop back to step 205. If the time since the second sensor was last activated is greater than the threshold, step 240 may be performed.

[0082] In certain embodiments, the activation process may check whether the duration since the last valid breathing rate determination is greater than a threshold duration D b and whether the duration since the second sensor was last activated is greater than a threshold duration D a Unless both of these checks are satisfied, step 240 may not be performed. In certain embodiments, these checks may conserve system resources by not activating the second sensor (e.g., microphone) when the sensor has just been recently activated or when valid respiration rate data (from any source) has recently been obtained.

[0083] In certain embodiments, D b The value of may vary based on, for example, the user's preferences and / or the user's currently detected motion. For example, as explained more fully below, the user may indicate how often they want the second sensor to be activated and / or may indicate a power setting configuration, and these preferences may adjust D accordingly. b As another example, if the user's motion indicates that the user is resting, then D bOn the other hand, if the user's motion indicates that the user is active, or the user is experiencing unclassified movement (e.g., "other"), then D b can be relatively low. Similarly, D a The value of D may vary based on, for example, the user's preferences and / or the user's currently detected motion. For example, if the user's motion indicates that the user is resting, or that the user is experiencing unclassified movement, then D a On the other hand, if the user's motion indicates that the user is active, then D a Can be relatively high.

[0084] After step 240 and before step 245, particular embodiments may activate the second sensor and analyze the data to initially determine whether the user's breathing rate can be detected from the data. For example, step 240 may include activating the microphone for, for example, 10 seconds, and particular embodiments may determine whether breathing can be detected from the audio data. If not, the audio data may be discarded and the process may loop back to step 205. If yes, the microphone (in this example, the particular second sensor) may continue to collect data for a period of time R. d (e.g., 10 to 60 seconds), and at step 245, this data is used to make a respiratory rate determination. In certain embodiments, R d The value of R may vary based on, for example, the user's preferences and / or the user's current motion. For example, if the user's motion indicates that the user is resting, then R d On the other hand, if the user's motion indicates that the user is active, or the user is experiencing unclassified movement (e.g., "other"), then R d Can be relatively long.

[0085] As an example of determining whether breathing can be detected from audio data, certain embodiments may use a lightweight algorithm to detect breathing events. For example, the audio data may first be normalized, for example using min-max normalization. Then, a sliding window having, for example, a 4 second window size and a 1 second step size may be applied to the normalized data. For each window of step size, features (e.g., a relatively low number of features, such as 30 features) may be extracted and passed to a trained machine learning classifier (e.g., a random forest classifier with 20 estimators (which may only use approximately 620KB of storage space)). The classifier detects breathing events in each window. Then, embodiments may use a clustering approach to identify respiratory events with a duration R. d The clustering of breathing audio in some or all of the complete breathing data segments of R (e.g., clusters corresponding to "inhale", "pause", or "exhale"). Since the typical breathing rate will be in R dThe period includes multiple breathing episodes, so the number of detected breathing episodes can be used (e.g., by comparing the number to a threshold number) to determine whether the audio data is robust enough to perform a breathing rate determination. If so, step 245 can be performed. If not, the process can return to step 205, or in certain embodiments, the process can return to step 240 to collect more data. Although the above examples are described in the context of audio data (i.e., the second sensor is a microphone), the present disclosure contemplates that similar processing can be used to analyze data from any other second sensor.

[0086] Figure 8 An example of step 245 is shown. As described below, Figure 8 The processing uses a machine learning classifier to detect transitions between breathing phases and then uses these classifications to calculate the breathing rate. A transition is a change in breathing phase from inspiration to exhalation or from exhalation to inspiration, such as Fig. 9 910 and transition 915 in the example of . Transition 910 starts the inhalation phase and transition 915 starts the exhalation phase. Certain embodiments only need to identify the beginning or transition of a breathing phase to determine the breathing rate. The transition phase always indicates a change in energy, and most of the time, the beginning of a breathing phase corresponding to a transition has a higher energy than the end of the previous breathing phase.

[0087] exist Figure 8 In the example of , preprocessing may include applying a window to the data and using min-max normalization to normalize the window. The data may then be analyzed using, for example, a sliding window with a 4 second window size and a 1 second step size. Because transitions may be instantaneous, particular embodiments may use another machine learning classifier on a smaller window (e.g., 800 milliseconds) with a 100 millisecond step size to identify transitions in the respiratory signal.

[0088] exist Figure 8 In the example of , differential feature generation may include dividing the 800ms window into two 400ms windows, such as a "left" window (such as, Fig. 9 902 in the example) and a "right" window (such as Fig. 9 904 in the example of ). Certain embodiments may then compute the differences between the signals in the two windows, and these differences are the differential features. To reduce computational overhead, certain embodiments may select only a relatively low number of features (e.g., 30) that have a high correlation with respiratory signs to use during the prediction step.

[0089] exist Figure 8In the example of , prediction may include using a lightweight classifier to detect transition points in an 800 ms window, for example because the transition prediction classifier can be run once every 100 ms (i.e., in this example, the classifier outputs a prediction each time the window steps by 100 ms). Particular embodiments may use a multilayer perceptron (MLP) architecture with a network size of 15×30×15, which may only take up approximately 45 KB of storage space. In particular embodiments, the classifier may use three categories: (1) breathing, (2) transitions, and (3) noise. In particular embodiments, the noise category is useful at least because there may be some transient noise that may otherwise be incorrectly labeled as a transition.

[0090] exist Figure 8 In an example of , post-processing may include clustering the labeled classifications. For example, because certain embodiments generate predictions every 100 ms, in such embodiments, there will be small clusters of different categories. Certain embodiments use clustering to identify the beginning and end of a cluster. Fig.10 Shown by Figure 8 An example set of classifications output by the prediction step of , and an example of clustering such classifications is shown. In this example, the class label "0" indicates a transition class, the label "1" indicates a "breathing" class, and the label "2" indicates a noise class. Particular embodiments may adjust the class labels based on, for example, the presence of one or more outlier labels within a block of outputs that are similarly labeled in other ways. For example, if 4 of 6 consecutive labels belong to a specific class, and two outliers are interspersed within 4 similar class labels, the two non-conforming class labels may be changed to a specific class label, i.e., so that all 6 labels are the same. The threshold used for reclassification may depend on, for example, the type of classification label being considered.

[0091] In certain embodiments, Figure 8 The respiratory rate estimation in the example of can be based on the time between transition clusters. For example, the time d between adjacent transition clusters can be calculated. t , and the average or median duration between transitions can be used to determine the user's breathing rate. For example, the breathing rate can be calculated from the median transition duration as

[0092]

[0093] Particular embodiments may determine a quality associated with a breathing rate determined by a second sensor. For example, in the example above, at a normal breathing rate of greater than 5 breaths per minute, there should be many (e.g., 10) breathing cycles in the last 60 seconds of audio data. Since the number of clusters provides an estimate of the number of cycles, the number of clusters (Nc, here a combination of the number of transition clusters and the number of breathing clusters) can be used as a quality parameter. Additionally or alternatively, the size Sc of the clusters may be used during post-processing. For example, when a 100ms sliding window is used, multiple transitions should be detected in a cluster, and if Sc is small for a particular cluster, this may indicate that the classifier may have detected an erroneous transition. Therefore, particular embodiments discard all small transition clusters, for example, a small transition cluster is a cluster with less than 3 elements in the cluster, which means that at least 3 transition class labels are required in the cluster (in this example) for valid breathing rate determination.

[0094] In certain embodiments, the noise to respiration ratio NBR may be used to determine the quality of the respiration rate determination. For example, NBR may be defined as:

[0095]

[0096] Since the noise signal can mask the breathing signal, a high NBR may mean that the breathing signal is too noisy for accurate breathing rate prediction. In a particular embodiment, the quality Qa for the breathing prediction may be defined as:

[0097]

[0098] If the quality is below a threshold value (which would be 1 in the example above), the respiration rate calculation by the second sensor may be discarded and the process may loop back to step 205 in some embodiments or to step 240 in other embodiments. If the quality is not less than the threshold value, the determined respiration rate may be used, e.g., as described below. The process then loops back to step 205.

[0099] Where appropriate, specific examples may be repeated Figure 2 Although the present disclosure describes and illustrates one or more steps of a method as occurring in a particular order, Figure 2 The specific steps of the method are described in detail below, but the present disclosure contemplates that they occur in any suitable order. Figure 2 In addition, although the present disclosure describes and illustrates the implementation of Figure 2 Specific components, devices or systems (such as, Fig.11 ), but the present disclosure contemplates executing Figure 2In addition, the present disclosure contemplates any suitable combination of any suitable components, devices, or systems of any suitable steps of the method. Fig.11 A computing device) comprising a circuit system, a processor coupled to a non-transitory computer-readable storage medium, or any suitable combination thereof to perform some or all of the computing operations described herein (including Figure 2 In certain embodiments, the wearable device (e.g., a device including Fig.11 An example of a computing device comprising a component of an electronic device (such as a microcontroller) executing Figure 2 In certain embodiments, some or all steps of the examples of FIG. may be performed by one or more other devices (such as a client device (eg, a smart phone) or a server device, or both).

[0100] Steps 205 to 235 may consume less power than steps 240 to 245. For example, a pair of headphones that continuously perform steps 205 to 235 on a pair of headphones may see a 4% to 6% reduction in battery life per hour, while when those steps are not performed, the battery may decrease by 3% to 5% per hour. In contrast, continuously performing steps 240 to 245 on headphones when the second sensor is a microphone may reduce battery life by nearly 20%, showing Figure 2 The efficiency of the example method that enables continuous, passive and accurate respiratory rate determination.

[0101] In certain embodiments, one or more aspects of the breathing rate determination may be based on user preferences and / or user personalization. For example, parameters associated with the audio activator and / or the audio recording scheduler may be based in part on the user's personalization. For example, if the user's audio tends to be less useful, for example, because the user's audio energy is often insufficient to capture the breathing rate, the system may run the audio activator less frequently and / or may record audio segments for longer periods of time. In certain embodiments, if the user's audio tends to be relatively more useful, for example, because the user's breathing rate tends to be easily detected from the audio signal, the system may run the audio activator more frequently. As another example, certain embodiments may activate a second sensor (such as a microphone) based on the user's preferences for, for example, accuracy, power consumption, etc.

[0102] In certain embodiments, the frequency of activation of the second sensor may be based on a combination of factors such as user preferences, the accuracy of the breathing determination made by the second sensor, and / or the accuracy of the breathing determination made using the motion sensor. For example, a user may select among various possible preferences regarding how frequently the sensor should be activated to detect the user's breathing rate. For example, a user may select a "frequent" activation setting, where the "frequent" activation setting may correspond to, for example, activating the microphone every minute. As another example, a user may select a "periodic" activation setting, where the "periodic" activation setting may correspond to, for example, activating the microphone every 5 to 10 minutes. As another example, a user may select an "occasional" activation setting, where the "occasional" activation setting may correspond to, for example, activating the microphone every 30 minutes.

[0103] In a particular embodiment, the frequency of activation of the second sensor may be based at least in part on the accuracy of the breathing determination performed by the motion sensor. In a particular embodiment, the accuracy may be personalized for a particular user, so that if the user's motion-based breathing rate is relatively inaccurate, the second sensor (e.g., microphone) may be activated relatively more frequently. As an example, a respiration correlation score based on IMU data may be determined. First, motion-based data may be obtained during initialization processing or by accessing a data block (chunk) of the respiration-related motion data obtained when the user is at rest. From these recorded signals, the system may determine the most periodic axis for respiratory cycle detection. Then, the system may perform noise filtering on the recorded signal, for example, by first performing median filtering and then performing bandpass filter and SavitzkyGolay filter to perform noise filtering on the recorded signal. Then, the system may use a peak detection algorithm to determine the peaks and valleys of the signal. Each valley-to-valley section or each peak-to-peak section may be marked as a respiratory cycle. Then, the system may, for example, randomly select some respiratory cycles to determine the quality of the respiratory cycle obtained from IMU data for the user. Then, a particular embodiment may calculate a dynamic time warping distance (DTW) for each breathing cycle with some (e.g., as collected from data from many users) pre-selected good breathing cycles. Then, the average DTW distance for each cycle is combined to calculate a breathing quality score for the user determined for the IMU-based breathing rate. If the DTW distance is relatively low, it means that the quality score is relatively high, and the second sensor (e.g., microphone) may be activated less frequently. If the DTW distance is relatively high, the correlation score is relatively low, and the second sensor may be activated relatively more frequently.

[0104] The activation frequency of the second sensor (such as, microphone) can be based at least in part on the personalized user-specific breathing rate determination quality performed by the corresponding second sensor. For example, the pipeline can be used to calculate the breathing correlation score for the user, such as calculating the breathing correlation score for the user during the initialization phase of collecting samples from the user's breathing within a period of time (for example, 1 minute), or calculating the breathing correlation score for the user by identifying that the breathing data is a set of discernible data segments (for example, audio segments). For example, a specific embodiment can access a sequence of short audio records collected by the user at rest, and try to detect the breathing phase from the record, such as, described in US Patent Application Publication No. US2022 / 0054039, the corresponding disclosure of which is incorporated herein. The system can identify the "inhale" phase and the "exhale" phase and merge continuous breathing phases to construct a breathing cycle. Based on the detected breathing cycles, the system can select some of the best N breathing cycles, for example, those breathing cycles with maximum breathing energy relative to the total energy of the signal. The number of optimal breathing cycles can be variable, and in a specific embodiment, three cycles are sufficient to determine the breathing pattern of the user. Once the system has selected the best breathing cycle, the system can determine the probability that the particular user will provide audio data with a good breathing cycle in the future. For each of the best N breathing cycles, the system can extract audio-based features and pass these features to a breathing cycle classifier. The classifier generates a probability of a good breathing cycle for each of the extracted cycles, which can then be averaged to calculate a breathing quality score. A relatively high breathing correlation (quality score) indicates that an audio-based algorithm may be relatively more useful for determining the user's breathing rate, and therefore an audio-based pipeline may be selected relatively more frequently for the user (e.g., by increasing a motion-based quality threshold).

[0105] In a particular embodiment, the second sensor may be one of a plurality of second sensors on the wearable device. For example, the wearable device may include a microphone, a photoplethysmography (PPG) sensor, a temperature sensor, and / or other sensors that can be used to estimate the user's breathing rate. For example, a PPG sensor can capture respiratory sinus arrhythmia (RSA) that can be used to estimate the breathing rate. In a particular embodiment, the quality associated with a breathing rate determined using a second sensor (e.g., a microphone) can be compared to a threshold, and if the quality is below the threshold, another second sensor (e.g., a PPG sensor) can be used to estimate the user's breathing rate. In a particular embodiment, when the motion-based breathing rate determination is insufficient, the second sensors may be sorted in order of use. For example, the second sensors may be sorted based on power consumption so that the second sensor with the lowest power consumption is sorted highest. In a particular embodiment, the second sensors may be sorted based on accuracy so that the most accurate second sensor is sorted highest. In a particular embodiment, the sorting may be based on a combination (e.g., a weighted combination) of multiple factors (e.g., power consumption, accuracy, and user relevance for the particular user). In a particular embodiment, whenever a sensor fails to adequately determine the breathing rate, the system selects the next sensor in the sorted order.

[0106] Particular embodiments may use a combination of sensors to determine the user's breathing rate. Similar to the discussion above about the second sensor of sorting, sensor groups may be sorted for determining the user's breathing rate. For example, data from a group of two or more sensors may be input into a machine learning model, and the data may be cascaded to calculate the breathing rate. For example, U.S. Patent Application Publication No. 2022 / 0054039 describes an embodiment and architecture for respiratory phase detection using a multimodal system, and these disclosures are incorporated herein by reference. A motion sensor and a microphone may be a sensor group, and a motion sensor and a PPG sensor may be another group, etc. In a particular embodiment, if a sensor group fails to fully estimate the user's breathing rate, the next sensor group may be activated. As described above, sorting may be performed in any suitable manner (such as by power consumption, accuracy, user relevance, or any suitable combination thereof). In a particular embodiment, sensor groups may be created dynamically, such as by treating the sensors sorted in the first N as one group, treating the sensors sorted in the last M as another group, etc.

[0107] Monitoring a user's breathing rate can be used for many purposes. For example, embodiments disclosed herein can be used for emergency event detection by tracking key breathing-related conditions, including health emergencies. Abnormalities in breathing rate are directly associated with health emergencies, and if an abnormality is detected, for example, when the user is at rest (e.g., the breathing rate is below an emergency threshold, where the emergency threshold can be based on the user's personalized data), certain embodiments can trigger a warning (e.g., by providing an audio or visual alert to the user via the wearable device and / or a connected device).

[0108] As another example, breathing rate increases in many medical conditions (such as asthma and COPD) before lung conditions worsen, and tracking breathing rate can help users intervene or treat episodes more quickly. As another example, breathing rate plays an important role as an indicator of physical effort during exercise, often more important than other physiological variables. Certain embodiments may track breathing rate and correlate the breathing rate with an estimate or prediction of the user's physical effort.

[0109] Respiratory rate is an important medical parameter. Embodiments disclosed herein may provide respiratory rate information to health professionals in real time and over prior time periods, such as providing respiratory rate information to a physician during a telehealth appointment for remote health monitoring. As another example, embodiments disclosed herein allow respiratory rate determination and monitoring without requiring a user to visit a health facility. For example, a user who has undergone surgery may be discharged when their condition permits, without requiring the patient to remain in the hospital simply to monitor the user's respiratory rate to ensure that the user remains stable. Respiratory rate and pattern are also important biomarkers for stress detection and management, and continuous respiratory rate monitoring may be used for stress detection and early intervention.

[0110] Fig.11 An example computer system 1100 is shown. In a particular embodiment, one or more computer systems 1100 perform one or more steps of one or more methods described or shown herein. In a particular embodiment, one or more computer systems 1100 provide functionality described or shown herein. In a particular embodiment, software running on one or more computer systems 1100 performs one or more steps of one or more methods described or shown herein, or provides functionality described or shown herein. A particular embodiment includes one or more portions of one or more computer systems 1100. In this document, references to computer systems may include computing devices, and vice versa, where appropriate. In addition, references to computer systems may include one or more computer systems, where appropriate.

[0111] The present disclosure contemplates any suitable number of computer systems 1100. The present disclosure contemplates computer systems 1100 in any suitable physical form. As an example and not by way of limitation, computer system 1100 may be an embedded computer system, a system on a chip (SOC), a single board computer system (SBC) (such as, for example, a computer on a module (COM) or a system on a module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a computer system grid, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these. Where appropriate, computer system 1100 may include one or more computer systems 1100; may be single or distributed; may span multiple locations; may span multiple machines; may span multiple data centers; or may reside in a cloud, where the cloud may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 1100 may perform one or more steps of one or more methods described or illustrated herein without substantial spatial or temporal limitations. As an example and not by way of limitation, one or more computer systems 1100 may perform one or more steps of one or more methods described or illustrated herein in real time or in batch mode. Where appropriate, one or more computer systems 1100 may perform one or more steps of one or more methods described or illustrated herein at different times or at different locations.

[0112] In a particular embodiment, computer system 1100 includes a processor 1102, memory 1104, storage 1106, input / output (I / O) interface 1108, communication interface 1110, and bus 1112. Although this disclosure describes and illustrates a particular computer system having particular numbers of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable numbers of any suitable components in any suitable arrangement.

[0113] In a particular embodiment, the processor 1102 includes hardware for executing instructions (such as those that constitute a computer program). As an example and not by way of limitation, to execute instructions, the processor 1102 may retrieve (or fetch) instructions from an internal register, an internal cache, a memory 1104, or a storage medium 1106; decode and execute them; and then write one or more results to an internal register, an internal cache, a memory 1104, or a storage medium 1106. In a particular embodiment, the processor 1102 may include one or more internal caches for data, instructions, or addresses. Where appropriate, the present disclosure contemplates a processor 1102 that includes any suitable number of any suitable internal caches. As an example and not by way of limitation, the processor 1102 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). The instructions in the instruction cache may be copies of instructions in the memory 1104 or the storage medium 1106, and the instruction cache may speed up the retrieval of those instructions by the processor 1102. The data in the data cache may be: a copy of data in the memory 1104 or storage medium 1106 for operation by instructions executed at the processor 1102; the result of a previous instruction executed at the processor 1102 for access by a subsequent instruction executed at the processor 1102 or for writing to the memory 1104 or storage medium 1106; or other suitable data. The data cache may accelerate read operations or write operations of the processor 1102. The TLB may accelerate virtual address translation for the processor 1102. In a particular embodiment, the processor 1102 may include one or more internal registers for data, instructions, or addresses. Where appropriate, the present disclosure contemplates a processor 1102 including any suitable number of any suitable internal registers. Where appropriate, the processor 1102 may include one or more arithmetic logic units (ALUs); may be a multi-core processor; or may include one or more processors 1102. Although the present disclosure describes and illustrates a particular processor, the present disclosure contemplates any suitable processor.

[0114] In a particular embodiment, the memory 1104 includes a main memory for storing instructions for execution by the processor 1102 or data for operation by the processor 1102. As an example and not by way of limitation, the computer system 1100 may load instructions from the storage medium 1106 or another source (such as, for example, another computer system 1100) to the memory 1104. The processor 1102 may then load the instructions from the memory 1104 to an internal register or an internal cache. In order to execute the instructions, the processor 1102 may retrieve the instructions from the internal register or the internal cache and decode them. During or after the execution of the instructions, the processor 1102 may write one or more results (which may be intermediate results or final results) to the internal register or the internal cache. The processor 1102 may then write one or more of those results to the memory 1104. In a particular embodiment, the processor 1102 executes only instructions in one or more internal registers or internal caches or instructions in memory 1104 (opposite to storage medium 1106 or elsewhere), and operates only on data in one or more internal registers or internal caches or in memory 1104 (relative to storage medium 1106 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple the processor 1102 to the memory 1104. Bus 1112 may include one or more memory buses, as described below. In a particular embodiment, one or more memory management units (MMUs) reside between the processor 1102 and the memory 1104 and facilitate access to the memory 1104 requested by the processor 1102. In a particular embodiment, the memory 1104 includes a random access memory (RAM). Where appropriate, the RAM may be a volatile memory, where appropriate, the RAM may be a dynamic RAM (DRAM) or a static RAM (SRAM). In addition, where appropriate, the RAM may be a single-port or multi-port RAM. The present disclosure contemplates any suitable RAM. Where appropriate, memory 1104 may include one or more memories 1104. Although this disclosure describes and illustrates particular memories, this disclosure contemplates any suitable memories.

[0115] In a particular embodiment, the storage medium 1106 includes a large-capacity storage medium for data or instructions. As an example and not by way of limitation, the storage medium 1106 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the storage medium 1106 may include a removable or non-removable (or fixed) medium. Where appropriate, the storage medium 1106 may be inside or outside the computer system 1100. In a particular embodiment, the storage medium 1106 is a non-volatile solid-state memory. In a particular embodiment, the storage medium 1106 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these. The present disclosure contemplates the use of a large-capacity storage medium 1106 in any suitable physical form. Where appropriate, storage media 1106 may include one or more storage control units that facilitate communication between processor 1102 and storage media 1106. Where appropriate, storage media 1106 may include one or more storage media 1106. Although this disclosure describes and illustrates particular storage media, this disclosure contemplates any suitable storage media.

[0116] In a particular embodiment, the I / O interface 1108 includes hardware, software, or both, providing one or more interfaces for communication between the computer system 1100 and one or more I / O devices. Where appropriate, the computer system 1100 may include one or more of these I / O devices. One or more of these I / O devices can enable communication between a person and the computer system 1100. As an example and not by way of limitation, the I / O device may include a keyboard, a keypad, a microphone, a monitor, a mouse, a printer, a scanner, a speaker, a static camera, a stylus, a tablet computer, a touch screen, a trackball, a camera, another suitable I / O device, or a combination of two or more of these. The I / O device may include one or more sensors. The present disclosure contemplates any suitable I / O device and any suitable I / O interface 1108 for them. Where appropriate, the I / O interface 1108 may include one or more devices or software drivers that enable the processor 1102 to drive one or more of these I / O devices. Where appropriate, the I / O interface 1108 may include one or more I / O interfaces 1108. Although this disclosure describes and illustrates a particular I / O interface, this disclosure contemplates any suitable I / O interface.

[0117] In a particular embodiment, the communication interface 1110 includes hardware, software, or both that provide one or more interfaces for communication (such as, for example, packet-based communication) between the computer system 1100 and one or more other computer systems 1100 or one or more networks. As an example and not by way of limitation, the communication interface 1110 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wired network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network (such as, a WI-FI network). The present disclosure contemplates any suitable network and any suitable communication interface 1110 for it. As an example and not by way of limitation, the computer system 1100 may communicate with one or more parts of an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or the Internet, or a combination of two or more of these. One or more parts of one or more of these networks may be wired or wireless. As an example, the computer system 1100 may communicate with a wireless PAN (WPAN) (such as, for example, a Bluetooth WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless networks, or a combination of two or more of these. Where appropriate, the computer system 1100 may include any suitable communication interface 1110 for any of these networks. Where appropriate, the communication interface 1110 may include one or more communication interfaces 1110. Although the present disclosure describes and illustrates particular communication interfaces, the present disclosure contemplates any suitable communication interface.

[0118] In a particular embodiment, bus 1112 includes hardware, software, or both that couples components of computer system 1100 to each other. As an example and not by way of limitation, bus 1112 may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front side bus (FSB), a hypertransport (HYPERTRANSPORT, HT) interconnect, an industry standard architecture (ISA) bus, an InfiniBand interconnect, a low pin count (LPC) bus, a memory bus, a microchannel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI Express (PCI-Express, PCIe) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus, or another suitable bus, or a combination of two or more of these. Where appropriate, bus 1112 may include one or more buses 1112. Although the present disclosure describes and illustrates a particular bus, the present disclosure contemplates any suitable bus or interconnect.

[0119] Herein, where appropriate, one or more computer-readable non-transitory storage media may include one or more semiconductor-based integrated circuits (ICs) or other integrated circuits (ICs) (such as, for example, field programmable gate arrays (FPGAs) or application-specific ICs (ASICs), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital (SECURE DIGITAL) cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these. Where appropriate, the computer-readable non-transitory storage media may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0120] As used herein, unless expressly stated otherwise or indicated otherwise by context, "or" is inclusive and not exclusive. Thus, as used herein, "A or B" means "A, B, or both," unless expressly stated otherwise or indicated otherwise by context. Furthermore, "and" is conjunctive and several, unless expressly stated otherwise or indicated otherwise by context. Thus, as used herein, "A and B" means "A and B, jointly or severally," unless expressly stated otherwise or indicated otherwise by context.

[0121] The scope of the present disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that would be understood by a person of ordinary skill in the art. The scope of the present disclosure is not limited to the example embodiments described or illustrated herein. In addition, although the present disclosure describes and illustrates various embodiments herein as including specific components, elements, features, functions, operations, or steps, any of these embodiments may include any combination or arrangement of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that would be understood by a person of ordinary skill in the art.

Claims

1. A system, include: one or more non-transitory computer-readable storage media storing instructions; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the one or more processors are operable to execute the instructions to perform the following operations: accessing motion data obtained by a first sensor of the wearable device from motion of a user wearing the wearable device; determining an activity level of the user from the athletic data; selecting an activity-based technique for estimating a breathing rate of the user based on the determined activity level; determining a breathing rate of the user using the selected activity-based technique and the accessed motion data; determining a mass associated with the determined breathing rate; comparing the determined quality associated with the determined breathing rate to a threshold quality; and In response to determining that the determined quality is not less than the threshold quality, using the determined breathing rate as a final breathing rate determination for the user; and In response to determining that the determined quality is less than the threshold quality, performing the following operations: activating a second sensor of the wearable device for a period of time; and A breathing rate of the user is determined based on data from a second sensor.

2. The system according to claim 1, in, The wearable device includes a head-mounted device.

3. The system according to claim 1, in, The activity level is an activity level selected from a set of activity levels including a resting activity level and a moving activity level.

4. The system according to claim 3, in, The one or more processors are operable to execute the instructions to determine a breathing rate of the user using a resting activity based technique associated with the resting activity level by: Apply a sliding window to the accessed data; selecting motion data corresponding to a specific axis from the accessed motion data; filtering the selected motion data corresponding to the specific axis; as well as Based on the filtered motion data, a breathing rate of the user is estimated.

5. The system according to claim 4, in, The one or more processors are operable to execute the instructions to perform the following operations: generating a plurality of respiration rate estimates according to a plurality of different respiration rate algorithms; and A breathing rate of the user is estimated by interpolating the multiple breathing rate estimates.

6. The system according to claim 3, in, The one or more processors are operable to execute the instructions to determine a breathing rate of the user using a movement activity based technique associated with the movement activity level by segmenting the accessed movement data into movement segments, wherein each movement segment corresponds to a breathing cycle of the user.

7. The system according to claim 1, in, The second sensor includes a microphone.

8. The system according to claim 7, in, The one or more processors are operable to execute the instructions to determine a quality of a breathing rate determined based on data from the microphone, wherein the quality is based at least on (1) a noise-to-respiration ratio associated with the data from the microphone, and (2) a number of breathing clusters identified in the data from the microphone.

9. The system according to claim 7, in, The one or more processors are operable to execute the instructions to perform the following operations: classifying data from the microphone using a set of categories including a breathing category and a transition category; Clustering classified data into multiple clusters; as well as A breathing rate of the user is determined based on the number of clusters having the transition class label.

10. The system according to claim 1, in, The one or more processors are operable to execute the instructions to perform the following operations: determining whether any activity-based techniques for estimating a breathing rate of the user correspond to the determined activity level; and When no activity-based technique for estimating the user's breathing rate corresponds to the determined activity level, then activating a second sensor of the wearable device.

11. The system according to claim 1, in, The one or more processors are operable to execute the instructions to: in response to determining that the determined quality is less than the threshold quality, determine whether to activate the second sensor based on one or more of the following: the amount of time that has elapsed since the second sensor was last activated; or The amount of time that has elapsed since the last valid determination of the user's breathing rate.

12. The system according to claim 1, in, The value of the threshold quality depends on one or more of the following: User preferences; a user-specific breathing quality score associated with the first sensor; or A user-specific breathing quality score associated with the second sensor.

13. The system according to claim 1, in, The second sensor is one second sensor of a plurality of second sensors, wherein each second sensor of the plurality of second sensors is ranked based on at least power consumption and detection accuracy of the sensor.

14. The system according to claim 1, in, The one or more processors are operable to execute the instructions to periodically repeat the process of claim 1 .

15. A method, include: accessing motion data obtained by a first sensor of the wearable device from motion of a user wearing the wearable device; determining an activity level of the user from the athletic data; selecting an activity-based technique for estimating a breathing rate of the user based on the determined activity level; determining a breathing rate of the user using the selected activity-based technique and the accessed motion data; determining a mass associated with the determined breathing rate; comparing the determined mass associated with the determined breathing rate to a threshold mass; as well as in response to determining that the quality of the determination is not less than the threshold quality, using the determined breathing rate as a final breathing rate determination for the user; or In response to determining that the determined quality is less than the threshold quality, performing the following steps: activating a second sensor of the wearable device for a period of time; as well as A breathing rate of the user is determined based on data from a second sensor.