Methods for operating multi-modal sensor devices and associated apparatuses

Abstract

This application relates to methods for operating a multi-modal sensor device and associated apparatus. The invention provides a BMD that has multiple device modes depending on the operating conditions of the device, such as motion intensity, device placement, and / or activity type, which are associated with various data processing algorithms. In some embodiments, the BMD is implemented as a wrist-worn or arm-worn device. In some embodiments, methods are provided for tracking physiological metrics using the BMD. In some embodiments, the process and the BMD apply time-domain analysis to data provided by the sensors of the BMD when the data has high signal (e.g., high signal-to-noise ratio), and apply frequency-domain analysis to the data when the data has low signal, which facilitates improved accuracy and speed of biometric data.

Description

[0001] This application is a divisional application of Chinese patent application No. 201911218374.X, filed on March 17, 2015, entitled "Method for Operating a Multi-Mode Sensor Device and Associated Equipment," which in turn is a divisional application of Chinese patent application No. 201510117698.X, filed on March 17, 2015, entitled "Method for Operating a Multi-Mode Sensor Device and Associated Equipment." Technical Field

[0002] The present invention relates to a method for operating a multi-mode sensor device and associated equipment. Background Technology

[0003] Sensor devices can infer biometrics of interest from sensor data associated with a user's activities. However, in many implementations of sensor devices, high accuracy in biometric estimation is achieved by limiting the types and / or intensity of activities that the sensor device can monitor. For example, it is recommended to wear the pedometer in the left mid-armpit position for the most accurate step count (Horvath et al., 2007). Even with ideal placement, the pedometer may not provide a reliable step count due to overestimating or underestimating steps taken during activities such as riding in a vehicle.

[0004] The placement of sensor devices is a significant constraint. Users of sensor devices prefer to wear their portable devices in convenient locations. However, these convenient locations are often not ideal for collecting biometric data. For example, the sensor device may be located far from the body parts(s) primarily involved in the activity or exhibiting the strongest biometric signals. For this reason, current sensor devices sacrifice convenience for accuracy or vice versa.

[0005] Recent advancements in the miniaturization of sensors, electronic devices, and power supplies have enabled the provision of personal health monitoring devices, also referred to herein as "biometric tracking" or "biometric monitoring" devices, in small sizes. These biometric monitoring devices can collect, derive, and / or provide one or more of the following types of information: steps, walking speed, distance traveled, pace, heart rate, calories burned, number of floors climbed and / or descended, location and / or orientation, altitude, etc. However, the small size of these products limits their power consumption. Therefore, there is a need for energy-efficient methods and hardware that allow for high-speed and accurate calculation of biometric information.

[0006] The invention disclosed herein enables sensor devices to use one or more modes to achieve computational speed and accuracy while maintaining energy efficiency. Summary of the Invention

[0007] This invention enables sensor devices to use one or more modes. In some embodiments, different types of modes operate simultaneously. In other embodiments, the most appropriate mode or group of modes is selected for use at any given time. These modes include, but are not limited to, different exercise intensities, sensor device placement locations (e.g., where they are worn), and / or activity types. By automatically or manually switching between these modes, the sensor device more accurately tracks biometric data regardless of exercise intensity, placement location, and / or activity type, while maintaining computational efficiency.

[0008] This invention provides a biometric analyzer (BMD) having multiple device modes depending on the device's operating conditions, such as exercise intensity, device placement, and / or activity type, and these device modes are associated with various data processing algorithms. In some embodiments, a method for tracking physiological measurements using the BMD is provided. In some embodiments, the process and the BMD apply time-domain analysis to the data when the data provided by the BMD's sensors has a high signal (e.g., a high signal-to-noise ratio) and apply frequency-domain analysis to the data when the data has a low signal, which contributes to improved accuracy and speed of biometric data.

[0009] Some embodiments of the present invention provide a method for tracking a user's physiological activity using a wearable biometric monitoring device (BMD). The BMD has one or more sensors that provide output data indicative of the user's physiological activity. The method involves analyzing the sensor output data provided by the biometric monitoring device to determine that the output data has a relatively low signal-to-noise ratio (SNR) during the user's active state. After this determination, the BMD immediately collects the sensor output data for a duration sufficient to identify periodic components of the data. The BMD then uses frequency domain analysis of the collected sensor output data to process and / or identify the periodic components. The BMD determines a measure of the user's physiological activity from the periodic components of the collected sensor output data. Finally, the BMD can present the measure of the user's physiological activity. In some embodiments, the one or more sensors of the BMD include motion sensors, and the output data includes motion intensity from the motion sensors. In some embodiments, the wearable biometric monitoring device includes a wrist-worn or arm-worn device.

[0010] Some embodiments of the present invention provide a method for tracking a user's physiological activities using a wearable biometric monitoring device (BMD). The method includes: (a) analyzing sensor output data provided by the biometric monitoring device to determine that the user is engaged in a first activity that produces a relatively high SNR in the sensor output data; (b) quantifying a physiological metric by analyzing the first set of sensor output data in the time domain; (c) analyzing subsequent sensor output data provided by the biometric monitoring device to determine that the user is engaged in a second activity that produces a relatively low SNR in the subsequent sensor output data; and (d) quantifying the physiological metric from the periodic components of the second set of sensor output data by processing the second set of sensor output data using frequency domain analysis. For example, the first activity may be running, in which the hands move freely. The second activity may be walking while pushing a stroller. In some embodiments, the frequency domain analysis includes one or more of the following: Fourier transform, cepstral transform, wavelet transform, filter bank analysis, power spectral density analysis, and / or periodogram analysis.

[0011] In some embodiments, the quantization operation in (d) requires more computation per unit of the sensor output data duration than the quantization in (b). In some embodiments, the quantization in (d) requires more computation per unit of the physiological metric than the quantization in (b).

[0012] In some embodiments, (b) and (d) each involve: identifying a periodic component from the sensor output data; determining the physiological measure from the periodic component of the sensor output data; and presenting the physiological measure.

[0013] In some embodiments, the sensor output data comprises raw data obtained directly from the sensor without preprocessing. In some embodiments, the sensor output data comprises data derived from the raw data after preprocessing.

[0014] In some embodiments, the wearable biometric monitoring device is a wrist-worn or arm-worn device.

[0015] In some embodiments, the operation of analyzing sensor output data in (a) or (c) involves characterizing the output data based on the signal norm, signal energy / power in certain frequency bands, wavelet scaling parameters, and / or the number of samples exceeding one or more thresholds.

[0016] In some embodiments, the process further involves analyzing biometric information previously stored on the biometric monitoring device to determine whether the user participated in the first activity or the second activity.

[0017] In some embodiments, the one or more sensors include motion sensors, wherein analyzing sensor output data in (a) or (c) involves using motion signals to determine whether the user participated in the first activity or the second activity. In some embodiments, the first activity involves free movement of the limb wearing the biometric monitoring device during the activity. In some embodiments, the second activity includes reduced movement of the limb wearing the biometric monitoring device during the activity. In some embodiments, the second activity involves the user holding a substantially non-accelerating object with the limb wearing the biometric monitoring device.

[0018] In some embodiments, analyzing the first set of sensor output data in the time domain involves applying peak detection to the first set of sensor output data. In some embodiments, analyzing the second set of sensor output data involves identifying periodic components of the second set of sensor output data. In some embodiments, the first set of sensor output data includes data from only one axis of the multi-axis motion sensor, wherein the second set of sensor output data includes data from two or more axes of the multi-axis motion sensor.

[0019] In some embodiments, the frequency domain analysis involves bandpassing the time-domain signal and then applying peak detection in the time domain. In some embodiments, the frequency domain analysis includes finding any spectral peaks as a function of the average step frequency. In some embodiments, the frequency domain analysis involves performing a Fisher periodicity test. In some embodiments, the frequency domain analysis includes using harmonics to estimate the period and / or test periodicity. In some embodiments, the frequency domain analysis includes performing a generalized likelihood ratio test with its parametric model and the harmonicity of the moving signal.

[0020] Some embodiments further involve analyzing sensor output data to classify motion signals into two categories: signals generated from walking and signals generated from activities other than walking.

[0021] In some embodiments, the physiological metrics provided by the BMD include step count. In some embodiments, the physiological metrics include heart rate. In some embodiments, the physiological metrics include number of stairs climbed, calories burned, and / or sleep quality.

[0022] Some embodiments further involve applying a classifier to the sensor output data and the subsequent sensor output data to determine the placement of the biometric monitoring device on the user. In some embodiments, the process in (b) includes using information about the placement of the biometric monitoring device to determine the value of the physiological measurement.

[0023] Some embodiments further include applying a classifier to the sensor output data and the subsequent sensor output data to determine whether the user participated in the first activity and / or the second activity. In some embodiments, the first activity is one of the following: running, walking, elliptical trainer, Stair Master, cardio machine, weightlifting, driving, swimming, cycling, stair climbing, and rock climbing. In some embodiments, the processing in (b) includes using information about the type of activity to determine the value of the physiological metric.

[0024] Some embodiments provide a method for tracking a user's physiological activities using a wearable BMD, the method involving: (a) determining that the user is engaged in a first type of activity by detecting a first signature signal in sensor output data, the first signature signal being selectively associated with the first type of activity; (b) quantifying a first physiological measure of the first type of activity from a first set of sensor output data; (c) determining that the user is engaged in a second type of activity by detecting a second signature signal in sensor output data, the second signature signal being selectively associated with the second type of activity and different from the first signature signal; and (d) quantifying a second physiological measure of the second type of activity from a second set of sensor output data. In some embodiments, the first signature signal and the second signature signal include motion data. In some embodiments, the first signature signal and the second signature signal further include one or more of the following: location data, pressure data, light intensity data, and / or altitude data.

[0025] Some embodiments provide a biometric monitoring device (BMD) that includes one or more sensors providing sensor output data, which includes information about the user's activity level when the biometric monitoring device is worn by the user. The BMD also includes control logic configured to: (a) analyze the sensor output data to characterize the output data as indicating a first activity associated with a relatively high signal level or a second activity associated with a relatively low signal level; (b) process the sensor output data indicating the first activity to generate a value for the physiological metric; and (c) process the sensor output data indicating the second activity to generate a value for the physiological metric. In some embodiments, the processing in (b) requires more computation per unit of the physiological metric than the processing in (c).

[0026] Some embodiments provide a BMD with control logic configured to: (a) analyze sensor output data provided by the biometric monitoring device to determine that the user participated in a first activity that produced a relatively high SNR in the sensor output data; (b) quantify a physiological metric by analyzing the sensor output data in the time domain; (c) analyze subsequent sensor output data provided by the biometric monitoring device to determine that the user participated in a second activity that produced a relatively low SNR in the subsequent sensor output data; and (d) quantify the physiological metric from the periodic components of the subsequent sensor output data by processing the subsequent sensor output data using frequency domain analysis. In some embodiments, the analysis in (d) requires more computation per unit of the physiological metric than the analysis in (b).

[0027] Details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Note that the relative dimensions in the following figures may not be drawn to scale unless explicitly indicated as such.

[0028] These and other implementation schemes are described in further detail with reference to the figures and the following detailed description. Attached Figure Description

[0029] The various embodiments disclosed herein are illustrated by way of example rather than limitation in the accompanying drawings, in which similar reference numerals may refer to similar elements.

[0030] Figure 1 Examples of portable biometric monitoring devices with buttons and displays according to some embodiments of the present invention are shown.

[0031] Figure 2 Examples of wristwatch-like biometric monitoring devices according to some embodiments of the present invention are shown.

[0032] Figure 3 A flowchart illustrating a method for tracking a user's physiological activities according to some embodiments.

[0033] Figure 4A The image displays acceleration data in the time domain (top) and frequency domain (bottom) for users' stationary, walking, and running activities. Figure 4B The data presented includes similar data for running while stationary, with hands on a bar, and running with hands free.

[0034] Figure 5A This is a flowchart illustrating a process for tracking steps using BMD according to some embodiments. Figure 5BThis demonstrates a process for determining three ranges of motion intensity patterns according to some embodiments.

[0035] Figure 6A This is a flowchart illustrating a process for implementing peak detection to calculate the number of steps in an activity mode, according to some embodiments. Figure 6B This is a flowchart illustrating a process that can be used to implement peak detection according to some embodiments. Figure 6C This is a flowchart illustrating a process for analyzing data in a semi-active mode in the frequency domain, according to some embodiments. Figure 6D This is a flowchart illustrating a process that can be used to perform spectrum analysis according to some embodiments.

[0036] Figure 7 A generalized schematic diagram depicting an example of a portable biometric monitoring device or other device capable of implementing the multi-mode functions described herein. Detailed Implementation

[0037] introduce

[0038] Sensor devices or biometric monitoring devices (BMDs) according to embodiments described herein typically have a shape and size suitable for coupling to (e.g., fastened to, worn, carried on, etc.) a user's body or clothing. BMDs are also referred to herein as biometric tracking devices. The devices collect one or more types of physiological and / or environmental data from embedded sensors and / or external devices.

[0039] In many applications, users of the BMD prefer to wear it on their wrist. Therefore, in some embodiments, the BMD is implemented as a watch-like, wrist-worn device. Although many activity signatures exist in data obtained from the wrist or arm, this data is inherently corrupted by unwanted motion and environmental noise. This poses a challenge when attempting to infer certain user activities, such as walking, using data obtained from a sensor device worn on the wrist. This invention provides a solution to this problem by offering multiple patterns to facilitate the inference problem. Some embodiments use automated methods to determine the patterns. Some embodiments use user input to determine the patterns. Various embodiments provide different data processing algorithms suitable for different user activities and conditions.

[0040] Biometric monitoring devices (BMDs) are typically very small due to practical considerations. People who wish to monitor their performance are unlikely to want to wear large, bulky devices that might interfere with their activities or appear unsightly. As a result, biometric monitoring devices are often offered in small form factors to allow for lightweight and portability. Such small form factors often inevitably lead to some design compromises. For example, there may be limited space for the display, controls, and other components of the biometric monitoring device within the device housing. One system component that may be limited in size or performance is the power supply for the biometric monitoring device, such as a battery or capacitor. In many implementations, the biometric monitoring device may be in a "constantly on" state to allow it to continuously collect biometric data throughout the day and night. Given that the sensors and processor of the biometric monitoring device must be kept substantially powered to collect biometric data, it can be advantageous to implement power-saving features elsewhere in the device, such as by causing the display to automatically turn off after a period of time or by measuring certain data, such as heart rate data, on demand based on user-initiated actions. Typical user actions can be demonstrated by pressing a button on the biometric monitoring device, flipping the biometric monitoring device back and forth, double-clicking the casing of the biometric monitoring device, touching the surface area, or bringing a body part close to the proximity sensor.

[0041] For example, there is often a trade-off between the speed and accuracy of biometric data such as steps, pace, and heart rate. This trade-off is further exacerbated by the limited power supply of miniaturized BMDs. The present invention addresses this problem by providing a BMD with multiple device modes depending on the device's operating conditions (e.g., exercise intensity, device placement, and / or activity type).

[0042] In some embodiments, a mode can be used alone. In other embodiments, multiple modes can be combined at a specific time. For example, when a user is wearing the BMD on their dominant hand, swinging their hand freely, and climbing a flight of stairs, the device can simultaneously use the free movement mode (movement intensity), the stair climbing mode (activity type), and the dominant hand mode (device placement). In some embodiments, one or more of the modes can be selected by automatic triggering, as further described below. In some embodiments, one or more modes can be manually selected by the user via a user interface.

[0043] In some embodiments, data collected by the sensor device is transmitted or relayed to other devices. For example, while a user is wearing the sensor device, the device may use one or more sensors to calculate and store the user's step count. The device then transmits the data representing the user's step count to an account on a network service such as a computer, mobile phone, or health station, where the user can store, process, and observe the data. In practice, the sensor device may measure or calculate a number of other physiological metrics in addition to or in lieu of the user's step count. These include, but are not limited to, energy expenditure (e.g., calories burned), number of floors climbed and / or descended, heart rate, heart rate variability, heart rate recovery, location and / or orientation (e.g., via GPS), altitude, walking speed and / or distance traveled, number of swimming laps, swimming style, cycling distance and / or speed, blood pressure, blood glucose, skin conductance, skin and / or body temperature, electromyography, electroencephalography, weight, body fat, calorie intake, nutrient intake from food, medication intake, sleep cycle (i.e., clock time), sleep stage, sleep quality and / or duration, pH level, hydration level, and respiratory rate.

[0044] In some embodiments, the sensor device may also measure or calculate metrics related to the user’s surrounding environment, such as atmospheric pressure, weather conditions (e.g., temperature, humidity, pollen count, air quality, rain / snow conditions, wind speed), light exposure (e.g., ambient light, UV light exposure, time spent in darkness and / or duration), noise exposure, radiation exposure, and magnetic fields.

[0045] Furthermore, the sensor device can calculate metrics derived from the combination of the aforementioned data. For example, the sensor device can calculate a user's stress and / or relaxation level using a combination of heart rate variability, skin conduction, noise pollution, and sleep quality. In another example, the sensor device can determine the efficacy of a medical intervention (e.g., medication) using a combination of medication intake, sleep, and / or activity data. In yet another example, the sensor device can determine the efficacy of an allergy medication using a combination of pollen data, medication intake, sleep, and / or activity data.

[0046] While the examples presented above illustrate the computation of measurements on a sensor device, these can be performed, partially or entirely, on an external system (e.g., a web server, mobile phone, or personal computer). In fact, these examples are provided for illustrative purposes only and are not intended to be limiting or exhaustive. Further embodiments and implementations of the sensor device can be found in U.S. Patent Application No. 13 / 156,304, filed June 8, 2011, entitled "Portable Biometric Monitoring Devices and Methods of Operating Same," which is incorporated herein by reference in its entirety.

[0047] Sensors are the basic sensing hardware of tracking devices, such as accelerometers, magnetometers, gyroscopes, and PPG sensors. Details of various sensors and their types are further described below.

[0048] Sensor output data is the direct output from the sensors of the tracking device. Examples include acceleration, light intensity, etc. This data varies over time and may contain constant or variable frequency and / or amplitude components. It may contain biometric information about user activity and / or environmental information about surrounding conditions that exist independently of user activity.

[0049] In some embodiments, the sensor output data comprises raw data obtained directly from the sensor without preprocessing. In some embodiments, the sensor output data comprises data derived from the raw data after preprocessing.

[0050] Physiological metrics are physiologically relevant measures determined from sensor output data of a tracking device. They are sometimes called biometric performance metrics. Physiological metrics can be characterized in various ways. For example, they can be characterized by: (1) basic units of physiological activity, such as steps, swimming strokes, pedal strokes, heart rate, etc.; (2) increments in physiological output, such as swimming pool laps, stair climbs, heart rate, etc.; or (3) goals, including default or customized goals, such as walking 10,000 steps a day.

[0051] In this document, "activity type mode" refers to a device mode associated with distinct user activities (e.g., walking / running, rock climbing, sleeping, cycling, swimming, etc.). Each activity type mode may have associated triggers and sensor data processing algorithms.

[0052] "Triggering event" is used to refer to an event that causes the tracking device to enter a specific device mode.

[0053] Some device operations may be unique for specific activity types. Examples include displayed content, display sequence, etc.

[0054] The "sensor data processing algorithm" is used in conjunction with the computational process associated with the device mode. The sensor data processing algorithm is used to convert sensor output data into physiological measures defined for the activity type. The tracking device will have multiple sensor data processing algorithms, each associated with one or more activity type modes. In some embodiments, different activity intensity modes have different sensor data processing algorithms.

[0055] Various exercise intensity modes can be combined with activity type modes. An exercise intensity mode may contain two or more modes. In some embodiments, the exercise intensity mode has high, medium, and low intensity modes. Each exercise intensity mode has its own triggering event and / or sensor data processing algorithm, and may have other features such as displayed content. In one instance, the exercise intensity mode distinguishes between high activity (e.g., walking) and low activity (e.g., running). Another instance distinguishes between walking with free arm swing and walking with the arm fixed to a stationary object such as a treadmill handle. Typically, the tracking device will determine the same physiological measure for different exercise intensity modes of the same activity type; therefore, the device can determine the number of steps for both walking with free arm swing and walking with the arm fixed.

[0056] Motion intensity patterns are often employed to address the current environment or context of a device. For example, data processing algorithms for motion intensity patterns can be designed to improve the accuracy of the output information for a specific environment or context and / or save power in such environments or contexts. Some data processing algorithms require more processing power and therefore consume more energy, and such algorithms should only be used when accuracy is required. As an example, subtypes of activity that generate periodic signals with large amplitudes or signal-to-noise ratios (SNR) can be processed in the time domain at low cost, while other subtypes that generate low amplitudes or SNRs may require computationally demanding algorithms in the frequency domain.

[0057] The term "monitoring" is used in reference to a tracking device mode that presents monitored information about different physiological activities, such as heartbeat or walking. Monitoring as a device mode differs from activity type modes seen in classic examples such as heart rate monitors, as it is not specific to any particular activity type. A heart rate monitor can measure and / or present the basic unit of cardiac activity (heartbeat) and / or the increase in cardiac activity (heart rate). A tracking device may have multiple monitors, each with its own triggers and sensor data processing algorithms. Other device operations that can be specific to the monitors include display content, display sequence, etc. Monitors may have sub-modes with their own triggers and data processing algorithms, as discussed for activity type modes.

[0058] Device state modes are operational modes used in reference to various hardware states. Examples include high / low battery mode, synchronization mode, timer mode, stopwatch mode, and comment mode.

[0059] Figure 1 A biometric monitoring device (BMD) capable of implementing the multimodal functions disclosed herein is demonstrated. Figure 1 The BMD 100 includes a housing 102 containing electronics associated with the biometric monitoring device 100. The housing 102 includes motion sensors and other sensors. The BMD also has a button 104 to receive user input via button press. In some contexts, a button press received via button 104 can represent a manual command to change the mode of the BMD in a manner described below. The BMD 100 also includes a display 106 accessible / visible via the housing 102. (The following is a description of the display 100.) Figure 7 The schematic diagram shown further illustrates the components that can be integrated into the BMD.

[0060] Figure 2 Another embodiment of a multi-mode BMD is depicted, which can be worn on a person's forearm like a wristwatch, very similar to Fitbit FLEX™ or FORCE™. The biometric monitoring device 200 has a housing 202 containing electronics associated with the biometric monitoring device 200. Buttons 204 and a display 206 are accessible / visible via the housing 202. A wristband 208 can be integrated with the housing 202.

[0061] Multimodal features

[0062] When using a BMD to track physiological activity, the speed and accuracy of measurements are affected by various factors, such as device placement, the type of activity the user is engaging in, and the characteristics of the user's movement. For example, a user may be wearing the BMD on the wrist of their dominant hand for step counting purposes. They may be running on a treadmill while holding the handles and occasionally flipping through a magazine. This situation poses challenges to conventional methods and devices for tracking steps and detection. The fact that the user is holding the handles reduces the motion signal in their wrist that can be detected by the BMD's motion sensors. Moreover, occasional hand movements (originating from flipping through a magazine) generate motion noise, which the BMD may misinterpret as walking.

[0063] In some embodiments, methods and apparatus are provided to overcome difficulties as seen in similar situations. In some embodiments, BMD uses peak detection analysis for user activity with high signal-to-noise ratio (SNR) because peak detection analysis is often time and energy efficient, requiring less data and processing, as well as the energy associated with processing. Furthermore, BMD uses periodicity analysis for activity with lower signal or SNR, which is better at picking up relatively low signals and filtering out motion noise that does not have a regular time pattern. In some embodiments, BMD has the capability to automatically trigger various device modes to apply appropriate algorithms for analysis and processing. In some embodiments, signal periodicity is obtained through frequency domain analysis. In some embodiments, signal periodicity can be obtained through time domain analysis. In some embodiments, frequency domain analysis and time domain analysis can be combined to obtain periodicity.

[0064] Figure 3 A flowchart illustrating a method 300 for tracking a user's physiological activities according to some embodiments is provided. The method uses a wearable biometric monitoring device (BMD) having one or more sensors to provide output data indicative of the user's physiological activities. Method 300 begins by analyzing the sensor output data to determine that the user is engaged in a first activity that produces output data with a relatively high SNR. See box 310. Method 300 proceeds to quantifying physiological metrics, such as steps or heart rate, by analyzing the first set of sensor output data in the time domain. See box 320. In some embodiments, the BMD includes motion sensors, and the sensor output data includes the amplitude of acceleration. In some of such embodiments, the time-domain analysis may involve the detection of peak acceleration. Method 300 also involves analyzing subsequent sensor output data to determine that the user is engaged in a second activity that produces a relatively low SNR (compared to the previous sensor output data) in the subsequent sensor output data. See box 330. Furthermore, method 300 involves quantifying physiological metrics from the periodic components of the second set of sensor output data by processing the second set of sensor output data using frequency domain analysis. See box 340. In some embodiments, the frequency analysis involves spectral analysis to detect spectral peaks and harmonics. In other embodiments, frequency analysis applies a bandpass filter to the data, followed by peak detection to the frequency-filtered data to obtain periodicity information in a second set of sensor output data. The peak detection algorithm may work for time-domain data, but filtering is performed in the frequency domain. In some implementations, instead of calculating the SNR, the sensor output data is characterized by a process of classifying it according to an indicated SNR. For example, a classifier can be used to classify data based on motion or signal strength using inputs such as acceleration amplitude or power and other characteristics of the accelerometer output.

[0065] Method 300 applies time-domain analysis to data with relatively high signal (or SNR) and frequency analysis to data with relatively low signal. In some embodiments, the method exclusively applies time-domain analysis to high SNR data and applies at least some frequency-domain analysis to low SNR data. In some embodiments, the BMD applies different motion intensity patterns triggered by different motion intensity levels measured by motion sensors, reflecting different user activity characteristics. The criteria for distinguishing the signal levels of the two analyses should reflect different characteristics of user activity, such as running with free hand movement versus running with hand gripping a stick. Different motion metrics can be used as measures for determining motion intensity patterns, such as SNR, signal norm, signal energy / power in certain frequency bands, wavelet scaling parameters, and / or the number of samples exceeding one or more thresholds. Different values ​​can be set as criteria for relatively low and relatively high signals. In some embodiments, a single value can be used to separate a first activity from a second activity. In some embodiments, a third activity may be determined to have a lower activity level than the second activity (relatively low activity). The device may enter an inactive mode and not perform further analysis on the sensor output data.

[0066] In some embodiments, the sensor device can measure a user's activity intensity via step counting. The sensor device may implement one or more motion sensors that provide continuous or digitized time-series data to processing circuitry (e.g., an ASIC, DSP, and / or a microcontroller unit (MCU)). The processing circuitry runs algorithms to interpret the motion signals and derive activity data. In the case of a pedometer, the derived activity data includes the number of steps. In some embodiments, when the sensor output data signal is relatively low, the method analyzes motion data from multiple axes of a multi-axis motion sensor. In some embodiments, when the sensor output data signal is relatively high, the method analyzes motion data from only a single axis of a multi-axis motion sensor, improving the time and energy efficiency of calculating physiological metrics.

[0067] Pattern categories

[0068] This subsection outlines the different types of modes. Sections below explain how various modes can be triggered and how different analyses and processes are applied to derive biometric information. In some embodiments disclosed herein, the BMD has different kinds of modes triggered by different conditions and associated with different treatments tailored to those conditions. In some embodiments, device modes are provided in various categories: exercise intensity mode, device placement mode, activity type mode, device state mode, etc. In some embodiments, modes from different categories can be combined for specific conditions. For example, a semi-active exercise intensity mode, a running activity type mode, and a dominant hand device placement mode can be combined for the scenario described above of holding the handles while running on a treadmill.

[0069] Activity Type Mode

[0070] In some embodiments, the BMD tracks motion-related activities. In some embodiments, the BMD applies different processing algorithms to different activity types to provide the speed and accuracy of biometric measurements and to provide activity-specific metrics. For example, the BMD can provide altitude and route difficulty level in climbing mode, but it can provide speed and pace in running mode.

[0071] In some embodiments, the activity type mode may include, but is not limited to, running, walking, elliptical trainer and step aerobic machine, cardio machine, weightlifting, driving, swimming, cycling, climbing stairs and rock climbing.

[0072] Exercise intensity mode

[0073] In some embodiments, two or more different motion intensity modes may exist. In some embodiments, the BMD applies different processing algorithms to different motion intensity modes to optimize the speed and accuracy of biometric measurements and provide activity-specific metrics. In some embodiments, three motion intensity modes may be described based on three levels or ranges of motion intensity measured by motion sensors. These are sometimes loosely characterized herein as active mode, semi-active mode, and inactive mode. The algorithmic determination of modes and the transitions between modes are further discussed herein, which enables step counting and subsequent measurements of the user's biometric signals in a continuous manner. It should be noted that the three-mode approach described herein is illustrative and does not limit the invention. Fewer or more than three modes may exist (e.g., in a two-mode system, active and inactive (e.g., in a car)). In practice, the number of modes may vary depending on the user and the typical activities the user performs. The number of modes may also be dynamically changed for each user depending on the likelihood of the user engaging in certain activities. For example, a highly active mode may be deactivated when GPS is detected being used by the user. The following description provides further details regarding the triggering events for entering different motion intensity modes. Often, exercise intensity modes are specifically designed for certain types of activities, such as step counting.

[0074] Device placement mode

[0075] Sensor devices can infer a user's activity level using algorithms by processing signals from sensors (e.g., motion, physiological, environmental, location, etc.). In the case of motion sensing, the signal can be affected by the placement of the sensor device. For example, the motion signatures of the dominant and non-dominant hands differ significantly, leading to inaccuracies in estimating activity levels from motion signals generated at the wrist, as users can choose to mount the sensor device on either hand and switch from one hand to the other as needed. A set of modalities is designed to account for different placements, ensuring accurate and consistent biometric measurements regardless of where the user wears their sensor device.

[0076] Placement patterns may include, but are not limited to, the user's pockets, belts, belt loops, wristbands, shirt sleeves, shirt collars, shoes, shoelaces, hats, bras, ties, socks, underwear, coin purses, other clothing, and accessories such as helmets, gloves, small purses, backpacks, waist packs (belt packs, fanny packs), goggles, swim caps, glasses, sunglasses, necklaces, pendants, pins, hair accessories, bracelets, wristbands, armbands, and earrings, as well as equipment such as skis, ski poles, skis, bicycles, roller skates, and ice skates. Additional patterns may include those listed above (additionally specified locations on the dominant or non-dominant limb and / or the left or right side of the user's body (e.g., a wristband on the user's dominant right side)).

[0077] Monitoring and device status modes

[0078] In some embodiments, the BMD has different monitoring modes. Monitoring is a tracking device mode that presents monitored information about different physiological activities, such as heart rate or steps. Monitoring as a device mode differs from activity type modes seen in classic examples of heart rate monitors, as it is not specific to activity type. A heart rate monitor can measure and / or present the basic unit of cardiac activity (heartbeat) and / or the increase in cardiac activity (heart rate). The tracking device may have multiple monitors, each with its own triggers and sensor data processing algorithms. Other device operations that may be specific to the monitors include display content, display sequence, etc. The monitor may have sub-modes with their own triggers and data processing algorithms, as discussed for the activity type mode.

[0079] Device states are operating modes associated with various hardware states. Examples include high / low battery mode, synchronization mode, timer mode, stopwatch mode, comment mode, etc.

[0080] Triggers for entering activity type mode, device placement mode, and monitoring.

[0081] Manual trigger

[0082] In some embodiments, a user can manually trigger one or more modes of the BMD. In some embodiments, direct user interaction with the BMD (e.g., touching, pushing buttons, performing gesturing actions, etc.) can trigger the device to enter a specific activity type mode, device placement mode, and monitoring. In some embodiments, a user can trigger the device to enter a mode by interacting with an assistive device communicatively connected to the BMD, as described later herein. For example, a user can select an activity type mode from a list of options in a smartphone application or web browser.

[0083] The mode of the sensor device can be manually selected by the user. In this case, several methods can be considered when setting the most suitable mode. In one embodiment, the mode selection can be determined entirely or partially from information collected during sensor device pairing and from the user's online account. Each sensor device can be paired with an online account or an auxiliary computing device such as a smartphone, laptop, desktop computer, and / or tablet computer (which enables the input and storage of user-specific information, including but not limited to the user's placement preferences). This user-specific information can be transmitted to the user's activity monitoring device via wireless or wired communication protocols. For example, in an embodiment where the sensor device can be worn on either wrist, the user can select a dominant or non-dominant hand setting to tune the biometric algorithm for the wearing location.

[0084] In some embodiments, a placement or activity mode can be set via a user interface on the device. The user can set the mode via an interface including a display, buttons, and / or a touchscreen. Mode selections can be stored in the device's local storage or in auxiliary electronics (including, but not limited to, a server) that communicate with the sensor devices.

[0085] Hand gestures observed via motion sensors can also be used to set such modes. There can be a one-to-one correspondence between modes and hand gestures, so that a specific hand gesture (e.g., shaking a device) triggers a mode. Alternatively, a series of hand gestures can be used to enter a mode, such as a hand shaking motion followed by a figure-eight motion. In these cases, the user can receive confirmation of the mode via assistive sensory stimuli (e.g., a vibrating motor or a pattern played on LEDs).

[0086] Automatic trigger

[0087] In addition to manual mode settings, automated algorithms (e.g., machine learning) can be applied to detect placement and / or activity types. In some embodiments, the tracking device sensor output contains a detectable activity type signature. The BMD can automatically detect the activity type signature and trigger the BMD to enter an activity type mode corresponding to the activity type signature. In some embodiments, the BMD interacts with an external signal that triggers the BMD to enter the activity type mode or is monitored. The external signal may be provided by, for example, an RFID tag or other short-range communication probe / signal attached to an activity type-related object (e.g., a bicycle handlebar or climbing hold). In some embodiments, the external signal may be provided by the environment, such as ambient light intensity.

[0088] In some embodiments, automatic triggering is implemented using only motion sensors. The signature of the motion signal varies significantly depending on the placement of the sensor device. Even at the same placement location, each user's activity will be recorded as a motion signal with different characteristics in the time domain and the transform domain (including but not limited to the spectral domain). Therefore, machine learning classification techniques (e.g., decision tree learning, hidden Markov models (HMM), and linear discriminant analysis) can be considered for this supervised learning. For offline training, data is collected and labeled according to the placement of the sensor device and the type of user activity. Features are then extracted from the data in the time domain and its transformed representation (including but not limited to Fourier transform and wavelet transform). The features are then used to train coefficients to determine the decision rule. This set of coefficients can be trained offline (e.g., in the cloud during post-processing). The set of coefficients is then incorporated into the embedded system of the sensor device to determine the user's device placement location and type of activity.

[0089] In some embodiments, additional sensors may be used in addition to motion sensors to detect activity. These additional sensors may include, but are not limited to, those further described below. Activity type can be statistically inferred from signals from the additional sensors, whether or not they contain motion signals. For example, a Hidden Model (HMM) can be used, where hidden states are defined as physical activities, and observed states are a subset or all of the sensor signals. An example of using additional sensors to automatically trigger activity type patterns is automatically detecting swimming via a pressure sensor by detecting an increase in immersion pressure or high pressure. GPS data or GPS signals combined with some signatures in the motion signals can be statistically modeled to determine the desired metric for user activity, such as speed, for activities like driving and cycling.

[0090] In some embodiments, a mode can be selected automatically or semi-automatically (e.g., automatically performing one or more, but not all, steps of the mode selection process) using short-range wireless communication, as described in U.S. Patent Application No. 13 / 785,904, filed March 5, 2013, entitled “Near Field Communication System, and Method of Operating Same,” which is incorporated herein by reference in its entirety. In some embodiments, the wireless device can be placed at a specific location associated with the activity to be detected. For example, an NFC chip can be attached to fitness equipment. A fitness user can tag the fitness equipment with its NFC-enabled sensor device before and after a specific workout. In one embodiment, the NFC chip mounted on the fitness equipment can also transmit workout data collected from the fitness equipment, which can be used to correct and / or improve activity data measured by the sensor device.

[0091] Even during the activity, wireless devices can be used to track the intensity and efficiency of the activity. One implementation of this idea involves NFC-equipped climbing ropes for indoor climbing (e.g., rock climbing). Climbers must make contact with the climbing rope with their hands and feet (to climb upwards), as well as with the initial and final climbing ropes that define the route (the route is a predefined area, path, and / or set of climbing ropes that can be used for climbing, and is typically given a level corresponding to its difficulty). Sensor devices installed on the user's hands, feet, and other body parts communicate with an NFC chip placed in or near the climbing rope. Information collected via the sensor devices is processed in the sensor devices and / or cloud computing systems to provide the user with a better understanding of the activity. Detailed implementations and examples are described in Section 4.a.

[0092] Pre-existing wireless devices can be used to detect user activity. Modern cars are often equipped with Bluetooth (BT) technology. BT-enabled sensor devices can pair with the car via the BT communication protocol. Once the monitoring device and the car are paired, entering the car will trigger synchronization between them, and the car will be able to transmit status and information about the user's activity (e.g., driving at x mph for n hours).

[0093] Triggers for entering exercise intensity mode

[0094] Manual trigger

[0095] Similar to activity type modes and device placement modes, motion intensity modes can also be triggered through user interaction with the tracking device (e.g., touching, pushing a button, performing a gesture, etc.) or with an assistive device (e.g., selected in a smartphone application).

[0096] Automatic trigger

[0097] In some embodiments, the sensor outputs of the tracking device or BMD contain a detectable motion intensity signature. This motion intensity signature can be detected by the BMD and trigger the device to enter various motion intensity modes. Combinations of sensor outputs can be used. The input to the triggering algorithm can come directly or indirectly from the sensor outputs. For example, the input can be a direct output from an accelerometer, or it can be a processed accelerometer output, such as the "sleep state" described below.

[0098] As explained above, certain activity characteristics are correlated with different levels of motion intensity detected by the motion sensors of the user's BMD. Under certain conditions, the user engages in movement activities, but the limb wearing the BMD exhibits reduced motion or limited acceleration compared to normal movement activities where the limb moves freely. For example, a user might run on a treadmill while holding a bar, walk while pushing a shopping cart, or walk while carrying a heavy object. Under these conditions, the motion intensity detected by the motion sensors may be significantly reduced. This is due to... Figure 4A The data shown in B is used to illustrate this. Figure 4A The image displays acceleration data in the time domain (top) and frequency domain (bottom) for users' stationary, walking, and running activities. Figure 4B The data presented includes similar data for running while stationary, with hands on a bar, and running with hands free. Figure 4A The image above shows that running produces a higher acceleration signal than walking, and walking produces a higher acceleration signal than being at rest. Figure 4B The above figure shows the highest signal strength level generated by running with hands free, which is higher than running with hands on a pole, and higher than running while stationary. It is worth noting that the acceleration signal generated by running with hands on a pole is more irregular and noisier than walking. At this lower signal level and / or higher noise level during running with hands on a pole, it is difficult to obtain step counts using peak detection analysis of time-domain data. Under certain conditions, BMD automatically analyzes the motion signals provided by the motion sensor and automatically switches motion intensity modes, employing different data processing algorithms to process the motion data.

[0099] In one embodiment, the device can use motion sensor signal strength to determine the mode of the device. The motion sensor signal strength can be determined, for example, by signal-to-noise ratio, signal norm (e.g., L1, L2, etc.), signal energy / power in certain frequency bands, wavelet scaling parameters, and / or the number of samples exceeding one or more thresholds. In some embodiments, accelerometer output power is used to determine different motion intensity modes, wherein the power is calculated as the sum of accelerometer amplitude values ​​(or amplitude squared values). In some embodiments, motion intensity can be determined using data from one, two, or three axes of one or more motion sensors. In some embodiments, data from one axis is used for further analysis when the signal is relatively high, and data from two or more axes is used for further analysis when the signal is relatively low.

[0100] When the activity level falls within a certain range, an activity intensity mode can be activated. In the case of a pedometer sensor device, there can be three different activity level ranges corresponding to three modes: active mode, semi-active mode, and inactive mode. The algorithm for determining the modes and the transitions between modes are further discussed below, which enables step counting and subsequent measurement of the user's biometric signals in a continuous manner. It should be noted that the three-mode method described herein is for illustrative purposes and does not limit the invention. There can be fewer modes (e.g., in a two-mode system, active and inactive (e.g., in a car)) or more than three modes. In practice, the number of modes can vary depending on the user and the typical activities the user performs. The number of modes can also be dynamically changed for each user depending on the likelihood of the user engaging in certain activities. For example, when GPS is detected being used by the user, a highly active mode can be deactivated.

[0101] In some embodiments, in addition to or instead of real-time or near-real-time motion sensor data, previously processed and / or stored sensor information can be used to determine motion intensity patterns. In some embodiments, such prior information may include motion information recorded at fixed time intervals (e.g., once per minute) over a previous period (e.g., 7 days). In some embodiments, prior information includes one or more of the following: sleep scores (wakefulness, sleep, restlessness, etc.), calories burned, number of stairs climbed, number of steps taken, etc. Machine learning can be used to detect behavioral signatures from prior information, which can then be used to predict the likelihood that a subject will have certain activity levels at the current time. Some embodiments use one or more classifiers or other algorithms to combine inputs from multiple sources (e.g., accelerometer power and data recorded per minute) and determine the probability that a user will engage in activities with certain characteristics. For example, if a user tends to work at a desk at 3 p.m. but goes shopping at 6 p.m., prior motion-related data will show a data pattern reflecting the user's trend, which can be used by a BMD in a classifier to determine that the user is likely to be pushing a shopping cart at 6:15 p.m. today.

[0102] In some embodiments, clustering algorithms (e.g., k-means clustering, nearest neighbor clustering, and expectation maximization) can be applied to the classified patterns based on prior knowledge that the user may perform each activity (e.g., driving) over a continuous time period.

[0103] In some embodiments, the exercise intensity mode can be automatically or semi-automatically selected using short-range wireless communication as described above for automatic selection of activity type mode and device placement mode.

[0104] Differences in sensor data processing - activity type mode, device placement mode, and monitoring

[0105] Users perform many types of activities throughout the day. However, sensor devices do not need to be optimized for all activities. Knowing a user's activities at a given time allows the sensor device to run one or more algorithms optimized for each specific activity. These activity-specific algorithms produce more accurate data. According to some embodiments, different data processing algorithms can be applied in each activity mode to improve the accuracy of activity measurements and provide activity-specific biometric data.

[0106] Users may wear the BMD in different positions. The device placement mode can be set manually or automatically, as described above. In each placement mode, a placement-specific algorithm is run to more accurately estimate the biometric data of interest. Variations of the placement-specific algorithm may be adaptive motion signal intensity thresholds, whose values ​​change according to the expected movement of the body part. Adaptive filtering techniques can be used to eliminate excessive movement of the body part using the prior placement mode. Pattern recognition techniques, such as support vector machines or Fisher discriminant analysis, can also be used to obtain a placement-specific classifier that will determine whether a signal or a signature of a signal represents the biometric data of interest.

[0107] Differences in sensor data processing - Motion intensity mode

[0108] Time Domain Analysis

[0109] In some embodiments, BMD applies algorithms that process data in the time domain. This is particularly useful when the data readily identifies the basic units of physiological activity in the time domain. This is typically used for data with high signal-to-noise ratios (SNR). In some embodiments, the time-domain analysis includes peak detection of motion amplitude data (e.g., acceleration). Returning to the discussion above and in Figure 4A As shown in the example data in the upper figure of B, the motion signal or SNR is high under conditions where the user is speaking or running with their hands free. Under these conditions, BMD uses time-domain analysis according to some embodiments.

[0110] In many embodiments, time-domain analysis is more time- and energy-efficient than frequency-domain analysis, which is suitable for data with insufficient signal or SNR. Peak detection of motion data typically requires analyzing a smaller amount of data compared to frequency analysis, thus having lower data volume and analysis requirements. In various embodiments, peak detection can be performed using data collected over durations on the order of seconds. In some embodiments, the data duration ranges from approximately 0.5 to 120 seconds or 1 to 60 seconds, 2 to 30 seconds, or 2 to 10 seconds. In contrast, in some embodiments, frequency analysis can use data with a longer duration than that used in peak detection.

[0111] In one embodiment, time-domain analysis can be applied to data with relatively low signal or SNR to identify features associated with the periodicity and / or cycle of the buffered motion sensor signal. These analyses may include, but are not limited to, autoregression analysis, linear predictive analysis, autoregressive moving average analysis, and auto / partial correlation analysis. One or more threshold rules and conditional decision rules are then applied to the coefficients of the features and / or analyses to detect the estimated periodicity, subsequently for the user's biometric data.

[0112] Frequency domain analysis

[0113] In some embodiments, when time-domain sensor data does not readily identify the basic units of physiological activity, algorithms operating in the frequency domain are used. Problems often arise because periodic signals have relatively low amplitudes, and peak detection algorithms may not be reliable enough. One example is step counting when the tracking device is on the user's wrist while the user is pushing a stroller or shopping cart. Another example is step counting when the user is on a treadmill or cycling. Yet another example is step counting when the user is in a car. In this case, frequency domain analysis helps us avoid counting steps that are attributable to movement caused by vibrations in the car (e.g., when the car is bumpy). A third example is when a user is walking while carrying a heavy object with the limb wearing the BMD.

[0114] Referring to the discussion above and in Figure 4B The example data shown in the above figure indicates that the acceleration signal or SNR is small when the user runs while holding onto the pole. Peak detection is difficult to use for the data shown in the figure because the data is noisy and peaks are unreliable. However, the frequency components show that for the two subplots of running while holding onto the pole, the two spectral peaks are... Figure 4B The values ​​in the figure below are located at approximately 65 Hz and 130 Hz. Under these conditions, according to some embodiments, BMD uses frequency domain analysis.

[0115] As mentioned above, frequency analysis can use data with a duration longer than that used in peak detection. In some embodiments, the duration of the data used for frequency analysis ranges from approximately seconds to minutes. In some embodiments, the range is approximately 1 second to 60 minutes, 2 seconds to 30 minutes, 4 seconds to 10 minutes, 10 seconds to 5 minutes, 20 seconds to 2 minutes, or 30 seconds to 1 minute.

[0116] In some embodiments, the length of the buffered motion signal may be set depending on the desired resolution of the classification. A motion intensity mode selection algorithm is applied to each segment of this buffered motion signal to return a classification mode (e.g., semi-active and driving modes) and a step (pace) count. Post-processing may then be applied to these resulting values ​​in the processing circuitry of the sensor device and / or remote processing circuitry (e.g., a cloud server). In one embodiment, a simple filter may be applied to the estimated step (pace) count to remove abrupt changes in the step (pace) count. In another, a clustering algorithm (e.g., k-means clustering, nearest neighbor clustering, and expectation maximization) may be applied to the classification mode based on prior knowledge that the user may be performing each activity (e.g., driving) over a continuous time period. These updated modes from the clustering are then used to update the step (pace) count of a given buffered motion signal.

[0117] In some embodiments, the BMD's motion intensity mode may include an active mode, a semi-active mode, and an inactive mode. In the active mode, the motion sensor of the sensor device detects acceleration, displacement, changes in altitude (e.g., using a pressure sensor), and / or rotation, which can be converted into steps using a peak detection algorithm. In the inactive mode, the user is seated (e.g., sitting still), and the pedometer (via the motion sensor) does not measure any signal with a walking signature. In this case, no further calculations are performed to detect steps. In the semi-active mode, the motion sensor observes some user movement, but the motion signal does not possess a sufficiently strong walking signature (e.g., a series of high-amplitude peaks in the motion sensor signal generated by walking) to accurately detect walking using a peak detection algorithm.

[0118] In semi-active mode, time-domain and / or frequency-domain analysis can be performed on a buffered motion signal of a certain length to identify features associated with periodic movements such as walking. If any periodicity or feature indicating the periodicity of the buffered motion signal is identified, the period is estimated and then interpreted as the user's biometric data, such as the average cadence of the buffered motion signal.

[0119] Frequency domain analysis can include different elements than, for example, Figure 4A and 4B The techniques described herein only utilize FFT or spectrograms. For example, one approach may involve first bandpass processing the time-domain signal and then running a peak counter in the time domain. Other methods may be used to process the data using frequency analysis, and the processed data may then be further processed to obtain the periodicity or peaks of the signal.

[0120] In some embodiments, frequency domain transformation / analysis can be performed on the buffered motion signal using techniques including, but not limited to, Fourier transform (e.g., Fast Fourier Transform (FFT)), cepstral transform, wavelet transform, filter bank analysis, power spectral density analysis, and / or periodogram analysis. In one embodiment, a peak detection algorithm in the frequency domain can be performed to find spectral peaks that are a function of the average step frequency of the buffered motion signal. If no spectral peaks are found, the algorithm concludes that the user's movement is not associated with walking motion. If peaks or a set of peaks are found, the period of the buffered motion signal is estimated, thereby enabling the inference of biometric data. In another embodiment, a statistical hypothetical test, such as the Fisher periodicity test, is applied to determine whether the buffered motion signal possesses any periodicity, and subsequently to determine whether it possesses biometric information associated with the user's activity. In yet another embodiment, a harmonic structure is employed to test periodicity and / or estimate the period. For example, a generalized likelihood ratio test with a parametric model and the harmonicity of the buffered motion signal can be performed.

[0121] In another embodiment, a set of machine-learned coefficients can be applied to a subgroup of frequency and / or time-domain features obtained from the frequency and / or time-domain analysis described above. A linear / nonlinear mapping of the inner product of the coefficients with the subgroup of said spectral features then determines whether a given buffered motion signal originates from user motion involving some periodic movement. The machine learning algorithm classifies the motion signal into two categories: signals generated from walking, and signals generated from activities unrelated to walking.

[0122] For example, this semi-active mode algorithm can detect steps even when a user wears the sensor device on their wrist and holds the treadmill handle while walking on the treadmill. In cases where the buffered motion signal does not have the signature of walking motion, the buffered motion signal can be ignored and no steps are counted to eliminate the possibility of incorrect step counting. For example, the motion signal in the time domain of a user driving on an uneven road will exhibit a series of high-amplitude peaks, which have a signature similar to walking. Running a peak detection pedometer algorithm on the time-domain motion signal of driving on an uneven road will cause the pedometer to count steps when it should not. However, in the frequency domain and / or in a signal to which appropriate time-domain analysis is applied, the same motion signal of driving on an uneven road is unlikely to have a signature associated with walking motion (e.g., a periodic signature). When the signal in the frequency domain and / or in which time-domain analysis is applied does not have the signature of walking motion, steps are not counted because it can be assumed that the user is not actually walking or running.

[0123] Example - Exercise intensity mode for walking / running activities

[0124] Figure 5A This is a flowchart of a process 500 for tracking steps using a BMD according to some embodiments. The process automatically selects an exercise intensity mode and applies different data processing algorithms for different exercise intensity modes. The BMD has one or more sensors that provide data indicative of the user's physiological activity, including motion data indicative of step count. The BMD uses one or more motion sensors to sense the user's motion, which are further described below. See box 504. The BMD analyzes the motion data provided by the motion sensors to determine the exercise intensity caused by the user's activity. See box 506. In such a case... Figure 5BIn some embodiments described herein, the BMD defines three ranges of exercise intensity associated with active mode, semi-active mode, and inactive mode: high, moderate, and low. In some implementations, active mode corresponds to running or walking with free hand movement; semi-active mode corresponds to running or walking on a treadmill while holding a fixed handle, typing at a table, or driving on an uneven road; and inactive mode corresponds to the user being stationary.

[0125] As stated above, some embodiments may use more or fewer than three motion ranges corresponding to more or fewer than three modes. The specific range for different modes may differ for different applications or different users. In some embodiments, the specific range may be supplied through offline prior knowledge. In some embodiments, the specific range may be influenced by a machine learning process that selects the range with optimal speed and accuracy for step counting.

[0126] In process 500, if the BMD determines that the user is engaged in an activity that allows the motion sensor to measure high motion intensity, the BMD may enter an active motion intensity mode. See box 508. In some embodiments, in addition to current motion data, the BMD may use other forms of motion-related data in its analysis to determine the motion intensity mode. For example, in some embodiments, the BMD may receive previously processed and / or stored data. Such data may include sleep quality, steps, calories burned, number of stairs climbed, altitude, or distance traveled, as described above. In some embodiments, the previous data is recorded at fixed intervals, such as every minute, every 10 minutes, every hour, etc. The BMD may use one or more classifiers to combine the current motion intensity signal with the previous motion-related data to determine that the user may be engaged in an activity that generates a high motion intensity signal, which triggers the BMD to enter an active mode as the motion intensity mode. The BMD then applies a peak detection algorithm to analyze the motion data. See box 514. The detected peaks and associated time information provide data for calculating steps.

[0127] In some embodiments, the BMD may determine that the motion intensity from the motion sensor data is moderate (as described above) and then trigger the BMD to enter a semi-active mode. See box 510. The range of motion intensity used to define the semi-active mode may be lower than the active mode and higher than the inactive mode. In some embodiments, the BMD applies frequency domain analysis and / or time domain analysis to detect periodicity in the motion data. See box 516. In some embodiments, the BMD applies FFP to obtain frequency information of the motion signal. Other frequency domain and time domain analyses described above may also be applied here. Using the information derived from the frequency domain or time domain analysis, the BMD determines whether the data contains periodic information. See box 518. If so, the BMD infers that the motion data was generated by a user engaged in some other activity involving walking or running on a treadmill or periodic movement of the limb wearing the BMD (e.g., typing at a table). See box 520. In some embodiments, the BMD may further apply one or more filters or classifiers to determine whether the periodic information relates to walking motion, as further described below. If so, BMD uses the periodicity information to calculate the number of steps; for example, a 1 Hz periodic motion lasting 10 seconds corresponds to a pace of 60 steps / minute and 6 steps. See box 524. If DMD determines that there is no periodicity information in the motion data, it infers that the user participated in an activity with regular motion, such as driving on an uneven road. See box 522. In some embodiments, BMD may ignore any steps that may have accumulated during the corresponding period for other reasons (e.g., steps from the start of the time-domain analysis).

[0128] When the motion intensity level is low, the BMD may enter an inactive mode. See box 512. The inactive mode may correspond to the user being stationary. In some embodiments, when the BMD is in inactive mode, it does not perform further processing on the motion data.

[0129] Figure 5BA flowchart illustrating process 530 for enabling the BMD to automatically select a mode for different user activity conditions, according to some embodiments. Different modes are then applied with different analyses to obtain step counts. Process 530 may be implemented as a subprocess of process 500. Process 530 for switching modes uses motion intensity detected by a motion sensor and previously analyzed and / or recorded motion-related information. In the embodiments shown here, the previous information is processed by a sleep algorithm. Process 530 begins by buffering samples of motion data. The amount of data buffered may vary depending on the application and conditions. In the process shown here, current motion data is buffered to determine whether the device should enter one of the motion intensity modes. This data used to trigger different motion intensity modes may be the same as or different from the data used to analyze step counts in different modes. The durations of these two data sets may also be the same or different. In some embodiments, the BMD continuously buffers data samples to determine whether to select, maintain, and / or change a motion intensity mode. The process proceeds to calculating the power of a signal from the buffered samples. In some embodiments, the calculation is based on the l1 norm, i.e., the sum of the absolute values ​​of the signal. See box 534.

[0130] Process 530 continues by determining whether the power of the signal is greater than an empirically determined threshold σ, as shown in box 536. In some embodiments, the threshold may be trained using a machine learning algorithm to improve the algorithm for selecting different modes, the machine learning training allowing the BMD to obtain accurate step counts efficiently. In some embodiments, the empirically determined threshold may be adjusted by the user or by knowledge based on other users. If the process determines that the power of the signal is greater than the empirical threshold σ, the BMD is triggered to enter an active mode. See box 538. The BMD then performs step count analysis in a manner similar to that of a classic pedometer as described above using a peak detection method. See box 540. If the process determines that the signal power is not greater than the empirical threshold σ, in some embodiments, it uses a sleep algorithm to further analyze whether it should enter a moderate or inactive mode. In some embodiments, the sleep algorithm analyzes previous motion-related information to determine whether the user may be sleeping, awake, or moving while awake. In some embodiments, previous motion-related information may be information derived from motion, such as step count, number of stairs climbed, etc., as further described herein. In some embodiments, if the sleep algorithm determines that the user may be sleeping, it enters an inactive mode. See box 548. In some embodiments, the BMD in inactive mode does not perform further analysis of the sensor signals, which helps conserve the BMD's battery power. See box 550. However, if the sleep algorithm determines that the user is not asleep, the BMD enters a moderate motion intensity mode. See box 544. The BMD performs FFT analysis of the motion data in the frequency domain to determine the number of steps. Some examples of applicable frequency analysis are further described below.

[0131] In some embodiments, BMD can be used Figure 6A The process 610 shown implements peak detection operation 514 in active mode. The process of implementing peak detection to calculate steps in process 610 begins by obtaining a new sample of motion data, such as acceleration data. In some embodiments, the sample is a digitized value recorded by a sensor that is approximately linear with the analog signal to be measured. In some embodiments, the analog signal is acceleration (e.g., m / s²). 2 The duration of the sample can be selected based on the different considerations described above. In some embodiments, the new sample contains acceleration data for a duration of about 0.5 to 120 seconds, or 1 to 60 seconds, 2 to 30 seconds, or 2 to 10 seconds.

[0132] The process then proceeds to perform peak detection analysis. See box 614. Figure 6B This demonstrates a peak detection process, implemented in block 614, according to some embodiments. The process begins by waiting for data to fill the data buffer described above. See block 650. Next, the process involves finding the global maximum value of the buffered data. See block 652. As shown, to the left of block 652, some embodiments may apply a rolling time window of duration N, which can be selected as described above. The start and end times of the rolling time window can be specified as t and t+N, as shown. The process searches for the global maximum value of the data within the rolling window. After calculating the global maximum value, the process determines whether the global maximum value is greater than an empirically determined threshold θ. See block 654. If the global maximum value is not greater than the empirical threshold, the process resumes waiting for new data to fill the buffer, as shown in operation 650. If the global maximum value is greater than the threshold, the process further determines whether the global maximum value occurs at or near the center of the rolling time window. If the maximum value is not in or near the center of the time window, the process determines that the peak may not be a walk, and therefore the process reverts to waiting for new data to fill the buffer, as in operation 650. If the peak is centered on the buffered time window, the process determines whether the peak was detected at or near t+N / 2.

[0133] Alternative procedures can be applied for peak detection analysis, which involve calculating the first derivative and identifying any first derivative with a downward zero-crossing as the peak maximum. Additional filters can be applied to remove noise from the detected peaks. For example, the presence of random noise in a real experimental signal would simply be attributed to noise, causing many erroneous zero-crossings. To avoid this problem, one embodiment may first smooth the first derivative of the signal, then look for downward zero-crossings, and then only obtain those zero-crossings whose slope exceeds a predetermined minimum (i.e., a "slope threshold") at points where the initial signal exceeds a certain minimum (i.e., an "amplitude threshold"). Adjustments to the smoothing width, slope threshold, and amplitude threshold can significantly improve peak detection results. In some embodiments, alternative methods can be used to detect peaks. Procedure 610 then proceeds to analyze whether the peak is associated with walking. See box 160. This analysis can be performed by applying one or more classifiers or models. If the analysis determines that the peak is not associated with walking, the procedure returns to obtaining new samples, as shown in box 612. If the analysis determines that the peak is associated with walking, the procedure increments the step count by 1. See box 618. Next, the step counting process returns to the step acquisition step shown in box 612. The step counting process continues in the same manner.

[0134] In some embodiments, BMD can be used Figure 6C The process 620 shown is performed in a semi-active mode for data processing. Process 620 begins by acquiring N new samples of motion data, such as acceleration data. These N new samples in block 622 typically contain more data than the samples in block 612 of process 610 for peak detection. In some embodiments, the samples contain several minutes of data. The necessary amount of data depends on various factors as described above and may include various amounts in various embodiments. N depends on the data duration and sampling rate and is limited by the memory budget used for step analysis. Process 620 proceeds to perform spectral analysis. See block 624. In some embodiments, spectral analysis is performed using a Fourier transform (e.g., FFT) to show the power at various frequencies. Any peaks in the frequency domain indicate the presence of periodicity in the motion data. For example, a peak at 2 Hz indicates periodic movement of 120 times / minute. Process 620 then continues by checking whether the spectral peaks correspond to walking. See block 626. This can be done by applying one or more filters or classifiers. If the analysis determines that the spectral peak does not correspond to walking, the process returns to box 622 to obtain N new samples. If the analysis determines that the spectral peak is actually related to walking, the process increases the step count by M, where M is determined from the frequency of the spectral peak and the duration of the data. For example, if the spectral peak occurs at 2 Hz and the N samples last for 60 seconds, then M will be 120 steps. In some embodiments, the harmonics of the maximum peak are also analyzed to assist in determining the step count.

[0135] Figure 6D Details of a process for implementing a spectrum analysis suitable for operation 624, according to some embodiments, are shown. The process begins by applying a Hanning window of duration period N, which prepares the data for Fourier transform. See box 660. Next, in some embodiments, the process performs a Fast Fourier Transform (FFT). See box 662. The FFT converts time-domain information into frequency-domain information, thereby revealing the power at various frequencies. The process then applies a peak detection algorithm in the frequency domain to determine whether any peaks exist at specific frequencies. Peak detection algorithms similar to those described above can be applied to the frequency-domain data. If one or more peaks are detected for a specific frequency, the process infers that the data contains a periodic component, which is used to calculate the step count. For example, if the spectral peaks occur at 1 Hz and N samples last for 30 seconds, the process determines that 30 steps have occurred during the activity providing the data.

[0136] Example - Rock Climbing Activity Type Pattern

[0137] In some embodiments, NFC or other short-range wireless communications such as Bluetooth, ZigBee, and / or ANT+ are used in the climbing setup. The climber makes contact with the climbing rope and / or climbing wall features with his hands and feet to climb upwards, including an initial climbing rope and a final climbing rope that define the route (a predefined area, path, and / or set of climbing ropes that can be used for climbing, and is usually given a level corresponding to its difficulty). In one embodiment, an active or passive NFC-enabled device or tag is installed at locations including, but not limited to, one or more climbing ropes or carabiners used for climbing routes to communicate with an active or passive NFC-enabled chip or device embedded in, on, or near: the user's hands, gloves, wristbands, feet, shoes, other body parts, wearable clothing, pockets, belts, belt loops, corsets, shirt sleeves, shirt collars, shoes, shoelaces, hats, bras, ties, socks, underwear, coin purses, gloves, other clothing, accessories such as purses, backpacks, waist bags, goggles, swimming caps, glasses, sunglasses, necklaces, pendants, brooches, hair accessories, bracelets, armbands, anklets, rings, toe rings, and earrings. Information collected by devices on the climber and / or on the climbing ropes or walls is processed in devices on the climber and / or on the climbing ropes or walls and / or in a cloud computing system to provide data to the user and / or the climbing gym related to the user's climb.

[0138] In one embodiment, this data can be used to help users track which climbs they have completed and / or attempted. The data can also be used by climbers to remember which climbing ropes and / or wall features they used and the order in which they used them. This data can be shared with other climbers to assist them in completing parts or the entire climbing route, competing, winning badges, and / or winning other virtual rewards. In some cases, climbers may receive data only from climbers with similar characteristics (including but not limited to height, weight, experience level (e.g., years of climbing), strength, confidence or concerns about height and / or flexibility) to improve the relevance of the data when assisting them in completing the climbing route. In some cases, optional climbing ropes may be physically added to or removed from the actual route to reduce or increase its difficulty. After completing the route, climbers can immediately share their achievements virtually on online social networks. Virtual badges can also be awarded for achieving certain climbing achievements, such as completing or attempting a climb of a specific difficulty or a certain number of climbs.

[0139] In another embodiment, the climber may wear a device that can detect freefall using motion sensors such as accelerometers. Freefall detection data can be wirelessly transmitted to assistive devices such as smartphones, tablets, laptops, desktop computers, or servers. In one embodiment, freefall detection can cause an automatic braking device to prevent the rope stabilizing the climber from falling further. This automatic braking device may be used in addition to or as an alternative to automated mechanical fall stop mechanisms and / or manually operated fall stop mechanisms (such as bollards).

[0140] Freefall data can also be used to determine when the rope needs to be untied and no longer in use. Measures including, but not limited to, the number of freefall events, the duration of the freefall, the maximum acceleration, the maximum force (estimated using the climber's weight), and / or the energy consumed by the rope can be used to calculate when the rope should be untied. This data can also be presented to the user.

[0141] Freefall data can also be used to determine when climbers and / or belayers are climbing unsafely. For example, if a climber falls to a certain value (as determined by one or more freefall metrics already disclosed in this article), it can alert climbing gym staff.

[0142] In another embodiment, the climbing ropes and / or features may have embedded proximal auditory and / or visual indicators. These indicators may be used instead of colored or patterned strips (which are commonly used to indicate which climbing rope and / or feature can be used during climbing). These indicators may also show which climbing ropes and the order in which they were used by the user, one or more other users, or one or more other users with similar features disclosed herein, during a previous climb.

[0143] In another embodiment, a weight sensor integrated into the climbing rope and / or feature can determine which climbing ropes and / or features were used during the climb. The order of climbing ropes and / or wall features can also be determined by a separate device communicating with the climbing rope, which has weight sensor functionality.

[0144] Climbing ropes and / or wall features can also be used to determine which climbing ropes and / or wall features were used by the feet, hands, and / or other body parts. In one embodiment, it can also determine which hand or foot (e.g., left or right) was used on which climbing rope.

[0145] In one embodiment, the visual characteristics of the climbing rope or wall feature (e.g., color, brightness, number of emitting LEDs) can be altered in response to use by a climber. This can be achieved, for example, by using RGB LEDs mounted within the translucent climbing rope and / or wall feature. Visual indicators can also be located near the climbing rope or wall feature rather than being directly integrated therein.

[0146] Biometric monitoring device

[0147] A BMD (Browser Analyzer) is required that provides accurate analysis of measurements under varying conditions while maintaining overall analysis speed and energy efficiency. In some embodiments, accuracy, speed, and efficiency can be achieved by employing multiple modes that process sensor output data in different ways. In some embodiments, the BMD can switch modes via automatic triggering as described above.

[0148] In some implementations, the BMD may be designed to be inserted into and removed from multiple compatible housings / shells / holders (e.g., a wristband worn on an individual's forearm or a belt clip housing attached to an individual's clothing). In some embodiments, the biometric monitoring system may also include other devices or components communicatively linked to the biometric monitoring device. The communication link may involve direct or indirect connections, as well as wired and wireless connections. Components of the system may communicate with each other via wireless connections (e.g., Bluetooth) or wired connections (e.g., USB). Indirect communication refers to the transmission of data between the first device and auxiliary devices by means of one or more intermediate third devices relaying data.

[0149] Figure 7The illustration depicts a generalized schematic diagram of an example portable biometric monitoring device (also referred to herein as a "biometric monitoring device") or other devices by which the various operations described herein can be performed. The portable biometric monitoring device 702 may include a processing unit 706 having one or more processors, a memory 708, a user interface 704, one or more biometric sensors 710, and an input / output 712. The processing unit 706, memory 708, user interface 704, one or more biometric sensors 710, and the input / output interface 712 may be communicatively connected via a communication path 714. It should be understood that some of these components may also be indirectly connected to each other. In some embodiments, Figure 7 The components can be implemented as external components that are communicatively linked to other internal components. For example, in one embodiment, memory 708 can be implemented as memory on an auxiliary device such as a computer or smartphone that communicates with the device wirelessly or via a wired connection through I / O interface 712. In another embodiment, the user interface may include components such as buttons on the device and components on an auxiliary device that is communicatively linked to the device via I / O interface 712, such as a touchscreen on a smartphone.

[0150] Portable biometric monitoring devices can collect one or more types of biometric data from one or more sensors 710 and / or external devices (e.g., external blood pressure monitors), such as data on physical characteristics of the human body (e.g., steps, heart rate, sweating levels, etc.) and / or data related to the physical interaction between the body and the environment (e.g., accelerometer readings, gyroscope readings, etc.). In some embodiments, the device stores the collected information in memory 708 for later use, such as for transmission to another device (e.g., a smartphone) via I / O interface 712 or to a server via a wide area network such as the Internet.

[0151] As used herein, biometric information refers to information relating to the measurement and analysis of the physical or behavioral characteristics of a human or animal subject. Some biometric information describes the relationship between the subject and the external environment (e.g., the subject's altitude or route). Other biometric information describes the subject's physical condition independent of the external environment, such as the subject's step count or heart rate. Information about the subject is generally referred to as biometric information. Similarly, sensors used to collect biometric information are referred to herein as biometric sensors. In contrast, information about the external environment independent of the subject's condition is referred to herein as environmental information, and sensors used to collect such information are referred to herein as environmental sensors. It is noteworthy that sometimes the same sensor can be used to obtain both biometric and environmental information. For example, a user-worn light sensor can act as part of a photoplethysmography (PPG) sensor, which collects biometric information based on the reflection of light (which may originate from a light source in the device configured to illuminate a portion of the person reflecting the light) from the subject. The same light sensor can also collect information about ambient light when the device does not illuminate a portion of the person. In this invention, the distinction between biometric and non-biometric information and the sensors are only used for organizational purposes. This distinction is not essential to the present invention unless otherwise specified.

[0152] Processing unit 706 can also perform analysis on the stored data and may initiate various actions depending on the analysis. For example, processing unit 706 may determine that data stored in memory 708 indicates that a target number of steps or pace has been reached and may then display content on the portable BMD's display to celebrate the achievement of the target. The display may be part of user interface 704 (e.g., undelineated buttons or other controls used to control functional aspects of the portable biometric monitoring device). In some embodiments, user interface 704 includes components in or on the device. In some embodiments, user interface 704 also includes components external to the device but still communicatively linked to the device. For example, a smartphone or computer communicatively linked to the BMD may provide the user interface component, through which the user can interact with the BMD.

[0153] Generally, a BMD can incorporate one or more types of user interfaces, including but not limited to visual, auditory, tactile / vibration, or combinations thereof. The BMD can display information relating to one or more types of data that can be used and / or tracked by the biometric monitoring device, for example, via a graphic display or via the intensity and / or color of one or more LEDs. The user interface can also be used to display data from other devices or Internet sources. The device can also provide tactile feedback via, for example, the vibration of a motor or changes in the texture or shape of the device. In some embodiments, the biometric sensor itself can be used as part of the user interface; for example, an accelerometer sensor can be used to detect when a person touches the housing of the biometric detection unit with a finger or other object, and such data can then be interpreted as user input for the purpose of controlling the biometric monitoring device.

[0154] A biometric monitoring device may include one or more mechanisms for interacting with the device locally or remotely. In one embodiment, the biometric monitoring device may visually convey data via a digital display. The physical embodiment of this display may use any one or more display technologies, including, but not limited to, LED, LCD, AMOLED, E-Ink, high-resolution display technology, graphic displays, and other display technologies (e.g., TN, HTN, STN, FSTN, TFT, IPS, and OLET). The display may show data acquired or stored locally on the device, or data acquired remotely from other devices or internet services. The device may use sensors (e.g., ambient light sensors, "ALS") to control or adjust the screen backlight. For example, in dark lighting conditions, the display may dim to conserve battery life, while in bright lighting conditions, the display may increase its brightness to make it easier for the user to read.

[0155] In another embodiment, the device may use monochrome or multicolor LEDs to indicate the device's status. The status indicated by the device may include, but is not limited to, biometric statuses such as heart rate or application statuses such as incoming messages or goal achieved. These statuses may be indicated by the LED's color, on / off state, intermediate intensity, pulsation (and / or its rate), and / or light intensity pattern (from completely off to maximum brightness). In one embodiment, the LED may adjust its intensity and / or color according to the user's pace or number of steps.

[0156] In one embodiment, using an E-Ink display allows the display to remain on without draining the battery of a non-reflective display. This "always-on" functionality provides a pleasant user experience, for example, in applications where the user can simply glance at the device to see the time. The E-Ink display always shows content regardless of the device's battery life, allowing the user to see the time as if it were on a traditional watch.

[0157] The device can display a user's step count or heart rate using light, such as LEDs, by adjusting the amplitude of light emitted at a frequency corresponding to the user's step count or heart rate. The device can be integrated into or incorporated into another device or structure (such as eyeglasses or goggles), or communicate with eyeglasses or goggles to display this information to the user.

[0158] Biometric monitoring devices can also convey information to users through the physical movement of the device. One such embodiment of a method for physically moving the device is the use of a vibration-induced motor. The device can be used alone or in combination with multiple motion-induced techniques.

[0159] The device can convey information to the user via audio. Speakers can convey information using audio tone, voice, songs, or other sounds.

[0160] In another embodiment, the biometric monitoring device can transmit and receive data and / or commands to and from an auxiliary electronic device. The auxiliary electronic device can communicate directly or indirectly with the biometric monitoring device. Direct communication herein refers to data transmission between a first device and an auxiliary device without any intermediary device. For example, the two devices can communicate with each other via a wireless connection (e.g., Bluetooth) or a wired connection (e.g., USB). Indirect communication refers to data transmission between the first device and the auxiliary device by means of one or more intermediate third devices that relay data. The third devices may include, but are not limited to, wireless repeaters (e.g., WiFi repeaters), computing devices such as smartphones, laptops, desktop or tablet computers, cellular phone towers, computer servers, and other networked electronic devices. For example, the biometric device can send data to a smartphone, which forwards the data to a server connected to a cellular network via the Internet.

[0161] In one embodiment, the auxiliary device serving as the user interface to the biometric monitoring device may be a smartphone. An application on the smartphone may facilitate and / or enable the smartphone to serve as the user interface to the biometric monitoring device. The biometric monitoring device may send biometric and other data to the smartphone in real time or with a slight delay. The smartphone may send one or more commands to the biometric device to, for example, instruct it to send biometric and other data in real time or with a slight delay.

[0162] The smartphone may have one or more applications that allow users to view data from their biometrics. These applications may open by default to a "dashboard" page when the user launches or opens the application. This page may display a summary of total data such as total steps, floors climbed, miles traveled, calories burned, calories consumed, and water consumed. Other relevant information may also be displayed, such as the last time the application received data from the biometrics monitor, metrics about the previous night's sleep (e.g., when the user fell asleep, woke up, and the duration of their sleep), and how many calories the user can consume that day to maintain their calorie goal (e.g., achieving a calorie deficit goal for weight loss). Users may be able to choose which of these and other metrics are displayed on the dashboard screen. Users may be able to see these and other metrics from previous days on the dashboard. Access to previous days may be possible by pressing buttons or icons on the touchscreen. Alternatively, gestures such as swiping left or right may allow users to navigate through current and previous metrics.

[0163] Biometric monitoring devices can be configured to communicate with users via one or more feedback mechanisms or a combination thereof, such as vibration feedback, audio output, or graphics output via a display or light-emitting device (e.g., LED).

[0164] In one example, while a user wears the biometric monitoring device 702, the biometric monitoring device 702 can measure and store the user's step count or heart rate while the user is wearing the biometric monitoring device 702, and then subsequently transmit the data representing the step count or heart rate to the user's account on a network service (such as fitbit.com), to a mobile computing device (such as a telephone) paired with the portable biometric monitoring unit, and / or to a stand-alone computer that can be stored, processed, and observed by the user. Such data transmission can be performed via communication through I / O interface 712. The device can measure, calculate, or use multiple physiological metrics, including but not limited to step count, heart rate, calorie expenditure, number of floors climbed or descended, location and / or orientation (e.g., via GPS), altitude, walking speed and / or distance traveled, swim lap count, cycling distance and / or speed, blood pressure, blood glucose, skin conductance, skin and / or body temperature, electromyography data, electroencephalography data, weight, body fat, and respiratory rate. Some of this data can be provided to the biometric monitoring device from external sources. For example, a user can input their height, weight, and stride length into a user profile on a fitness tracking website, and then transmit this information to the biometric monitoring device via I / O interface 712 to combine with data measured by sensor 710 to assess the distance traveled or calories burned. The device can also measure or calculate metrics related to the user's surrounding environment, such as air pressure, weather conditions, light exposure, noise exposure, and magnetic fields.

[0165] As previously mentioned, biometric data collected from the biometric monitoring device can be transmitted to an external device via a communication or I / O interface 712. The I / O or communication interface may include wireless communication functionality, enabling the stored data to be automatically uploaded to an internet-visible source, such as a website, like fitbit.com, when the biometric monitoring device enters the range of a wireless base station or access point. Wireless communication functionality can be provided using one or more communication technologies known in this art, such as Bluetooth, RFID, Near Field Communication (NFC), Wi-Fi, Ant, optical data transmission, etc. The biometric monitoring device may also include wired communication capabilities, such as USB.

[0166] Other embodiments using short-range wireless communication are described in U.S. Patent Application No. 13 / 785,904, filed March 5, 2013, entitled “Near Field Communication System, and Method of Operating Same,” which is hereby incorporated herein by reference in its entirety.

[0167] It should be understood that Figure 7 This describes a generalized embodiment of biometric monitoring device 702 that can be used to implement a portable biometric monitoring device or other device capable of performing the various operations described herein. It should be understood that in some embodiments, in... Figure 7 The functionality represented in the middle can be provided in a distributed manner between, for example, external sensor devices (e.g., an external blood pressure monitor that can communicate with a biometric monitoring device) and communication devices.

[0168] Furthermore, it should be understood that, in addition to storing program code for execution by the processing unit to implement the various methods and techniques described herein, memory 708 may also store configuration data or other information used or configured during the execution of various programs or sets of instructions. Memory 708 may also store biometric data collected by the biometric monitoring device. In some embodiments, the memory may be distributed across more than one device, for example, across both the BMD and an external computer connected via I / O 712. In some embodiments, the memory may be exclusively located on an external device. Regarding memory architecture, for example, multiple different types of storage devices may be provided within memory 708 to store different types of data. For example, memory 708 may include non-volatile storage media (e.g., fixed or removable magnetic, optical, or semiconductor-based media) to store executable code and related data and / or volatile storage media (e.g., static or dynamic RAM) to store more temporary information and other variable data.

[0169] It should be further understood that the processing unit 706 may be implemented by a general-purpose or special-purpose processor (or processing core group), and therefore may execute a sequence of programmed instructions to perform various operations synchronously associated with the sensor device and to interact with users, system operators, or other system components. In some embodiments, the processing unit may be an application-specific integrated circuit (ASIC).

[0170] Although not shown, numerous other functional blocks may be provided as part of the biometric monitoring device 702, depending on the additional functions they may perform (e.g., environmental sensing functionality, etc.). Other functional blocks may provide wireless telephony operation for smartphones and / or wireless network access to mobile computing devices (e.g., smartphones, tablets, laptops, etc.). The functional blocks of the biometric monitoring device 702 are depicted coupled via a communication path 714, which may include any number of shared or dedicated buses or transmission links. However, more generally, the functional blocks shown may be interconnected using a variety of different architectures and implemented using a variety of different underlying technologies and architectures. The various methods and techniques disclosed herein may be implemented or programmed into programmable hardware devices (e.g., FPGAs (Field-Programmable Gate Arrays)) or within or outside the processing unit 706 by executing one or more sequences of instructions (e.g., software programs) via the processing unit 706 or custom hardware ASICs (Application-Specific Integrated Circuits).

[0171] Further embodiments of the portable biometric monitoring device can be found in U.S. Patent Application No. 13 / 156,304, filed June 8, 2011, entitled “Portable Biometric Monitoring Devices and Methods of Operating Same,” which is hereby incorporated herein by reference in its entirety.

[0172] In some embodiments, the biometric monitoring device may include computer-executable instructions for controlling one or more processors to acquire biometric data from one or more biometric sensors. The instructions may also control one or more processors to receive requests (e.g., inputs), specific patterns of biometric sensor data (e.g., double-click reading), etc., from buttons or touch interfaces on the biometric monitoring device to display aspects of the acquired biometric data on a display of the biometric monitoring device. These aspects may be quantities, graphics, or simply indicators (target process indicators, for example). In some embodiments, the display may be an illuminated display so that it is visible when displaying data but otherwise invisible to a casual observer. The instructions may also cause one or more processors to turn the display on from an off state to display aspects of the biometric data. The instructions may also cause the display to turn off from an on state after a predefined time period has elapsed without any user interaction with the biometric monitoring device; this may help save power.

[0173] In some embodiments, one or more components of 702 may be distributed across multiple devices, thereby forming a biometric monitoring system 702 spanning multiple devices. Such embodiments are also considered to be within the scope of the invention. For example, the user interface 704 on the first device may not have any mechanism for receiving physical input from the wearer, but the user interface 704 may be included in components on a second paired device (e.g., a smartphone) that communicates wirelessly with the first device. The user interface 704 on the smartphone allows the user to provide input to the first device, such as providing a username and current location. Similarly, in some embodiments, the biometric monitoring device may not have any display at all, i.e., it may not be able to directly display any biometric data. Biometric data from such a biometric monitoring device may instead be wirelessly transmitted to a paired electronic device, such as a smartphone, and such biometric data may then be displayed on a data display screen on the paired electronic device. Such implementations are also considered to be within the scope of the invention, namely, such pairing electronics can serve as components of a biometric monitoring system 702 configured to communicate with biometric sensors located inside or outside the pairing electronics (such biometric sensors may be located in separate modules worn elsewhere on the wearer).

[0174] Biometric sensors

[0175] In some embodiments, the biometric monitoring device discussed herein can collect one or more types of physiological and / or environmental data from sensors embedded within the biometric monitoring device (e.g., one or more sensors selected from the group including accelerometers, heart rate sensors, gyroscopes, altimeters, etc.) and / or external devices (e.g., external blood pressure monitors), and can transmit or relay this information to other devices (including devices capable of acting as Internet-accessible data sources), thus allowing the collected data to be viewed, for example, using a web browser or a web-based application. For example, when a user wears the biometric monitoring device, the device can use one or more sensors to calculate and store the user's step count. The device can then transmit data representing the user's step count to an account on a web service (e.g., fitbit.com), a computer, a mobile phone, or a health station where the user can store, process, and observe the data. In practice, the device can measure or calculate several other physiological metrics in addition to or in lieu of the user's step count or heart rate.

[0176] The measured physiological metrics may include, but are not limited to, energy expenditure, such as calories burned, number of floors climbed and / or descended, steps taken, heart rate, heart rate variability, heart rate recovery, location and / or orientation (e.g., via GPS), altitude, walking speed and / or distance traveled, number of swimming trips, cycling distance and / or speed, blood pressure, blood glucose, skin conductance, skin and / or body temperature, electromyography data, electroencephalography data, weight, body fat, calorie intake, nutrient intake from food, medication intake, sleep cycles, sleep stages, sleep quality and / or duration, pH level, hydration level, and respiratory rate. The device may also measure or calculate metrics related to the user's surrounding environment, such as atmospheric pressure, weather conditions (e.g., temperature, humidity, pollen count, air quality, rain / snow conditions, wind speed), light exposure (e.g., ambient light, UV light exposure, time spent in darkness and / or duration), noise exposure, radiation exposure, and magnetic fields. Furthermore, the biometric monitoring device, or an external system receiving data from it, can calculate metrics derived from the data collected by the biometric monitoring device. For example, the device can derive one or more of the following from heart rate data: average heart rate, minimum heart rate, maximum heart rate, heart rate variability, heart rate relative to a target heart rate zone, heart rate relative to resting heart rate, heart rate changes, heart rate decreases, heart rate increases, training recommendations based on a reference heart rate, and medical conditions based on a reference heart rate. Some of the derived information is based on heart rate information and other data provided by the user (e.g., age and gender) or other sensors (altitude and skin conductivity).

[0177] Biometric sensors may include one or more sensors that assess the physiological aspects of the wearer, such as heart rate sensors, skin conductance sensors, skin temperature sensors, and electromyography sensors. Biometric sensors may also, or alternatively, include sensors that measure physical environmental characteristics reflecting how the wearer interacts with their surroundings, such as accelerometers, altimeters, GPS devices, and gyroscopes. All of these are biometric sensors that can be used to gain insight into the wearer's activities, for example, by tracking movement, acceleration, rotation, orientation, altitude, etc.

[0178] A list of potential biometric sensor types and / or biometric data types is presented below in Table 1, which includes motion and heart rate sensors. This list is not exclusive, and other types of biometric sensors different from those listed may be used. Furthermore, data potentially derived from the listed biometric sensors may also be derived wholly or partially from other biometric sensors. For example, assessing the number of stairs climbed may involve assessing altimeter data to determine changes in altitude, clock data to determine the rate of change in altitude, and accelerometer data to determine whether the biometric monitoring device is worn by a person walking (as opposed to standing still).

[0179] Table 1: Biometric Sensors and Data ( Physiological and / or environmental )

[0180]

[0181] In addition to the above, some biometric data can be calculated using biometric monitoring devices without directly referencing data obtained from biometric sensors. For example, an individual's basal metabolic rate (a measure of the "default" calorie expenditure a person experiences throughout the day at rest—in other words, the energy available only for basic bodily functions such as breathing and blood circulation) can be calculated based on user-input data and then combined with data from an internal clock indicating the time of day to determine how many calories a person has consumed throughout the day to provide only the energy needed for basic bodily functions.

[0182] Physiological sensors

[0183] As mentioned above, some biometric sensors collect physiological data, others collect environmental data, and some collect both types of data. Optical sensors are examples of sensors that can collect both types of data. Many of the sensors and data described below overlap with the biometric sensors and data presented above. They are organized and presented below to indicate the physiological and environmental sources of the information.

[0184] The biometric monitoring device of the present invention may use one, some, or all of the following sensors to acquire physiological data, including the physiological data outlined in Table 2 below. All combinations and arrangements of physiological sensors and / or physiological data are intended to be within the scope of the present invention. The biometric monitoring device of the present invention may include, but is not limited to, one, some, or all of the sensors specified below to acquire corresponding physiological data; in fact, other types of sensors may be used to acquire corresponding physiological data, which are intended to be within the scope of the present invention. In addition, the device may derive physiological data from the output data of the corresponding sensors, but is not limited to the number or type of physiological data that can be derived from said sensors.

[0185] Table 2: Physiological Sensors and Data

[0186]

[0187] In one exemplary embodiment, the biometric monitoring device includes an optical sensor to detect, sense, sample, and / or generate data that can be used to determine information representing heart rate. Additionally, the optical sensor may optionally provide data for determining a user's stress (or its level) and / or blood pressure. In one embodiment, the biometric monitoring device includes an optical sensor having one or more light sources (LEDs, lasers, etc.) to emit or output light to the user's body and / or a photodetector (photodiode, phototransistor, etc.) to sample, measure, and / or detect responses or reflections and provide data for determining information representing the user's heart rate (e.g., using photoplethysmography (PPG)), stress (or its level), and / or blood pressure.

[0188] Environmental sensors

[0189] The biometric monitoring device of the present invention may use one, some, or all of the following environmental sensors to acquire environmental data, including the environmental data outlined in Table 3 below. The biometric monitoring device is not limited to the number or type of sensors specified below, and may use other sensors to acquire the environmental data outlined in the table below. All combinations and arrangements of environmental sensors and / or environmental data are intended to fall within the scope of the present invention. Furthermore, the device may derive environmental data from the output data of the corresponding sensors, but is not limited to the types of environmental data that can be derived from said sensors.

[0190] The biometric monitoring device of the present invention may use one or more or all of the environmental sensors and one or more or all of the physiological sensors described herein. In fact, the biometric monitoring device of the present invention may use any sensor now known or to be developed later to acquire any or all of the environmental and physiological data described herein, all of which are intended to fall within the scope of the present invention.

[0191] Table 3: Environmental Sensors and Data

[0192]

[0193] In one embodiment, the biometric monitoring device may include, for example, an altimeter sensor disposed or located inside the device housing. In this case, the device housing may have vents that allow the internal components of the device to measure, detect, sample, and / or experience any changes in external pressure. In one embodiment, the vents prevent water from entering the device while facilitating changes in pressure measured, detected, and / or sampled via the altimeter sensor. For example, the outer surface of the biometric monitoring device may include a vent-type configuration or architecture (e.g., a GORE™ vent) that allows ambient air to move in and out of the device housing (which allows the altimeter sensor to measure, detect, and / or sample changes in pressure), but reduces, prevents, and / or minimizes the inflow of water and other liquids into the device housing.

[0194] In one embodiment, the altimeter sensor may be filled with gel, which allows the sensor to experience pressure changes on the outside of the gel. Using a gel-filled altimeter can provide a high level of environmental protection to the device with or without an environmentally sealed vent. The device may have a high survival rate in locations where the gel-filled altimeter is located, including but not limited to locations with high humidity, washing machines, dishwashers, dryers, steam rooms, shower rooms, swimming pools, and any location where the device may be exposed to moisture, exposed to liquids, or submerged in liquids.

[0195] Generally, the technologies and functions outlined above can be implemented in a biometric monitoring device as a set of machine-readable instructions, as software stored in memory, as an application-specific integrated circuit (ASIC), as a field-programmable gate array (FPGA), or as other mechanisms for providing system control. Such a set of instructions can be provided to one or more processors of the biometric monitoring device so that the processors control other aspects of the biometric monitoring device to provide the functionality described above.

[0196] Unless the context of the invention (wherein the term “context” is used according to its typical general definition) explicitly requires otherwise, throughout the description and claims, the word “comprising” and the like will be interpreted in an inclusive sense, contrary to the meaning of exclusivity or exhaustiveness; that is, in the sense of “including but not limited to.” The use of singular or plural terms generally also includes the plural or singular, respectively. Furthermore, the words “in this document,” “in the following text,” “above,” “below,” and similar terms refer to the entire application and not to any particular part of the application. When the word “or” is used in a list referring to two or more items, the word covers all of the following interpretations: any one of the items in the list, all the items in the list, and any combination of the items in the list. The term “implementation” refers to an implementation of the techniques and methods described herein, and a physical object embodying the structure and / or having the techniques and / or methods described herein.

[0197] Numerous concepts and embodiments are described and illustrated herein. While specific features, attributes, and advantages of the embodiments discussed herein have been described and illustrated, it should be understood that many other, different, and / or similar embodiments, features, attributes, and advantages will be apparent from the description and illustration. Therefore, the above embodiments are merely exemplary and are not intended to be exhaustive or to limit the invention to the precise forms, techniques, materials, and / or configurations disclosed. Many modifications and variations are possible according to the invention. It should be understood that other embodiments can be utilized, and operational changes can be made without departing from the scope of the invention. Therefore, the scope of the invention is not limited to the above description, as the above description of embodiments has been presented for purposes of illustration and description.

[0198] Importantly, the present invention is not limited to any single aspect or embodiment, nor to any single combination and / or arrangement of such aspects and / or embodiments. Furthermore, each aspect and / or embodiment of the invention may be used alone or in combination with one or more other aspects and / or embodiments. For the sake of brevity, many of those arrangements and combinations will not be discussed and / or described separately herein.

Claims

1. A method for tracking a user's physiological activities using a wearable biometric monitoring device, said wearable biometric monitoring device having one or more heart rate sensors providing heart rate data and one or more processors, the method comprising: (a) Determining that a first portion of the heart rate data includes data indicating that the user is participating in a first activity and that the signal strength characteristics of the first portion of the heart rate data provided by the one or more heart rate sensors are higher than a first threshold, wherein the first activity is walking or running; (b) The heart rate metric is updated using information obtained from a time-domain analysis of the first portion of the heart rate data, based at least in part on the determination that the signal strength characteristics of the first portion of the heart rate data are higher than the first threshold. (c) Determine that the second portion of the heart rate data includes data indicating that the user is participating in a second activity and that the signal strength characteristics of the second portion of the heart rate data are less than or equal to a second threshold, wherein the second activity is selected from the group consisting of: elliptical trainer exercise, stair trainer exercise, aerobic machine exercise, weightlifting, driving, swimming, cycling, climbing stairs and rock climbing. (d) The heart rate metric is updated using information obtained from frequency domain analysis of the second portion of the heart rate data, based at least in part on the determination that the signal strength characteristics of the second portion of the heart rate data are less than or equal to a second threshold. (e) Repeat (b) to (d) for additional portions of the heart rate data; and (g) Control the display device of the wearable biometric monitoring device to display the heart rate measurement.

2. The method of claim 1, wherein the frequency domain analysis requires more computation per unit duration of the heart rate data than the time domain analysis.

3. The method of claim 1, wherein the frequency domain analysis requires more computation per unit of the heart rate measurement than the time domain analysis.

4. The method according to any one of claims 1 to 3, wherein the wearable biometric monitoring device comprises a wrist-worn biometric monitoring device or an arm-worn biometric monitoring device.

5. The method according to any one of claims 1 to 3, further comprising applying a classifier to the heart rate sensor output data to determine the placement of the wearable biometric monitoring device on the user.

6. A method for tracking a user's physiological activities using a wearable biometric monitoring device, the wearable biometric monitoring device having one or more sensors providing sensor output data indicative of the user's physiological activities and one or more processors, wherein the one or more sensors include one or more motion sensors, the method comprising: (a) Operating the one or more sensors when the user wears the biometric monitoring device; (b) When the user walks and / or runs, the one or more sensors are used to generate a first set of motion data and a first set of sensor output data, wherein the first set of motion data and the first set of sensor output data are the same set of data or different sets of data; (c) By detecting a first signature signal in the first set of motion data, the one or more processors automatically determine that the user is walking and / or running, wherein the first signature signal represents the user's motion and is selectively associated with walking and / or running; (d) By the one or more processors and based on the determination that the user is walking and / or running, by automatically selecting time domain analysis via frequency domain analysis to analyze the first set of sensor output data, thereby quantifying a first physiological measure; (e) Present the first physiological measure; (f) When the user participates in an activity consisting of a group selected from the following, the one or more sensors are used to generate a second set of motion data and a second set of sensor output data: elliptical machine exercise, stair climber exercise, aerobic machine exercise, weightlifting, driving, swimming, cycling, climbing stairs, rock climbing, and any combination thereof, wherein the second set of motion data and the second set of sensor output data are the same set of data or different sets of data. (g) By detecting a second signature signal in the second set of motion data, the one or more processors automatically determine that the user is participating in an activity selected from the group consisting of: elliptical trainer exercise, stair climber exercise, aerobic exercise, weightlifting, driving, swimming, cycling, climbing stairs, rock climbing, and any combination thereof, wherein the second signature signal is different from the first signature signal, and the second signature signal indicates the user's motion and is selectively associated with the activity selected from the group consisting of: elliptical trainer exercise, stair climber exercise, aerobic exercise, weightlifting, driving, swimming, cycling, climbing stairs, rock climbing, and any combination thereof; (h) By means of the one or more processors and based on determining that the user is engaged in the activity selected from the group consisting of: elliptical machine exercise, stair machine exercise, aerobic exercise, weight training, driving, swimming, cycling, climbing stairs, rock climbing and any combination thereof, the frequency domain analysis is automatically selected via the time domain analysis to analyze the second group of sensor output data to quantify the second physiological measure. and (i) Present the second physiological measure.

7. The method of claim 6, wherein the sensor output data includes one or more of the following: motion data, location data, pressure data, light intensity data, and / or altitude data.

8. The method of claim 6, wherein detecting the first signature signal in the first set of motion data comprises characterizing the first set of motion data based on the signal norm, signal energy / intensity in certain frequency bands, wavelet scaling parameters, and / or the number of samples exceeding one or more thresholds.

9. The method of claim 6, wherein detecting the second signature signal in the second set of motion data comprises characterizing the second set of motion data based on the signal norm, signal energy / intensity in certain frequency bands, wavelet scaling parameters, and / or the number of samples exceeding one or more thresholds.

10. The method of claim 6, wherein the sensor output data includes raw data obtained directly from the sensor.

11. The method of claim 6, wherein the first physiological measure or the second physiological measure includes heart rate.

12. The method of claim 6, wherein the first or second physiological measure includes the number of stairs climbed, calories burned, and / or sleep quality.

13. The method according to any one of claims 6 to 12, wherein the biometric monitoring device comprises a wrist-worn biometric monitoring device or an arm-worn biometric monitoring device.

14. The method according to any one of claims 6 to 12, wherein the one or more sensors comprise one or more accelerometers, one or more gyroscopes, one or more inertial sensors, and / or one or more GPS devices.

15. The method of any one of claims 6 to 12, further comprising applying a classifier to the sensor output data to determine the placement of the biometric monitoring device on the user.

16. A biometric monitoring device, comprising: One or more sensors, which provide output data including information about the user's activity level when the biometric monitoring device is worn by the user; as well as Control logic configured to perform any one of the methods according to any one of claims 6 to 12.

Images