Cluster-Based Sleep Analysis

Abstract

To provide sleep analysis related to a specific individual.SOLUTION: The various types of pieces of data, which may be related to sleep of a user in a certain period and related to a physical or mental health of the user, may be collected. On the basis of the data which may associate the user with a specific health type or category, a health index may be determined. Sleep analysis may include associating the user with a sleeping animal which has specific characteristics, and this may be useful for further understanding his / her own sleep of the user, and quality of the sleep of the user in comparison with sleep of other people. In addition to provision of the health information in an understandable method, such approach may also be useful for providing more precise propose, or performing a specific action, for improving a health state of the user. The invention may include providing propose to the user, or automatically adjusting an operation of at least one device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] Priority claim This application claims the benefit of priority to U.S. patent application Ser. No. 17 / 162,286, filed Jan. 29, 2021, entitled "Cluster-Based Sleep Analysis." [Background technology]

[0002] background In various conventional approaches for sleep testing and analysis, an individual may visit a sleep clinic and have their sleep monitored over the course of a night. While such analysis can provide useful information, it is difficult to determine how relevant this sleep data is to other nights or longer periods of time, and how an individual sleeps in their own bed in a comfortable environment. While recent advances in wearable technology, such as fitness trackers and smart watches, allow for the collection of sleep data at home, the information displayed to the wearer of these devices is often too generalized and does not provide the individual with a full understanding of their sleep, such as whether there are areas for improvement related to the specific individual and whether there is anything they can do to achieve that improvement.

[0003] Various embodiments in accordance with the present disclosure will now be described with reference to the drawings. [Brief description of the drawings]

[0004] [Figure 1] 1A-1C illustrate example interface states that may be generated on different devices according to various embodiments. [Diagram 2] FIG. 2 illustrates an example sleep cluster that may be generated in accordance with various embodiments. [Diagram 3] FIG. 1 illustrates an example radial graph of a particular user and corresponding sleeper type that may be generated according to various embodiments. [Figure 4]FIG. 1 illustrates example comparative sleep data that may be provided according to various embodiments. [Figure 5A] FIG. 1 illustrates an example display that may be utilized in accordance with various embodiments. [Figure 5B] FIG. 1 illustrates an example sensor configuration that may be utilized in accordance with various embodiments. [Figure 6] FIG. 1 illustrates an example device interaction that may be utilized in accordance with various embodiments. [Figure 7] FIG. 1 illustrates an example process for determining, providing, and utilizing sleep data in accordance with various embodiments. [Figure 8] FIG. 2 illustrates an example monitoring device light path that may be utilized in accordance with various embodiments. [Figure 9] FIG. 2 illustrates components of an example networked monitoring device that may be utilized in accordance with various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Detailed Description In the following description, various illustrative embodiments are described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the embodiments. However, it will also be apparent to one skilled in the art that the embodiments may be practiced without the specific details. Additionally, well-known features may be omitted or simplified in order not to obscure the embodiments being described.

[0006] People are becoming increasingly health conscious, in part due to the availability of digital health monitoring devices such as fitness trackers. Such devices can capture various types of information about a user, such as activity or motion data and resting heart rate, which can be used to infer information about a person wearing or otherwise associated with one or more of the devices. At least some of these devices, or systems or services in communication with these devices, can analyze this and other data to provide information about the person's physical health, which may include one or more sleep patterns of the person. This may include, for example, the person's sleep onset and wake times, as well as periods during the sleep during which the user may have been in different sleep stages, such as deep sleep, light sleep, REM (rapid eye movement) sleep, wakefulness, etc. At least some of these devices may also provide a quantitative assessment of the person's sleep, such as providing a sleep score that provides a measure of the quality of the person's sleep obtained on a particular night or over a series of nights.

[0007] While this data may be at least somewhat useful to users, these users often do not fully understand the data or know how to interpret it. For example, a user may receive information indicating that they had 21 minutes of REM sleep one night, but not know if that is too long, too short, or reasonable. Additionally, a user may receive a sleep score of 89, but not know what that represents, or what the goal or average value should be. In some instances, based on this collected or inferred sleep data, these devices may make suggestions to exercise more, not consume alcohol or caffeine before bed, go to bed earlier, etc. Unfortunately, such information may be relatively general and may not be particularly useful to any individual. Additionally, individuals who do not consume caffeine or alcohol may not only find the suggestions irrelevant, but may actually find them annoying, which may reduce the overall value of the information and the usefulness of the product or service.

[0008] Thus, approaches according to various embodiments can provide more detailed and individualized sleep analysis. The various approaches can provide suggestions, visualizations, and insights that are not only relevant to a particular individual, but should be easy to understand and understand for most people. The various approaches can automatically make or cause changes based at least in part on this sleep-related data to devices worn by the user or to devices associated with the individual that may affect the individual's sleep in some way.

[0009] FIG. 1 illustrates an example display approach 100 that may be utilized in accordance with at least one embodiment. In this example, a user may wear a wearable computer 102 or monitoring device, such as a fitness tracker or smart watch, on the user's wrist or arm 104. Other wearable computers or monitoring devices that may be worn elsewhere or in other ways are also available, which may include smart rings, bands, earphones, straps, clothing, contacts, patches, and other "smart" or networked devices. In many embodiments, the wearable computer or device includes a touch-sensitive display that allows the wearing user to input or receive information related to their physical or mental state, as described herein. In this example, the wearable device wirelessly communicates (e.g., with at least one other computing device 110 associated with the user) with at least one other computing device 110 associated with the user. The devices 102, 110 may communicate with one another over a network, such as a wearable computer, a desktop computer, a laptop, or a tablet computer, using a wireless network. The devices 102, 110 may communicate with one another over a wireless network, such as a wearable computer, a fitness tracker, a wearable computer ...

[0010] As described in more detail later herein, the user's data may be collected using the wearable device 102, the associated computing device 110, or other electronic devices or mechanisms. The wearable device 102 may include various sensors, such as motion and temperature sensors, that may be used to measure or detect information about the user. The sensors in some illustrative embodiments are non-invasive and do not require the insertion of any instruments into the body of the wearing user. In one embodiment, the user interface may provide the ability for the user to input specified data. The wearable device 102 may further include an optical measurement subsystem that includes at least one light emitter and at least one light detector or receiver. The light emitter may emit light of one or more wavelengths that may be reflected from the user's skin surface or diffusely reflected after traveling below the surface and detected by at least one of the receivers. Such an optical assembly may enable the monitoring device to measure various types of information while the monitoring device 102 is worn by a user.

[0011] At least a portion of this collected data may be analyzed to attempt to determine the type or category of "sleeper" to which the user corresponds. Being able to determine the type of sleeper, i.e., sleeper type, may be beneficial to the user, as it may allow the user to better understand their sleep relative to other sleepers of that sleeper type. Furthermore, suggestions may be provided that are more relevant to the type of sleeper, and thus more likely to be relevant to a particular user of that type. And further, such classification may help provide an analysis that is relatively specific to the user, but can be provided or visualized in a manner that should be relatively easy to understand for most users, especially with respect to the sleep of similar users.

[0012] In at least one embodiment, sleep-related data collected about a user can be analyzed and used to determine a category or type of sleeper to which the user belongs. In at least one embodiment, data of multiple users can be analyzed and used to generate a set of sleeper types or categories into which users can be classified. As described in more detail elsewhere herein, once these categories or sleeper types are determined, the most characteristic habits (or patterns or characteristics, etc.) of each sleeper type can be determined by determining relevant aspects of each type and contrasting them with aspects of other types. In at least one embodiment, these habits can then be analyzed to attempt to associate a category label representing the respective sleeper type that should be easy for the user to understand.

[0013] An example of a sleeper type label may be "sleeping animal." Each sleeping animal may represent a respective sleeper type and may be at least somewhat indicative of that sleeper type for a user to understand. For example, a "bear" sleeping animal may be associated with a sleeper type where people tend to sleep deeply for long periods of time, such as bears hibernating. Another example may be a "kangaroo" sleeping animal where a user frequently "jumps" between different sleep states, or where a user is most likely to sleep in a "kangaroo" sleep state. A "hummingbird" sleeper animal may be a "hummingbird" sleeper animal, which is a much lighter sleeper than a "hummingbird" sleeper animal. Various other labels, types, and characteristics are also usable within the scope of various embodiments. Such an approach may provide a relatively simple and straightforward way for a user to understand their sleeper type, at least compared to other sleepers, and particularly compared to other similar sleepers. Furthermore, if a user's sleeper type switches over time, this approach provides a relatively easy way for the user to understand and follow the changes over time. In some embodiments, the sleepers may be associated with different health outcomes, particularly specific profiles of body mass index (BMI), resting heart rate (RHR), and heart rate variability (HRV).

[0014] An advantage of such an approach for labeling is that it provides a visualization that different types of users can easily understand and follow. For example, a relatively simple animated version of the sleeping animal 108 can be presented on the display of the wearable computer along with other relevant information 106. This can be presented at various times to remind the user of their sleeper type, which can help the user make decisions regarding activities based at least in part on the characteristics of their sleeper type. Additionally, the sleeping animal can be presented at specific times when it is desirable to prompt the user or convey specific information, such as when the user should think about going to sleep, unwinding, etc. In at least some embodiments, the sleeping animal can be animated to convey this or other such information, such as turning off the lights or device, getting into bed, setting an alarm, etc. In at least some embodiments, the presentation of the sleeping animal on the wearable computer may further convey that additional information, which may provide more detailed information, is available on an associated device, such as a connected smartphone. For example, a simple summary may be provided on the wearable computer, and the user can access the connected user device 110 for additional details or content 114. A user may also be able to access additional information on other devices, such as via a browser on a desktop or laptop computer that is accessible through the user account.

[0015] In one example, six typical sleeper types present across the identified population were defined and developed. In this example, there were also eleven sleep metrics selected or defined from available metrics or data. These sleep metrics can be used to determine how the user slept over a period of time, such as the past 28 nights, compared to others similar to the user, such as the same or similar sleeper type. However, it should be understood that in different embodiments or implementations, there may be additional, fewer, or alternative types or metrics, and the number of types utilized may depend on factors such as the type and results of clustering, the type of health metric being monitored, the population being evaluated, etc. A user profile, such as a sleep history, can then be generated based at least in part on the respective sleeper types and the values ​​of these advanced sleep metrics. The user profile can serve to teach the user the user's sleeper type using the user's sleep animal, and how the user is rated across those advanced sleep metrics. These histories may be static based on the information or sleeper type sought about the user, or they may be dynamic and updated as new sleep-related data, suggestions, or other information is acquired, received, updated, or generated.

[0016] In at least one embodiment, the user's past sleep data is available so that a personalized characterization of the user's sleep behavior can be selected, for example, in the form of a sleeping animal or other characterization that provides a straightforward yet personalized assessment of the user's sleep. Other examples of characterizations include types of people, geography, weather, vehicles, etc. Characterizations of a user's typical sleep can be identified and different aspects of those characterizations can be associated with different aspects of sleep for different users. Moreover, as described herein, such approaches are not limited to sleep, but can be used for other aspects of physical or mental health that may relate to mental state, physical health, fitness, personality type, etc. A biographical or other interface or presentation of content can be used to provide visualizations that teach the user how they are sleeping compared to others similar to them across advanced sleep metrics. Characterizations of a user's typical sleep can identify opportunities where the user can improve their sleep, making the user feel that the health monitoring system, service, or device understands them well.

[0017] In various embodiments, the sleep history can be supplemented with other sleep-related information, such as a sleep score that evaluates the quality of sleep from the previous night. In particular, a given sleep score can evaluate or be calculated from factors such as last night's sleep onset and wake time, deep sleep and REM (rapid eye movement) sleep time, and sleep heart rate and restlessness. However, as mentioned above, such scores alone may lack analysis of long-term sleep patterns, such as sleep consistency, sleep chronotype, sleep fragmentation that may be due to insomnia or apnea, and polyphasic sleep patterns. The sleep history can help characterize and visualize the user's sleep over a period of time, such as the past 28 nights, which can be used as a basis for developing a sleep report card that determines the user's sleep over the past month, or other such content presentations. In at least one embodiment, such a sleep profile can help quantify "good" or "bad" sleep, or "good" or "bad" sleep compared to one or more previous periods or other users, and this information can be used to create an actionable sleep report card that can also support a larger range or presentation of health-related actionable summary scores. Collecting data over several nights or several sleep periods can also help determine the user's sleep probabilities, which can help make better suggestions to the user based not only on past sleep but also on predicted sleep.

[0018] To determine a user's sleeper type, in at least some embodiments, a determination is first made of possible sleeper types. This may include collecting and analyzing data for multiple users and then determining a sleeper type based at least in part on the analysis. As mentioned above, this may include analysis of any potentially sleep-related data that may be collected, detected, or otherwise provided by any of a number of sources. For example, a wearable computer or sensor may be capable of providing information such as movement, activity level, breathing patterns, heart rate, blood chemistry, skin temperature, ambient temperature, and body position data. A device with a camera or microphone, such as a smartphone, may be capable of providing sound, image, or video data and other data that may be captured by the device's sensors, which may include ambient light, geographic location, pressure, room temperature, and activity data. Another nearby smart device or connected device may be capable of providing additional data such as movement, activity, weather, humidity, device status, user status, and other such information. A user may also provide various types of information through one or more interfaces, which may include information such as gender, age, race, medical history, medications being taken, mental status information, physical activity level, etc. At least a portion of this information may also be obtainable from one or more other sources, such as a user account, a public database, a medical database, a genealogy database, or a third party source.

[0019] In at least some embodiments, some or all of this data may be aggregated for analysis. This data may be analyzed using various approaches to attempt to select data to include in the analysis. The selected data may then be further analyzed to attempt to reduce the dimensionality of the data set. For example, processing the data may be performed to By using a set of parameters, it may be attempted to eliminate redundancy in the data set. Then, it may be attempted to reduce the data set to a relatively small number of factors that are particularly indicative of different sleeper types. In at least one embodiment, this may involve performing clustering using one or more clustering algorithms, such as k-means clustering, with different clustering parameters. This process may be continued iteratively until a relatively small number of clusters that are sufficiently different from each other are obtained. FIG. 2 shows a set of radial plots 200 illustrating an example set of sleeper types that may be utilized in accordance with various embodiments. In this particular example, the clustering resulted in six distinct sleeper types. These types were determined using ten sleep parameters that were determined to be the most important or at least influential in determining sleeper type or quality. As mentioned above, other numbers and selections of sleep parameters may also be used for such purposes. In at least one embodiment, additional parameters may be used to determine clusters or types, and then the number of most important factors used to distinguish between those types may be determined.

[0020] By providing a user with a visualization such as that of FIG. 2, the user can compare the values ​​of his or her sleeping animal or other sleeper type with the values ​​of other sleep categories. For example, the user who is a "turtle" type in this example takes the longest time to fall asleep among these animals, but wakes up relatively few times per hour after falling asleep. Without being able to visually compare plots of these different types, the user may not know how to interpret the data of the user's sleeping animal alone, or how it compares to the data of other animal types. Furthermore, these comparisons can help the user understand why a different sleeper type may be more accurately provided to the user, even though the user may think he or she has a different sleeper type. Furthermore, the user can be provided with information that the user's perception may be misleading, since the user is asleep most of the time and is only aware of a small portion of the total sleep of the night.

[0021] While such visualizations may provide the user with insight into how the user's sleeper type may compare to other sleeper types, the visualization may not provide sufficient information about how the user compares to other animals of the same sleeper type. Thus, a user may be provided with a visualization 300 such as that shown in FIG. 3. In this example, the user is provided with a plot showing the sleep index values ​​of the individual user (on the left side of the figure) and a plot showing the average values ​​for users of that particular sleeper type (here, kangaroos) (on the right side). It can be seen that the overall shape of these plots is somewhat similar so that the user can understand why they were classified as having that particular sleeper type. However, when these plots are displayed together, the user can understand how the user's sleep compares to other users of that same sleeper type. For example, the user may determine that a recovery value of about 30 is typical for users of this sleeper type. That same user may determine that the user's deep sleep minutes are longer than the average user, closer to the 100th percentile, but that the deep sleep time is also significantly longer than the average user of this sleeper type, around the 50th percentile. The user can also see that this user tends to go to bed much later and sleep much longer than the average user of this type. Based on this type of comparison, suggestions may be made for the user to go to bed earlier or sleep less to have a pattern more typical of this sleeper type. Because this user's resilience is lower than average, but typical of this sleeper type, attempts to increase resilience may not be as beneficial for this sleeper type. In some embodiments, the user may also be able to see a comparison of the user's sleep metrics to those of different sleeper types, such as the other people shown in FIG. 2, to understand how the user compares to other sleeper types. If there is a sleeper type that the user would like to be, the user may , they can see how their sleep metrics differ from that of the target sleeper type and attempt to adjust according to those differences.

[0022] Such a presentation may help overcome the shortcomings of previous approaches in that it may at least help communicate to the user whether an overall sleep pattern or condition should be considered normal or not. It may also help visualize whether a particular sleep metric should be considered normal or not compared to people in general or other people of the same sleeper type. This may help the user determine whether they should be concerned based on any of these values ​​and whether changes can or should be made. In at least some embodiments, a suggestion engine may be used to provide the user with context regarding what goals are achievable in the short and medium term. This information and other such information may be provided as part of the user's sleep history and may help inform the user of things the user is doing well and things that could be improved. Such an approach may also provide at least some context regarding how other people compare to this user and what they could have improved and what has worked for other users of the same sleeper type.

[0023] In one example, over 1000 sleep-related features were analyzed that may help describe how a person slept over a period of time, such as the last month. This data may be analyzed, such as with sleep experts and data analytics, to attempt to select values ​​that are most relevant to sleep determinations, such as the time a person goes to bed and the number of times the person awakens during the night. It may then be possible to reduce this vast collection of features to a much smaller number, such as about 50-75 feature values, or in one example, 64 feature values. As discussed above, these features may be selected based at least in part on what the data indicates is important and what sleep experts believe to be important. This selection of data may also be based on the quality of clustering that can be performed based on those features. As discussed above, various clustering algorithms may be utilized, which may include a k-means clustering approach in which selected features are projected into a feature space and Euclidean distance is utilized for clustering. In at least some embodiments, the clustering may be optimized for aspects such as interpretability, compactness of individual clusters, or separation between clusters. In various embodiments, because clustering may not work well in high-dimensional spaces, approaches such as principal component analysis (PCA) may be used to take selected features and reduce the dimensionality. In one example, PCA helped reduce the dimensionality from 64 principal components to about 22 principal components, and then clustering was performed on those principal components. A set of heuristics or rules may then be applied to finalize the sleeper type based on these clusters. Instead of showing all relevant features, a subset of these features (e.g., 10 features) may be selected and displayed to the user along with the sleeper type data. These selected important features may correspond to advanced sleep metrics, which are metrics determined to be most important for determining and / or understanding sleep. In cases where a user may resemble two different sleeper types or where the user does not have an association with a dominant sleeper type, at least some rules may be applied to select or adjust the sleeper type shown to the user.

[0024] Various other types of visualizations may also be provided. For example, instead of multi-dimensional plots such as those shown in Figures 2 and 3, a user may wish to obtain a visualization or report that more directly compares the user's specific sleep metrics to the values ​​of an average user, or even to users of a given sleeper type. One such visualization 400 is shown in the graph of Figure 4. In this example, the user's values ​​for each individual sleep metric are compared directly to the average metric. Such a visualization may be useful to show those metrics (e.g., time in bed, hours asleep, minutes in deep sleep, and number of awakenings) where the user is significantly higher than the average, as well as those metrics (e.g., time in bed, hours asleep, minutes in deep sleep, and number of awakenings) where the user is significantly higher than the average. This may provide at least some users with a more intuitive way to quickly determine which of those metrics (e.g., % REM) are lower than the average value for each sleep metric. Additionally, the graph shows what the average value for each sleep metric is. The user may be able to select or see different types, and may be able to provide preferences or favorites that can be used to present such information to the user in the future. The user may also have the ability to indicate which formats they do not want, and may have at least some ability to customize these presentations, such as comparing user values, average values, and average values ​​for a given sleeper type on the same graph.

[0025] As mentioned above, this information can be provided as part of a personalized sleep analysis, summary, or history. In addition to providing sleep index values ​​and sleep animal visualizations, information can be provided that can specifically show the user what the user is doing well and what they may be able to improve. This information can also show what approaches may help improve things the user may not be doing well or may vary greatly, based at least in part on data collected about the user and what has worked for other users of the same or similar sleeper type. While percentages are provided in various examples, other types of data may be provided that may be useful to the user, such as showing the user's average minutes for each sleep type relative to the average minutes across all users and across users of a given sleeper type. Information about why each of these sleep metrics is important and the individual user's goal values ​​can also be provided. Trends can also be identified, predictions made, and suggestions provided to help avoid undesirable trends or predictions. In some embodiments, predictions or anomalies may trigger an interface to request information from the user that may help narrow down potential causes or generate more targeted suggestions. This may include asking about stress, pain, caffeine or alcohol intake, dietary patterns, etc., which may be difficult to determine by monitoring data.

[0026] As described herein, such information can be collected at least in part from and presented at least in part using a device such as the wearable monitoring device 502. This can help reduce the barrier to obtaining the type of data necessary to perform a meaningful analysis of sleep state, since a wearable device can collect data points more frequently and perform such analysis periodically, possibly multiple times a day, to determine relationships such as sleep state with caffeine intake, how sleep is based on activity and / or food consumed, and how stressed the user is. Such devices can also allow this sleep data to be collected over a period of time, such as days, weeks, or months. Data collected over a longer period of time can help provide better averages or ranges for various sleep metrics and can also help identify sleep trends or patterns. Such data collection can also help determine whether data for a given night is typical or atypical for a given user, which can help interpret any collected data in context. For example, sleep studies that only capture one night of data may generally not be able to make such determinations, limiting the accuracy of their analysis.

[0027] In one embodiment, a user may wear a monitoring device 502 that includes components such as an optical measurement subsystem photoplethysmography ("PPG") component 552 and an accelerometer. PPG can obtain volumetric measurements by illuminating the skin, such as with one or more light emitters and one or more detectors proximal to the wrist of the wearer of the device, as shown in FIG. 5B, and measuring changes in light absorption over time. The frequency of these changes may represent the user's heart rate or pulse. These measurements are affected by motion and may be influenced by a variety of factors, including the nature of the device, the type of device used, the type of device used, and the type of device used. Since it may be easy to detect low activity periods (or even periods that are not dependent on activity) during sleep, at least some embodiments may prefer to attempt to determine the user's resting heart rate (RHR). This may be accomplished, for example, during the night while the user is asleep, although other periods of low activity (or even periods that are not dependent on activity) may be used within the scope of various embodiments. The monitoring device 502 may determine the heart rate, for example, by detecting peaks in the optical signal. In some cases, the heart rate may not be reliably detected because there may not be a clear peak due to excessive movement or the like. For monitoring devices 502 that include an accelerometer, inertial sensor, or other such sensor or component, periods of excessive movement may be determined and excluded from the analysis. In other illustrative embodiments, data from these periods may be utilized as is, but the analysis takes into account the effects of movement. A sensor such as an accelerometer may also be used to determine the entire sleep period (e.g., from 11 pm to 7 am). An estimate of the user's RHR may then be calculated. In one embodiment, a histogram of the overnight heart rate values ​​may be generated. A designated measurement, such as the 10th percentile of this histogram, may then be taken as a representative value for the RHR. The RHR value may also be determined in other ways, such as by only using time segments during which the user is motionless for at least a minimum period, such as at least five minutes, and the optical signal has a sufficiently high signal-to-noise ratio. Once the RHR value is determined, it can be used to characterize the subject's entire day, including periods of sleep. The monitoring device may also include other sensors or detectors, such as a temperature sensor that can be used to measure skin temperature, as described herein, which the device can then display to the user through one or more additional displays of health-related data 506.

[0028] The physiological data captured may include any potentially relevant data, including, but not limited to, heart rate (HR), RHR, SpO2, hemoglobin concentration, water retention, sebum or collagen content, lipid content in blood or interstitial tissue, sleep logging, sleep quality, sleep duration, sleep stage architecture (including, but not limited to, time since sleep onset, total time in bed, total time awake), daytime and sleep HRV indices, indices derived from HR, time spent in different HR zones, respiration rate, active minutes, exercise logging, elevation change, step count, food log, hydration log, weight measurements, body mass index, body impedance analysis, mood log, symptom logging, time of day changes, location, basal body temperature, oral body temperature, ear body temperature, hormone levels detected by urine or blood test samples, and the like. In one example, RHR and hemoglobin concentration data may be collected using a selection of optical sensors as described herein, although other approaches may be used. It should be understood that the data described herein is only an example, and other combinations or types of indices and information may be used within the scope of various embodiments.

[0029] In at least some embodiments, the sleep-related information determined about the user can be presented or provided to the user or associated entities in a number of different ways using a number of different devices or communication channels. This may include raw data for analysis, or results of such analysis, among other such options. In at least one embodiment, the sleep-related data may be collected or determined using a device such as a wearable monitoring device 602 worn by the user, or a computing device 606 associated with the user, as shown in the system overview 600 of FIG. 6. In at least some embodiments, such data may further be collected by one or more smart devices 604 associated with the user. Such smart devices may include any device capable of collecting or determining data that may be useful for sleep analysis or other such health or condition analysis, and capable of providing that data in some manner to another computing device, such as via a wired or wireless connection. This connection may be direct, through at least one network 610, or through one or more other devices or The data may be collected via a channel. Smart devices may include, for example, a smart thermostat capable of providing temperature data, a smart television capable of providing status data (e.g., on / off status, brightness, volume, blue light status, or type of content being displayed), a smart alarm capable of providing settings data, a smart refrigerator capable of providing access data, and other such devices. In at least one embodiment, this data may be collected for analysis by a user computing device 606, such as a tablet or desktop computer running an application having sleep analysis capabilities. In at least some embodiments, at least a portion of this data may be provided to a service provider system 608 associated with the health monitoring device 602 or health monitoring software. In some embodiments, the user may instead subscribe to a service provided by this service provider 608, in which case the service provider 608 may receive the data and provide sleep-related analysis or suggestions. Some embodiments may further utilize a third party system 612 or service for at least a portion of this data collection or analysis, or to collect related data useful for the analysis. For example, this third party system 612 may provide other individuals' sleep data, updated sleep analysis, etc. In some embodiments, the third party system 612 may be associated with one or more of the smart devices 604 and may provide data collected by those devices and provided to the third party system. For example, a smart alarm system capable of providing information regarding open doors and windows, motion, smoke, and other such data may provide that data to the third party security system 612, which may then provide at least a portion of this data to the sleep-related service provider 608 for analysis, if permitted by the user, subject to local privacy laws, etc.

[0030] Such an approach has various advantages since multiple types of devices 602, 604, 606 can be used to collect data that may be related to an individual's sleep. This data can then be transmitted over at least one network 610 to a device, system, or provider that can aggregate and analyze the data to determine various sleep-related metrics and other such information. Such an approach is further advantageous since the devices used to collect and transmit this data do not require substantial processing or memory capacity since most of the data analysis and processing can be performed by a remote system, service, or device. Such an approach is further advantageous at least in that the service provider can also collect data from other users' or individuals' devices, allowing for more accurate comparisons with other data over similar time periods.

[0031] The results of such sleep analysis, or other health or condition analysis, may also be returned to any of these or other such systems, devices, services, or providers. For example, data or instructions may be provided to the smart device 604 to adjust conditions or perform tasks, such as turning down the volume on a television or dimming the lights in a room. In some embodiments, the data or instructions may be provided to a central system or device, such as a management device, which may then send individual instructions to associated devices to perform specific tasks or make specific changes. The data or instructions may also be provided to the monitoring device 602 worn by the user to provide prompts or suggestions to the user to be received relatively instantly. The data or instructions may also be provided to the user computing device 606 to be presented to the user, such as to provide updated sleep animals or animal status, changes in sleep metrics, suggestions to improve sleep, etc.

[0032] In situations where a user may live in a smart home, or at least a home having smart features or devices that may or may not have a centralized control system, at least some of this smart data may be captured for sleep analysis. Furthermore, instructions or suggestions may be provided to at least some of these smart devices or features based at least in part on the results of the sleep analysis or prediction. For example, a home may have a smart controller (whether a standalone device or software running on a computer inside or outside the home). This controller may be capable of interfacing with a number of smart devices, or at least networked devices. This network may be provided using one or more networking or communication protocols or channels, which may include Wi-Fi, Bluetooth, direct wired or wireless connections, infrared communication, near field communication (NFC), and the like. These devices may include any device that can provide data that may be useful in making a sleep decision, or that can take action based on one or more instructions generated in response to sleep-related data. Such devices may include, but are not limited to, smart appliances, televisions, speakers, security systems, monitors, sensors, and the like. These devices may provide various types of data that may be captured or otherwise sought, such as by utilizing one or more sensors or other such mechanisms, which may include temperature, pressure, motion, light, sound, color, operational state, changes in device state, etc. These devices may further make changes that may be associated with desired changes related to current or future sleep states based on instructions received from a sleep monitoring application or a central monitoring service. This may include, for example, adjusting operational states, changing configurations, powering on or off, etc. Having the device make at least one adjustment may include providing or executing, with software and / or hardware resources, an instruction, request, or call.This may include using circuits, mechanisms, or electronics to perform a task that affects the function, state, or operation of a device, system, service, or process. As noted above, this may include providing instructions internally to the device or system, providing instructions to another device or system, or providing instructions over a network, among other such options. Various types of adjustments may also be made to a device or component, such as a display changing the activation or brightness of its display or changing the content displayed via its display, among other such options.

[0033] For example, if a user is determined to be going to bed too late or having difficulty falling asleep, instructions may be sent to one or more smart devices at appropriate times to attempt to help the user fall asleep. This may include vibrating a smartwatch or fitness tracker, sounding an alarm, or otherwise providing a prompt or notification, such as animating an associated sleeping animal to yawn or prepare for bed. If the user is watching a connected smart TV, instructions may be sent to turn down the volume, turn down the brightness, and transition to a display mode with less blue light content. If a dimmable smart light or plug is connected, instructions may be sent to dim the lights. Other types of instructions may also be sent, which may be for a single instance or for actions to be taken over a period of time, among other such options. Alternatively, if the user is trying to sleep less or wake up earlier, instructions may be sent to start the coffee maker, start the media player, or turn on the lights in an attempt to wake the user up.

[0034] In at least some embodiments, these instructions can be determined based on the sleeper type or sleep goal, as well as the user's current sleep or health state. For example, if a smart alarm is set to ring at a certain time, but the user is in deep or REM sleep and does not want to be woken up from such a state, the alarm can be delayed (as permitted by the user or schedule) until the user is awake or in light sleep. Similarly, actions are to be taken at certain times to help the user fall asleep, but at least some of those actions may not be taken at this time if the user is already asleep. Some actions, such as turning off the lights, may still be taken, while other actions, such as triggering a sleep notification, may not be taken. In some devices, other actions may be taken based on the sleep state, such as turning off a smart light instead of lowering its brightness if the user is already asleep and does not need help falling asleep at this time. Various other changes can be made to one or more devices in an environment associated with the user, such as an environment in which the user is not currently present. For example, if a user is on his way home and needs to fall asleep soon after arriving home, devices in the smart home may activate, stop, change operating states, or adjust at least one setting corresponding to that desired goal. This may include, for example, adjusting to a cooler temperature for sleep, adjusting lights to a relatively low brightness, playing soothing music, turning off things that may distract the person, etc. Adjustments may also be made automatically to the smart watch or wearable device, such as providing a prompt or notification that the user should attempt to fall asleep soon. In some embodiments, at least some of these changes may not occur automatically, but may occur in response to a prompt to the user based on that desired goal. For example, if the user is returning home with guests, the user may not want the smart home to go into sleep mode.If the system has access to a user's calendar or task list and can determine that there is a qualifying activity or event, the system can use this to determine not to enter sleep mode, but instead to enter a default mode, guest mode, or other such mode. If it can determine that a guest has left, such as by using audio or video data, the smart home may determine to enter sleep mode at that time.

[0035] In some embodiments, adjustments may be made to help the user fall asleep or wake up, as well as to attempt to improve or adjust the user's sleep based on the determined sleep-related data. For example, if the user sleeps less than normal on a night, or if the user wakes up multiple times, the system may make adjustments to help the user get back to sleep or enter a deeper sleep state. This may include, for example, adjusting the temperature, activating a sound machine, adjusting the settings (e.g., firmness) of a smart bed, adjusting the brightness of a night light, etc. As discussed above, at least some of these adjustments may also be based at least in part on information known about that particular user or information determined to be relevant to a particular category of sleeper. The impact or effectiveness of such changes may be monitored over time to make more appropriate changes for this individual user, and information may also be aggregated across users or based on sleeper type.

[0036] As mentioned above, in various embodiments, some of this functionality may be performed automatically, while other parts of this functionality may require manual command or confirmation. This may include providing suggestions for the user to take an action or approve taking such an action. Actions taken by the user and their subsequent impact on sleep state or other health conditions may then be monitored over time to attempt to make better suggestions. This may be based not only on actions the user is likely to take, but also on the impact of those actions once taken. In some embodiments, the method described herein may include providing suggestions to the user to take actions or approve taking such actions. Machine learning, as described elsewhere, can be used to attempt to infer actions to take based at least in part on such information. Prompts to the user can take a variety of forms, which may include sound, audio, video, image data, vibration or haptic feedback, etc. These prompts may be provided by a wearable computer or any suitable device capable of receiving a prompt or determining the need for a prompt and then providing an appropriate prompt, notification, or presentation. As noted above, a computing device, such as a smartphone or tablet computer, is often in communication with a wearable device or monitoring system and one or more other smart devices, and this information can also be provided via the computing device. If possible, determining which device the user is actively using can help make a more accurate determination, for example, of the device or mechanism utilized for notifications as those the user is likely to receive. The type of notification or prompt provided can also be based at least in part on the device's ability to provide a prompt or allow confirmation, because a smart button that provides only press information can only provide binary data, such as confirmation or no confirmation, whereas a tablet computer can provide much more granular input. If the smart device can provide information about the activity the user is engaged in, such as playing a game, watching a movie, or browsing the web, the device can respond to present suggestions and prompts in the most impactful manner based at least in part on the type of activity. This activity information can also be used to determine an action to take, such as turning down the volume or changing to a different game or movie. This activity information can also be combined with health data, for example if the user is playing a game and has a high heart rate, a suggestion may be made to at least stop playing and relax more if the user is planning to go to sleep soon.If the user is listening to music, the media player may automatically adjust the playlist to play more (or less) relaxing music or a different type of music. In at least some embodiments, health data such as heart rate or breathing patterns can be used to monitor the impact of these changes and make further adjustments as needed. Statistics about the activity, such as screen time or type of activity, can also be used to attempt to identify sleep-related patterns or factors, which may help determine or suggest actions to take to improve sleep or other health-related aspects.

[0037] In at least some embodiments, these suggestions, prompts, or changes may be based at least in part on location. Data may be available, such as geographic location data, that may enable at least certain decisions to be made, such as whether the user is at home, at work, on vacation, etc. The suggestions or actions may differ based at least in part on this location data. This may be based at least in part on the fact that there are likely to be different devices available that may or may not be accessible to make the changes. In such examples, instead of an action, a prompt to the user may be created. If the user needs to stay up late and is at a friend's house, as may be determined from the user's contact information, the device may suggest to the user to turn up the volume or brightness, or may recommend a game or other activity. However, if the user is in a store, other suggestions may be made, such as to walk faster, as such actions may not be possible. In such examples, the user may instead be provided with a communication that motivates the user to start thinking about going to sleep or staying up and taking the appropriate action, even if the device or communication does not provide an actual suggestion for an action to take. This location data can also be used to determine the user's likely state, such as that the user should not sleep in a store or while driving.

[0038] In at least some embodiments, a device or application may be able to connect to different systems or environments at different times or locations, and actions or suggestions may be made or adjusted based at least in part thereon. For example, a user device may be able to connect to a smart home when the user is in the smart home or close enough to the smart home, such as when the device is able to connect to the user home network, or when geographic data can be used to determine proximity. Alternatively, the user device may connect to the user's car network and be able to access the car's smart features. Similarly, the device may be able to connect to a work network, or be able to determine that the device is in a location that has internet-connected devices or features. In any of these locations or environments, the device may be able to determine what features are available and what types of data are available or collectable, and may make suggestions or take actions based at least in part on the types of features available in a given environment.

[0039] In some embodiments, the user may not be prompted with suggestions, but the information may instead be provided for the user to access at a time convenient to the user. This may include, for example, providing information or tips through an interface of a health application available through a computing device or a networked interface. For example, the user may access a requested sleep history for the user that provides the user's sleeper type and related data. This history may provide easy-to-understand information about things the user is doing well, such as getting enough sleep, having enough deep sleep minutes, and things the user is not doing well, such as going to bed too late, waking up too often, etc. For areas that could be improved, suggestions may be provided that may be appropriate for the user based on the data collected about the user as well as the type of sleeper currently determined for the user. This may include, for example, making changes to the environment that are generally beneficial to improving that aspect of sleep, at least for a given type of sleeper. In some examples, there may be suggestions that could benefit from additional information. However, as discussed above, some suggestions may be off-putting for some users, so it may be desirable to request additional information. For example, rather than suggesting that a non-drinker abstain from alcohol before bed, the interface may prompt the user in a less irritating manner to select from a range of beverages (which may include several options, such as coffee, soda, alcohol, water, or other such beverages) that the user may consume before bed, and this information may be used to help determine the relevance of a particular suggestion or to better determine health or condition information for the user. Information may also be provided about why consuming certain beverages or foods may be particularly impactful for certain sleeper types.If a device such as a smart refrigerator can determine, such as by scanning a barcode, that a user typically drinks a beer before going to bed, this information can be used to make such inferences and suggestions. Questioning the user can be useful for other reasons as well, such as to gather information about actions the user may have taken (that information may not otherwise be available) and the user's impressions of whether those actions made any difference, and if so, to provide information about what difference. Such information can help to better understand the user and make better suggestions, including not only actions that are likely to help achieve a desired goal or outcome, but also actions the user is likely to actually take at a given time, place, or condition.

[0040] In some embodiments, a user interface, such as a dynamic sleep history, may attempt to gather information from the user in a number of different ways, such as by asking the user questions at specific times. This is because certain data may lead to certain decisions, actions, or recommendations. The data may include requesting the data if it is determined to be useful for the proposal, or may include collecting data over time that may help understand the user while not overloading the user with questions or actions. In some embodiments, the questions may be simple yes / no answer questions that may be handled via a device that may have a limited interface, such as a smartwatch or fitness tracker with a small touch screen. In other embodiments, the questions may be presented when the user accesses a corresponding app on a device with more resources and interface capabilities, such as a smartphone or tablet computer. In one embodiment, one or more surveys may be designed that may help determine whether information in the content presentation, such as a sleep history, helps the user understand their sleep state and how they can improve their sleep. This may include asking questions in one survey or a series of smaller surveys about whether the user understands their sleep animal independent of the advanced indicators, whether the advanced indicators are confusing, which other sleep animals may be relevant to the user, whether the user thinks they should have a different sleep animal, whether a certain type of chart or display is useful and easy to understand, whether the user clearly understands the difference between their sleep state and that of other users, etc. Such surveys may help improve the bio content or presentation of users overall based at least in part on the aggregated feedback, but may also help customize bios for specific users so as to present them with a selection of information that will be most useful to them and present that information in a way that is useful and appealing to the user. As discussed above, sleep histories may help users engage with their sleep on a new level. Presentation of this information may help users better understand their type of sleeper and what they can do to sleep better.

[0041] In some embodiments, a short survey or just one question may be presented at a particular time to identify a particular activity. For example, the question may ask the user if they are engaged in one or more activities, such as eating, drinking, or smoking, that may affect their ability to fall asleep quickly. Any audio, camera, or sensor data that may help identify such activities may also be used if available and permitted by the user. This may include, for example, a smart refrigerator that may indicate that the door has just been opened or food has been removed, or a smoke detector that detects at least some smoke, etc. Some activities, such as browsing the web or listening to a podcast, may be determinable through a questionnaire or by communicating with the device being used for the activity. Also, one or more questions may be presented to inquire about the user's perception. This may include, for example, whether the user is tired, stressed, anxious, anxious, wide awake, depressed, etc. The perception data, while ultimately not completely accurate, may help to understand the user better and correlate perception with sleep patterns. This may also include asking about other health information, such as whether the user is still, in any pain, injured, diabetic, pregnant, has problems with bladder control, etc. It may attempt to gather information about sleep apnea, seizures, medical conditions, or medication intake (or prescriptions). The inquiries may relate to other issues that are not directly related to health but may be useful in making a determination, such as whether the user is a shift worker or otherwise has a particular sleep schedule, whether the user is in an environment where it is difficult to sleep at certain times, whether the user is a parent of small children who may wake up frequently or irregularly throughout the night, etc. This information may also help make better suggestions to the user, as the user may be less likely (or more likely) to perform certain activities or certain actions when tired, depressed, stressed, etc.The user may also be able to provide information as to whether or not they currently think they are closer to another sleeping animal, which may be used to better understand user perception. , a suggestion or action may be tailored by inferring that by selecting a particular sleeping animal, the user is essentially providing a perceived value of one or more sleep metrics that predominate for the other sleeping animal. Any or all of this information may also be used to adjust the weighting of one or more factors in the decision, thereby more closely matching the user to a particular sleeping animal, or matching that animal with more certainty or confidence.

[0042] In some embodiments, the sleep history can provide additional information as to why a user is associated with a particular sleeping animal or health type. For example, information can be presented that identifies a defining metric for each sleeping animal based on the distribution of advanced sleep metrics at, as an example, the 25th, 50th, and 75th percentiles. The defining metrics can be the "peaks" and "valleys" on a radar chart, so that they are typically high or low metrics for the 25th or 75th percentiles. Additional information can also be provided for users determined to be edge cases. These users have sleep characteristics that are "in between" sleeping animals, or have similar characteristics for two different sleeping animals without clearly matching one sleeping animal. For example, if a user has both high "awakens per hour" and "minutes to fall asleep" values, the user may be classified as a kangaroo or a turtle. To help determine the appropriate sleeping animal or health type for a user, a post-clustering decision can be made based on one or more heuristics that calculate different distance metrics based at least in part on advanced sleep metrics in a radar chart, or other such multi-dimensional sleep space. In at least one embodiment, such a distance metric may consider whether a user has similar "peaks" and "valleys" of advanced sleep metrics compared to the advanced sleep metrics associated with sleeping animals. For example, a kangaroo's peaks may be an "awakens per hour" metric, so the distance metric may weight the "awakens per hour" metric more heavily to ensure that a user must have a similar "awakens per hour" metric to be associated with a kangaroo.

[0043] Various distance heuristics can be used after clustering. These can include the use of metrics such as Euclidean distance with assignment based on centroid distance, as may be used in k-means clustering approaches. Other distance metrics can include Manhattan distance, Chebyshev distance, cosine similarity, Levenshtein or Hamming distance, Kendall tau distance, or weighted Manhattan distance, among others. In one implementation, weighted Manhattan distance is utilized after clustering to reward extreme percentiles. In this example, the final sleeping animal was determined in part by:

[0044] Final sleeping animal = the animal with the smallest weighted Manhattan distance from the advanced metrics of a "typical" sleeping animal derived from K-means clustering Additional indices describing something akin to sleep onset, relating to the time from bedtime to the last wake-light-wake cycle (light epoch must be less than 1 hour), were also generated and validated. Such an approach may resemble the latency to sustained sleep and may help identify people who suffer from insomnia.

[0045] 7 illustrates an example process 700 for soliciting a user's sleep information and making or suggesting one or more changes to improve sleep that may be utilized in accordance with various embodiments. For this and all other processes described herein, it should be understood that additional, alternative, or fewer steps may be performed in similar or alternative orders or in parallel within the scope of various embodiments, unless otherwise stated. Additionally, while sleep is used as an example for purposes of explanation, such It should be understood that the approach can be used for other physical or mental states, situations, or activities of one or more users within the scope of various embodiments. In this example, sleep-related data is collected 702 for a user over multiple sleep periods, such as over days, weeks, or months. The sleep-related data may include data directly related to the user's sleep, such as the time the user fell asleep and the time the user woke up, but may also include data that may be less related to aspects of sleep, such as the room temperature or amount of noise when the user was in a particular sleep state or transitioned between states. This data may be collected by any suitable method, such as receiving it by direct input or from a device having a sensor or processor to generate or provide the data. In at least some embodiments, at least some pre-processing of the data may be performed to remove noise, remove redundant data, select relevant data, perform normalization, etc. Data may also be obtained from multiple sources in multiple locations, such as manual input, device determination, sensor detection, etc.

[0046] At least a portion of this collected data may be used to determine a set of sleep metric values ​​for the user 704. These may include sleep metrics determined to be of particular importance to analyzing sleep, which may include, among others, percentage of REM time, average sleep time, average time to bed, variance in time to bed, average recovery length, average minutes to fall asleep, number of awakenings per hour, number of long awakenings, number of daytime naps per period, and average minutes of deep sleep. These user sleep metric values, as well as other potential metrics, may then be compared 706 to a set of clusters of values, each of which is associated with a sleeper type (e.g., sleeping animals). Each sleeper type may include values ​​determined to be representative of a user of that sleeper type, as may be determined using various approaches described and suggested elsewhere herein. One of these sleeper types may be determined 708 or otherwise selected as the one that most closely matches the determined user sleep metric using one or more distance heuristics, or another such approach. As the user sleep metrics change over time, the sleeper type (e.g., animals) determined for the user may also change. In this example, visualizations and information regarding the user's selected sleeper type, along with information regarding the user's individual variance from the average or expected value for that sleeper type, and other such information, may then be provided for presentation to the user 710. Information contrasting metrics of the selected sleeper type with metrics of other sleeper types may also be provided.

[0047] In addition to providing the user with information regarding the user's sleeper type and the user's individual sleep data, it may be desirable to provide the user with options or suggestions that may help improve the user's sleep or otherwise achieve another sleep-related goal. In this example, the user may be optionally requested 712 for additional information that may be related to the user's state, activity, situation, or perception, but that may be difficult or impossible to obtain as part of the sleep-related data otherwise collected, which may be relevant to making a sleep judgment or suggestion. As described above, this may include whether the user is currently sleepy or anxious, whether the user has engaged in a particular activity at one or more points in the day, etc. Using any or all of the sleep metrics, sleeper type, and sleep-related data or additional information, one or more changes that may help improve sleep or achieve another sleep-related goal may be determined 714. The changes may include changes in user actions or behavior, or may include changes in the operation or state of a device, system, or service that may affect the user's sleep. For each of these changes, a determination may be made 716 as to whether the change is a change that may be automated or automatically performed by the device, system, or service. If not, present the user with a suggestion to take a given action, modify a behavior, change the operating state of the device, or otherwise make a determined change. 718. If the changes can be automated, one or more instructions (or requests or calls, etc.) can be provided to one or more associated devices, systems, applications, modules, or services to make the determined changes or to perform tasks that effect or correspond to the determined changes 720. After providing the suggestions or sending the change instructions, the status and / or impact of the changes can be monitored to determine whether or to what extent the changes were made, whether a change in behavior or deviation from expected sleep occurred, etc. This information can then be provided along with the collected sleep-related data to update the user sleep data, which can result in changes to the determined sleeper type, profile, history, suggestions, or other such aspects as presented or suggested herein.

[0048] Various illustrative embodiments incorporate and consider objective physiological data acquired and recorded non-invasively through wearable monitoring device sensors, such as activity, sleep, and heart rate ("HR"). These physiological data variables and indicators may further include comparable biomarkers, such as the user's resting heart rate ("RHR") and / or other HR derived data, blood oxygen concentration (SpO2) levels, heart rate variability ("HRV"), sleep time and quality, exercise levels, weight, hemoglobin and water concentrations, and oil / lipid / collagen concentrations on the skin, among other options, to name just a few. Various systems and methods may additionally or alternatively utilize electrodermal measurements and information from other types of devices, such as from chest bands or electrocardiogram ("ECG") patches. Details and conditions of the user's interactions with the wearable monitoring device, smartphone, and / or other devices, such as how often the device is checked and how hard buttons are pressed, may also be incorporated and analyzed.

[0049] In illustrative embodiments, heart rate, sleep, and physical activity patterns are analyzed by one or more predictive model algorithms that may vary depending on factors such as system configuration and what physiological objective data is collected and analyzed. In particular, potential categories of objective physiological data points may include the most predictive ones, such as resting heart rate, heart rate variability, average number of steps per day, active minutes, average sleep schedule such as the user's average bedtime and wake-up time, length of time from going to bed to falling asleep, number of sleep interruptions, and sleep stages, including amount of "deep" sleep. In at least some embodiments, the user may wear a smartwatch or some other monitoring device that can capture at least a portion of the sleep-related data to determine length, quality, stage, etc.

[0050] The categories and types of indicators may be customized and the data may be collected passively so that the user does not have to take any action to send the data to a cloud or other network. One or more of the health-related indicators may be monitored over time to determine patterns or cycles of variation in the indicators that may be correlated with mental conditions. In particular, an analysis of how much variation there is among the various categories of inputs will be useful in performing the various types of analysis presented herein. This information may then be used to update the predictive model and update individual suggestions based at least in part on the current values ​​of those indicators for the user. Information about the predictions, and updates to the predictions, may be displayed to the user to help them plan for life events.

[0051] At least some of the physiological or other health data may come from multiple devices. For example, a user may have a “smart” ring that can provide accurate heart rate information, A wrist temperature sensor that measures both skin and ambient temperature, an oral temperature sensor, or earphones configured to provide accurate body temperature information may be worn. This information may then be used with other available data to attempt to generate more accurate results. For example, temperature data from only the earphones may be used for hand tracking, or since wrist temperature measurements may be more susceptible to changes in ambient temperature, etc., temperature data from the earphones, smart ring, and monitoring device may all be analyzed together and time-synchronized before removing temperature fluctuations due to external factors. Results may be averaged or otherwise collated, or if two of the three measurements match the fluctuations but the third does not, data from the third device may be removed from consideration over the fluctuation time. Data from other external devices may also be used within the scope of various embodiments. As another example, if data from blood testing machines, urine analyzers, etc. is available, hormone levels or other information about body chemistry may also be used to predict various conditions and time points. Data such as temperature may also be obtained from numerous other types of devices, which may include smart clothing, bed sheets, wearables, etc. Optical devices for measuring body properties such as chemicals in the skin through diffuse reflectance spectroscopy, photoacoustic effect, optical coherence tomography, diffuse optical tomography, time-gated spectroscopy, or spatial frequency domain imaging can also be used within the scope of various embodiments. Different aspects of the human body have different patterns that can be learned and applied to available data to make predictions as accurate as possible. As described herein, machine learning can be used to attempt to improve the accuracy of pattern recognition and classification over time.

[0052] It should be noted that the choice of sleep-related application, and even its version, may differ and be customized depending on the type of user device and its storage space and computing limitations. The number of graphical elements displayed may differ, for example, between a desktop monitor and a worn monitoring device, and the user movements required may also differ (i.e., mouse movements and clicks as opposed to one or more swipes on a smartwatch). High-resolution screens can display more information. Doing so, for example, for tablets and smartwatches, a portion of the touchscreen is pre-configured as a user input area, making the pixels responsive to touch operations. Of course, some interfaces that require a larger display area simply do not lend themselves to the small screens found on typical smartwatches or even mobile phones.

[0053] In one embodiment, the user's overnight heart rate recording can first be divided into various sleep stages, which may include light sleep, deep sleep, and rapid eye movement ("REM") sleep. If desired, HRV parameters and HR can then be calculated only for the sleep stages, rather than for the entire night. A model of RHR (or an index derived from HRV) calculated only over non-REM sleep intervals (e.g., light sleep and deep sleep) can be used in conjunction with a predictive model for the purposes described herein.

[0054] In another embodiment, a person's breathing rate can be extracted from the PPG signal. Breathing rate is measured, typically 12-20 breaths per minute. The average breathing rate for each night or series of nights can be extracted. The subject's activity level can also be tracked and used to correct for confounding factors such as strenuous exercise and other strenuous physical activity that may affect the resting HR and HRV parameters used in the predictive model.

[0055] Concurrent with the analysis and prediction in at least some embodiments, HR information, such as RHR, for the user can be monitored. As described above, this can be achieved by monitoring the user's HR information, such as RHR, during periods of sleep and after minimal periods of inactivity, using the monitoring device 102 as described herein. This may include obtaining RHR data for the user using one or more approaches as described and suggested in. If it is determined that no resting heart rate pattern information is available, the process may continue without utilizing such pattern information.

[0056] Again, these HR data and other indicators can be used to predict health problems, and in some embodiments, a combination of these indicators and approaches can be used to try to improve the accuracy of the prediction. In other embodiments, two or more measurements can be combined to try to improve the prediction, whether using predictions based on user input as described above, or based solely on measured or detected physical and health data. For example, in one embodiment, the user's HR information and blood or tissue chemistry can be used to examine mental status. For example, there may be variations in the concentration or number of red or white blood cells, or in the concentration or amount of hemoglobin, ferritin, serum iron, peripheral capillary oxygen saturation (SpO2), water, lipids, collagen, sebum, or other components typically found in a person's blood or skin tissue or skin surface. Temperature sensors can also be used to determine variations in body temperature.

[0057] A monitoring device according to various illustrative embodiments can use light emitters, sensors, and other components as described and suggested herein to perform non-invasive real-time measurements of hemoglobin and water content in a user's body (e.g., the ratio of hemoglobin to water, or the relative changes in hemoglobin or water concentration alone in blood and tissue). The amount of light absorption in human skin can vary with differences in hemoglobin and water concentration. This can be particularly noticeable for light having wavelengths in the infrared ("IR") or near-infrared spectrum. As hemoglobin concentration decreases, the amount of light absorption by hemoglobin decreases. The amount of light absorption also changes by different amounts based on changes in oxygen saturation, with the absorption differences being more significant at different wavelengths. Thus, in some embodiments, the monitoring device 102 may include a first light emitter at a first wavelength and a second light emitter at a second wavelength suitable for detecting variations in hemoglobin and water concentrations, while in other embodiments the device 102 may include a third light emitter at a second wavelength suitable for detecting variations in SpO2, and some tracking devices 102 may include both (or a light emitter assembly capable of selectively or simultaneously emitting in both target wavelength bands). In one embodiment, the monitoring device 102 may include two light emitting diodes (LEDs) with two wavelengths in the range of about 600-1000 nm for detecting changes in SpO2 and hemoglobin content, and another LED with a wavelength in the range of 1000-1500 nm for measuring variations in water content and in combination with one of the first two LEDs for measuring the hemoglobin to water ratio.

[0058] Similar to resting heart rate data, the values ​​of indices such as hemoglobin, hydration, and SpO2 may be cyclical. By monitoring how these indices vary over time in the user's body and how these variations correspond to mental state, the measurements of the indices may be used to predict the timing of health-related events and episodes. Changes in these parameters may also indicate other potential problems in the user's body, and in some embodiments, these changes in parameters may be used to suggest a visit to the doctor or other actions to be taken.

[0059] In some embodiments, the software application may ask the user questions in response to detected changes in the user's body. For example, a change in sleep patterns may be due to a change in location or stress. Other changes, such as new medications or exercise patterns, may also affect at least some of the measurements. By obtaining this information, the software can determine whether certain values ​​or time periods should be excluded, whether those values ​​should be weighted differently, etc. Within the scope of various embodiments, motion sensors or other One can also attempt to identify some of these factors using information available from activity tracking.

[0060] With respect to hemoglobin concentration, in one embodiment, the concentration within the user's body is measured using optical techniques such as near-infrared ("NIR") spectroscopy. The NIR approach can utilize a light emitter that emits radiation within the NIR spectrum, which may have wavelengths ranging from 780 nm to 2500 nm. NIR has the advantage of being able to penetrate deeper into the skin than other optical techniques. The portion of the radiation that is not absorbed can be reflected back to one or more detectors having sensors capable of detecting radiation at least in a corresponding wavelength band. The absorption data determined by the detectors can be analyzed using multivariate approaches such as principal component analysis ("PCA") or neural networks, among other such options, to determine information regarding the composition of blood within the subject's body. One or more optical dispersive elements may be used to isolate specific wavelengths for measurement. In one example, two detectors are used at different locations to attempt to account for skin surface artifacts and skin variations that may be due to compression or other external influences. The specific wavelengths (e.g., 900-1500 nm) used to measure hemoglobin and water concentrations may depend in part on the specific implementation and design. Because some devices have less sensitive detectors, some devices may be closer to the skin, and some devices may be some distance away, there may be a trade-off between penetration depth and sensitivity to concentration variations. Detectors made of different materials and therefore with different sensitivities and accuracies may also be used, which may include detectors made of silicon and indium gallium arsenide, among other such options. Devices according to various embodiments may also utilize multiple emitters with different emission wavelengths, or emitters that emit multiple wavelengths, and the like. In some embodiments, accuracy may depend in part on the orientation of the device relative to the skin, its proximity to the skin, or the compression of the skin by the device (such as by the tracker being worn snugly against the skin or compressed by another part of the body during sleep). Thus, in some embodiments, pressure sensors, cameras, or other sensors may be used to attempt to account for such factors or variations.

[0061] After physiological data is collected over time, it can be filtered to reduce noise and random fluctuations in the data that may be due to natural variations and external influences such as changes in exercise, diet, stress, and sleep. Other types of data processing can be used, as would be apparent to one of skill in the art in light of the teachings and suggestions contained herein. As mentioned above, in various embodiments, measurements can be taken during periods of rest or sleep where there is relatively little change in position over a period of time. Because changes in oxygen or hemoglobin signals can be caused by movement or position changes, periods of rest can provide more accurate or consistent results, or a representation of the body's true state independent of many external factors. In some embodiments, the monitoring or other associated device can utilize an accelerometer, altimeter, inertial sensor, or other such components to monitor movement, and the device may wait until the subject has been still (within an acceptable threshold amount of movement) for at least a determined period of inactivity before taking a measurement. This can provide sufficient time for levels to reach an equilibrium point, which can further improve accuracy in at least some embodiments.

[0062] The filtered data collected over time can be analyzed to determine or update the patterns found for each indicator. This may include inputting the physiological data into one or more predictive models to determine whether the physiological data (including any patterns found therein) correlates with one or more biomarkers related to a physical or mental condition. Biomarkers used for comparison to the physiological data are not limited to, and include, but are not limited to:

[0063] Low activity level (average total number of minutes per day when your heart rate is in the cardio or peak zone. The lower the value of this biomarker, the higher your risk of depression, for example) Inconsistent sleep (standard deviation of minutes of sleep per night or standard deviation of time to bed or wake up. The higher the value of this biomarker, the higher the risk of depression) High resting heart rate (your maximum (or average) resting heart rate. The higher the value of this biomarker, the higher your risk of depression) Short sleep latency (the average number of minutes between bedtime and the onset of sleep. The higher this biomarker, the lower the risk of depression) Low average daily steps (the higher the value of this biomarker, the higher the risk of depression) Long periods or long intervals of wakefulness during sleep (i.e. severely disturbed sleep. The higher the value of this biomarker, the higher the risk of depression) Low heart rate variability (the higher the value of this biomarker, the higher the risk of depression) Late bedtime (the higher the value of this biomarker, the higher the risk of depression) Long REM sleep latency (the higher the value of this biomarker, the higher the risk of depression) The predictive model can update the pattern information based on additional data to obtain more accurate pattern information. In some embodiments, the condition data may be weighted or attenuated to account for changes in the user's health, such as age and changes in hormone levels, such that more recent physiological data has a greater influence on the pattern determination. Current information may be sufficient to form a screening or initial analysis, but the predictive model becomes more accurate as additional information is received and analyzed.

[0064] Within the scope of various embodiments, physiological data can be analyzed and correlated using various algorithms and approaches. Information about the user's body can be obtained by monitoring devices or other such tracking devices that can be correlated with mental and physical state and health information. This can include information known about various individuals as a base pattern, but can also be updated or determined for specific people to provide more accurate predictions. The analysis, correlation, and determination can be performed by advanced signal processing methods, averaging or otherwise aggregating data obtained over additional time periods, and / or using machine learning algorithms, among other such options. This can be done by providing data on the rhythm of the physiological data. A prediction can be generated based on any patterns determined using the physiological data. Additionally, as changes in RHR information are determined over time, the prediction can be updated, for example, if the RHR begins to indicate the onset of a depression or anxiety state.

[0065] The data can provide input to a predictive modeling or machine learning process that can use the information to predict future mental or physical states. As mentioned above, there can be a variety of physiological data inputs that can relate to heart rate, activity, sleep, and others described herein. Patterns can be determined and used for each type of data available to attempt to arrive at a more accurate decision. Data values ​​may be weighted differently, such as based on the strength or accuracy of the prediction, among other such factors. These weights can be updated or modified over time, such as based on machine learning or changes in the user's body or condition. There may also be different confidence levels or other factors that can affect the relative weightings. The weight values ​​selected may also depend on the signal-to-noise ratio of some signals.

[0066] In general, with machine learning techniques (and by extension) one or more systems can be trained on a set of indicators, physiological data and / or other indicators for a particular user or the general population. Physiological data and other indicators are then captured from a particular user and analyzed by the trained system to determine relationships between the indicators for that user, and then mental states. Health status and / or guidance can be pushed to the user.

[0067] For example, a machine learning system using a convolutional neural network ("CNN") can be designed to extract HR metrics during sleep. The CNN can be trained on a dataset, such as during different sleep stages, to determine the optimal sleep stage. A long short-term memory neural network ("LSTM"), hidden Markov model, or other time series model can be designed to predict state events based on previous history, which model can also consider any of the appropriate variables described herein. Multiple LSTM models can be trained to predict different factors related to mental or physical health in various embodiments. Furthermore, applying feed-forward, recurrent, radial basis function, modular, and / or self-organizing neural networks is certainly within the scope of the present disclosure.

[0068] As described, a pattern that is a function of both RHR and sleep data, or other such metrics, may be generated in some embodiments. In some embodiments, deep neural networks or other machine learning approaches may be used to "learn" patterns based on the obtained metrics, among other such information. A variety of other predictive modeling patterns and approaches may also be used, including those described and suggested herein. Correlations and patterns identified through predictive modeling may also be used to predict the upcoming timing of mental health related events, and other potentially related events. Pattern and correlation information may be updated in some embodiments as additional information is provided or obtained.

[0069] As discussed above, predictive modeling can use and apply various types of information that can affect different body data values. For example, information may be obtained regarding the amount of exercise or physical activity experienced by a person on a given day or over a period of time, which can explain differences in detected physiological data values, as described herein. Also, there may be variations in diet, stress, weight, body fat percentage, body mass index ("BMI"), medications, or other such factors, which can also be explained. In at least some embodiments, regressions can be applied after providing these and other such factors to one or more predictive modeling schemas to validate conditions for accurate predictions. If correlations with one or more biomarkers are found, at least potential health issues can be identified for the user. This process can continue and be repeated to update the user's condition status, and as additional types of information become available, this information can be added to the process for consideration. In at least some embodiments, the suggestions or weightings can change over time due to changes in body or activity levels, age, etc.

[0070] Once generated or updated, the health status can be output or published to the user or another appropriate or authorized entity. The health status information can be displayed in a variety of ways. There may be various options that the user can navigate through or there may be a specific interface or display provided, among other possibilities. In some embodiments, various user symptoms can be determined and the application can predict when those users will suffer from a health-related illness and deliver a corresponding notification to the user. A given application may also provide different views depending on the user's goals. In some embodiments, the app may further provide suggestions for improving health or achieving goals based at least in part on the monitored health information. If physiological and other data may indicate a potential medical condition, a visit to the hospital may also be suggested.

[0071] As discussed above, various embodiments may be realized as a system including one or more monitoring / tracking devices for a given user. In other embodiments, embodiments may be provided as a service that users can utilize on their devices. Also, other fitness trackers and health care providers may contract or utilize such services for their customers. In some embodiments, application programming interfaces ("APIs") or other such interfaces may be exposed that allow collected physiological data and other information to be received by a service, which can process the information and send the results back to the monitoring or associated computing device for access by the user. In some embodiments, at least a portion of the processing may occur on the monitoring or tracking device itself, although processing by a remote system or service may allow for more robust processing, especially for tracking devices with limited capacity or processing power.

[0072] As can be seen in the illustrative embodiment 800 shown with FIG. 8, light emitted from one or more light emitters 802 may be reflected back from the skin to the detector 804. While the user in various embodiments may wear a monitoring device such as a smartwatch or fitness tracker, or another PPG device near the wrist location, in other embodiments such a device may be worn in locations such as the ear, fingertip, ankle, neck, upper arm, torso, leg, and / or forehead (e.g., such that the light source of the PPG device is adjacent to a human blood vessel). To account for skin surface artifacts and skin variations that may be due to compression or other external influences, two detectors 804 are used in this embodiment at different locations.

[0073] The path that light travels from the light emitter 802 to the skin and back to one of the detectors 804 can be referred to as an "optical path." In addition to having its usual meaning, optical path can refer to the probabilistic path of a photon from one location to another, typically from a light source (or light emitter) to a light sensor (or detector). Photons emitted by the light emitter 802 will take many different paths to each detector 804. For simplicity and clarity, the path resulting from the optical power weighted average of all possible paths is simply referred to as an optical path in some embodiments. In some alternative embodiments, "optical path" refers to the path taken by the majority of the photons. In still other embodiments, "optical path" refers to an approximate vector that originates at the center of the light source and terminates somewhere on the surface area of ​​the detector 804, representing the approximate path of light from the light emitter 802 to the detector 804.

[0074] An optical path refers to the approximate path of light from a given light emitter source 802 to a given detector 804, so that, for example, if there are multiple light emitters 802 and multiple detectors 804, there will be a separate optical path between each of the multiple light sources and each of the multiple detectors. Consistent with embodiments described herein, PPG signals associated with any of the above-mentioned optical paths may be selectively acquired and utilized to estimate HR and / or other physiological indices. For example, PPG signals corresponding to any of the multiple paths may be compared using a quality / confidence metric such as a signal-to-noise ratio ("SNR"), and the highest quality PPG signal may be selected to be used to estimate HR and / or other physiological data.

[0075] 9 illustrates components of an example computing device 900 that may be utilized in accordance with various embodiments. In this example, the monitoring or tracking device includes at least one processor 902, such as a central processing unit ("CPU") or a graphics processing unit ("GPU"), for executing instructions that may be stored in a memory device 904, which may include flash memory or DRAM, among other such options. As will be apparent to one of ordinary skill in the art, the device may include a processor 902, a graphics processing unit 904, a memory device 904 ... The device may include many types of memory, data storage, or computer readable media, such as data storage for program instructions executed by the device. The same or different storage may be used for images or data, removable memory may be available to share information with other devices, and any number of communication approaches may be available to share with other devices. The device typically includes some type of display 906, such as a touch screen, organic light emitting diode ("OLED"), or liquid crystal display ("LCD"), although the device may communicate information by other means, such as through audio speakers or a projector.

[0076] A monitoring device or similar tracking device includes at least one motion detection sensor, which may include at least one I / O element 908 as shown. This type of sensor can determine and / or detect the orientation and / or movement of the device. Such elements may include, for example, an accelerometer, inertial sensor, altimeter, or gyroscope operable to detect the device's movement (e.g., rotational movement, angular displacement, tilt, position, orientation, or movement along a non-linear path). The orientation determination element may further include an electronic or digital compass, which may indicate the direction (e.g., north or south) in which the device is determined to be pointing (e.g., relative to a major axis or other such aspect). The device may further include an I / O element 908 for determining the location of the device (or a user of the device). Such positioning elements may include or comprise a Global Positioning System ("GPS") or similar location determination element operable to determine the relative coordinates of the device's location. The positioning elements may include wireless access points, base stations, etc., which may broadcast location information or may be able to determine the location of the device by signal triangulation. Other positioning elements may include a QR code, a barcode, an RFID tag, an NFC tag, or the like that allows the device to detect and receive location information, or an identifier that allows the device to obtain location information (e.g., by mapping the identifier to a corresponding location). Various embodiments may include one or more such elements in any suitable combination. I / O elements 908 may further include one or more biometric sensors, light sensors, barometric sensors (e.g., an altimeter), and the like.

[0077] As mentioned above, some embodiments track the location and / or motion of the user using elements. Upon determining an initial location of the device (e.g., using GPS), the device may track the location of the device by using the above elements, or in some examples, by using orientation determining elements or combinations thereof as described above. As will be appreciated, the algorithm or mechanism used to determine the location and / or orientation may depend at least in part on the selection of elements available to the device. An example device further includes one or more wireless components 910 operable to communicate with one or more electronic devices within communication range of a particular wireless channel. The wireless channel may be any suitable channel used to allow the device to communicate wirelessly, such as Bluetooth, cellular, NFC, or Wi-Fi channels. It should be appreciated that the device may have one or more conventional wired communication connections as known in the art. The device further includes one or more power components 912, which may include a battery operable to recharge through a conventional plug-in approach or through other approaches, such as capacitive charging by being in close proximity to a power mat or other such device. In some embodiments, the device may include at least one additional input / output device 908 capable of receiving conventional input from a user. This conventional input may include, for example, push buttons, a touch pad, a touch screen, a wheel, a joystick, a keyboard, a mouse, a keypad, or other such devices or elements that allow a user to input commands into the device. These I / O devices 908 may, in some embodiments, include wireless infrared or Bluetooth or other input devices. may also be connected by other links. Some devices may further include a microphone or other audio capture element to accept voice or other audio commands. For example, a device may not include any buttons at all, but may be controlled only through a combination of visual and audio commands, allowing a user to control the device without having to come into contact with it.

[0078] As mentioned above, many embodiments include at least some combination of one or more light emitters 914 and one or more detectors 916 for measuring data of one or more indicators of the human body, such as for a person wearing a tracker device. In some embodiments, this may include at least one imaging element, such as one or more cameras capable of capturing images of the surrounding environment as well as imaging the user, people, or objects in the vicinity of the device. The image capture element may include any suitable technology, such as a CCD image capture element having sufficient resolution, focal length, and visibility range to capture an image of the user as the user operates the device. Methods of capturing images using a camera element with a computing device are well known in the art and will not be described in detail herein. It should be understood that image capture may be performed using one image, multiple images, periodic imaging, continuous image capture, image streaming, etc. Additionally, the device may include the ability to start and / or stop image capture, such as upon receiving a command from a user, an application, or another device.

[0079] In keeping with the optically oriented description herein, an example device in the system of FIG. 9 includes a light emitter 914 and a detector 916 that can be used to obtain optical photoplethysmography ("PPG") measurements. Some PPG techniques rely on detecting light at a single spatial location, while others rely on adding signals taken from two or more spatial locations. Both of these approaches result in a single spatial measurement from which an HR estimate (or other physiological data) can be determined. In some embodiments, the PPG device uses a single light source 914 (i.e., a single optical path) coupled to a single detector 916. Alternatively, the PPG device may use multiple light sources 914 (i.e., two or more optical paths) coupled to a single detector or multiple detectors 916. In other embodiments, the PPG device uses multiple detectors 916 (i.e., two or more optical paths) coupled to a single light source or multiple light sources 914. In some cases, the light source 914 may be configured to emit one or more of green light, red light, and / or infrared light. For example, a PPG device may use a single light source 914 and two or more light detectors 916, each configured to detect a particular wavelength or wavelength range. In some cases, each detector 916 is configured to detect a different wavelength or wavelength range from the other detectors. In other cases, two or more detectors 916 are configured to detect the same wavelength or wavelength range. In yet other cases, one or more detectors 916 are configured to detect a particular wavelength or wavelength range that is different from one or more other detectors. In embodiments using multiple light paths, the PPG device may average the signals obtained from the multiple light paths before determining an HR estimate or other physiological indicator. Such PPG devices may not be able to resolve the individual light paths or to separately utilize the individual signals obtained from the multiple light paths.

[0080] In some embodiments, a user wearing a monitoring device with PPG functionality may perform activities that involve motion (or, in the case of a wrist-worn PPG device, affecting the dynamics of blood flow in the wrist, e.g., by twisting the wrist). In such instances, the accuracy of the HR estimate provided by the PPG device may be reduced or impaired. The light intensity received by the photodetector 916 may be modulated by these motions, which are typically an order of magnitude or more larger than the desired cardiac signal. Therefore, a pre-processing step in which the effects of these motion signals are removed may be utilized to improve the accuracy of the HR estimation during motion. In addition to the adverse effects of motion, another source of signal quality degradation in PPG devices may be the characteristics of the local area being sensed. For example, moving a wrist-worn PPG sensor a few millimeters on the wrist may significantly change the signal quality. Furthermore, during motion, certain parts of the wrist-worn PPG device may experience greater motion depending on its location, position, and / or orientation, and thus PPG sensors located in such parts may experience greater motion-induced degradation of the PPG signal.

[0081] Various embodiments allow the PPG device to utilize signals based on two or more independently addressable light source-detector combinations, such that the signal quality of the PPG device improves, especially during activities involving motion. In some embodiments, the PPG signal can be obtained via multiple optical paths including one or more light sources and one or more detectors located at different spatial locations. These multiple PPG signals can then be processed to separate cardiac components from the PPG signal (e.g., by removing the motion component). For example, the motion component can be removed based on input from an accelerometer, unsupervised learning and / or previously performed supervised learning. Additionally or alternatively, by comparing the PPG signals corresponding to these multiple optical paths using a quality metric, the highest quality PPG signal can be selected for estimating HR or other physiological metrics, and sleep time or other potential aspects.

[0082] To filter out the motion signal utilizing two or more light source-detector pairs, a PPG device according to various embodiments can use a computer program to identify the motion component of a given signal and remove the motion component from the composite signal, leaving only the cardiac signal as residual. In some implementations, the time phase of the cardiac waveform is assumed to be constant between different light paths, but the phase of the motion signal is expected to differ between light paths depending on how the PPG sensor interacts with the skin surface during motion activities (e.g., pressure at the PPG / skin interface may differ depending on the spatial location of the light source and photodetector of the light path). Using this concept, the PPG device can fit a mathematical model to the spatial light path signal to identify the cardiac and motion components. First, the PPG signal is extracted by each light source-detector combination. For example, two light sources 914 and two light detectors 916 result in four light source-detector combinations. Then, a mathematical model can be fitted to different spatial points from which characteristic signals related to the cardiac and motion components of the PPG signal are extracted. PPG devices may also implement other techniques, including, but not limited to, independent component analysis ("ICA") and other forms of blind source separation.

[0083] Although some embodiments are described with reference to HR or cardiac components of PPG signals, the techniques described herein may be extended to other types of physiological data described herein that may relate to SpO2 or other types of signals that may be extracted from PPG signals to determine physiological data or indicators. For example, in some embodiments, the method of determining an SpO2 value includes receiving a first set of one or more PPG signals from one or more PPG sensors 916, the first set of PPG signals may include analog signals, or digital data sampled from the analog components and stored in computer memory. The first set of PPG signals may correspond to red and / or infrared light previously emitted by the one or more light emitters 914 after the emitted light interacts with the user's skin when the user is wearing the monitoring device. The first set of PPG signals may include a noise component. The method of determining an SpO2 value may further include receiving a second set of one or more PPG signals from one or more PPG sensors or detectors, the second set of PPG signals may include analog signals, or digital data sampled from the analog components and stored in computer memory. For example, the second set of PPG signals may be derived from a range of wavelengths emitted by a different light source 914 than the first set of PPG signals. In some cases, the second set of PPG signals is obtained from a system in the device used to track the user's heart rate. In other cases, the second set of PPG signals is received from a system separate from the HR detection. The method of determining an SpO2 value may further include generating a filtered set of PPG signals by filtering the first set of PPG signals based on the characteristics of the second set of PPG signals. Various filtering techniques may be used to remove noise or other features from the first set of PPG signals based on the characteristics of the second set of PPG signals. By way of example only, HR may be a characteristic of the second set of PPG signals. In the case of HR, the device may create a filter based at least in part on the detected frequency of the HR signal. Examples of filters include low pass filters, high pass filters, and narrow band filters that filter out frequencies that do not match the frequency of the HR signal. The method of determining an SpO2 value may further include using a wavelength range to better measure the underlying signal on which the wavelengths of the first set of PPG signals operate. Based on this underlying signal (or features derived therefrom), the device can improve the first set of PPG signals based on filtering noise from the first set of PPG signals. Further, the filtered set of PPG signals can be used to create and store SpO2 values. As an example, the filtered set of PPG signals can be a more accurate basis for creating and storing SpO2 values ​​because noise components may have been reduced or eliminated.

[0084] In some embodiments, an intermediate HR estimation can be performed based on PPG signals from two or more optical paths. For each acquired PPG signal, the PPG device may determine an estimate of HR as beats per minute ("BPM") and calculate a confidence index associated with the PPG signal that indicates the signal quality of the particular optical path associated with the PPG signal. It may also be possible to calculate the confidence index without intermediate HR estimation, for example via characteristics (e.g., statistics) of the PPG signal or a filtered version of the PPG signal. In some embodiments, each confidence index corresponds to a single PPG signal. In other cases, each confidence index corresponds to multiple PPG signals. As a specific example, a confidence index may be calculated for each way of combining PPG signals (e.g., signals A+B, signals A+C, signals B+C, and signals A+B+C) and for various combinations of PPG signals (e.g., selecting at least two of signals A, B, and C). In other cases, one confidence index corresponds to a single PPG signal and another confidence index corresponds to a combination of multiple PPG signals. The PPG device may select one HR estimate from the multiple HR estimates corresponding to the multiple optical paths (e.g., by selecting the HR estimate of the PPG signal with the highest confidence index). Alternatively, the PPG device may assign different weight values ​​to the multiple HR estimates based on confidence index values ​​associated with individual and / or multiple PPG signals and calculate a final HR estimate based on the weight values. As with other aspects of the present disclosure, the confidence values ​​and / or weight values ​​may be updated or optimized using machine learning. The PPG device may implement hysteresis logic to prevent jumps between optical paths within a short time window when confidence index values ​​corresponding to two optical paths are within a threshold. The PPG device may also implement logic configured to bias the selection of the HR estimate based on user data, activity data, motion data, or other data accessible to the PPG device. The PPG device may apply a smoothing filter to the HR estimates, for example to improve accuracy and provide a better user experience.

[0085] One advantage of such an approach is that spatial information associated with the light source 914 and / or the light detector 916 can be used by various algorithms to improve the accuracy of the PPG sensing device's estimation of HR or other physiological indicators, especially when the user of the device is exercising or engaged in an activity involving movement. Existing implementations typically rely on algorithms to improve the performance of estimating HR or other physiological indicators. However, it does not have the advantage of redundant sensor data that is generated based on multiple light paths.

[0086] With reference to FIG. 9, an example monitoring device may include one or more processors 902 coupled to a memory 904, a display 906, a bus, one or more input / output (I / O) elements 908, and a wireless networking component 910, among other such options. The display 906 and / or the I / O device 908 may be omitted in certain embodiments. If included, the display 906 may provide an interface for displaying data such as the user's HR, blood oxygen saturation (SpO2) level, and other metrics. For example, the processor 902 may calculate values ​​of physiological metrics monitored by the device based on one or more PPG signals generated by a light detector 916. In one embodiment, the PPG device is a wristband and the display is configured such that the display 906 is on the outside of the user's wrist when the user wears the PPG device. In other embodiments, the display 906 may be omitted and data detected by the PPG device may be transmitted using a wireless networking interface over at least one network 918 via near field communication ("NFC"), Bluetooth, Wi-Fi, or other suitable wireless communication protocol to a host computer 920 for analysis, display, reporting, or other such uses.

[0087] The memory 904 may comprise a RAM, ROM, FLASH memory, or other non-transitory digital data storage and may include a control program comprising a series of instructions that, when loaded from the memory and executed by the processor 902, cause the processor 902 to perform the functions described herein. The light emitter 914 and the detector 916 may be coupled to a bus, either directly or indirectly using a driver circuit 922, by which the processor 902 may drive the light emitter 914 to obtain a signal from the light detector 916. The host computer 920 may communicate with the wireless networking component 910 via one or more networks 918, which may include one or more local area networks, wide area networks, and / or the Internet using either terrestrial or satellite links. In some embodiments, the host computer 920 executes a control program and / or an application program configured to perform some of the functions described herein.

[0088] In some embodiments, each light emitter 914 can be individually controlled, or each light detector 916 can be individually read out if multiple detectors 916 are used, and in such embodiments, PPG sensor data along several different optical paths can be collected. The control program can utilize the collected data to provide more accurate estimates, or values, of HR and / or other physiological indices. In a related aspect, the processor 902 and other components of the PPG monitoring device may be implemented as a system-on-chip ("SoC"), which may include one or more CPU cores using one or more reduced instruction set computing ("RISC") instruction sets, and / or other software and hardware supporting the monitoring device.

[0089] In various embodiments, the light emitters (or light sources) 914 include electronic semiconductor light sources such as LEDs, or generate light using either filaments, phosphors, or lasers. In some implementations, each of the light sources 914 emits light having the same central wavelength or within the same wavelength range. In other cases, at least one light source 914 may emit light having a different central wavelength than another one of the light sources 914. The central wavelength of the light emitted by the light sources 914 may be in the range of 495 nm to 570 nm. For example, a particular green light source 914 may emit light having a central wavelength of 528 nm. In other embodiments, one or more of the light sources 914 may emit red light (e.g., central wavelength 660 nm) or IR light (e.g., central wavelength 940 nm). In some embodiments, one or more of the light sources 914 may emit light having a peak wavelength typically in the range of 650 nm to 940 nm. More specifically, the red light source 914 may emit light having a peak wavelength of 660 nm, and one or more of the infrared light sources 914 may emit light having a peak wavelength in the range of 750 nm to 1700 nm. By way of example and not of limitation in any way, a particular infrared light source 914 may emit light having a peak wavelength of 730 nm, 760 nm, 850 nm, 870 nm, or 940 nm. In some cases, commercially available light sources such as LEDs may provide output at approximately 20 nm intervals with a center wavelength tolerance of ±10 nm from the manufacturer's specified wavelength, so one possible range of useful peak wavelengths for the light sources is 650 nm to 950 nm. The green light source 914 may be configured to emit light having a wavelength in the range of 495 nm to 570 nm. For example, a particular green light source 914 may emit light having a wavelength of 528 nm. The green light source 914 may be equally spaced from the photodetector 916, as may the pair of red and infrared light sources 914. For example, if the distance between the photodetector 916 and the center of the first red light source 914 is 2 mm, the distance between the photodetector 916 and the green light source 914 may also be 2 mm (e.g., equidistant). In some other cases, the distance between the photodetector 916 and one or more light sources 914 is not equidistant. Further, in some embodiments, one or more of the light sources 914 may include a single LED package that emits multiple wavelengths, such as green, red and infrared wavelengths, at the same or substantially the same (e.g., less than 1 mm apart) location relative to the multiple detectors 916. Such an LED may include multiple semiconductor elements co-located with a single die in a single package.

[0090] The spacing of the light sources 914 may be measured from a side of the light source 914 or from the center of the light source 914. For example, the light sources 914 may be configured such that the center of each light source 914 is a first distance from an edge of the nearest light detector 916 of the light detectors 916. In an illustrative embodiment, the first distance may be 2 mm. In some implementations, each light source 914 is located a second distance from the nearest light source 914 of the light sources 914, and each light detector 916 is located a third distance from the nearest light detector 916 of the light detectors 916. In some embodiments, the second and third distances are the same as the first distance. In other embodiments, each of the second and third distances is different from the first distance. The second distance may be the same as or different from the third distance. The particular amount of spacing may depend on numerous factors, and this disclosure does not limit the embodiments to any particular spacing. For example, spacings ranging from 1 mm (or less) to 10 mm would be viable in various embodiments.

[0091] In some embodiments, independent control of all light sources 914 is provided. In other embodiments, several light sources 914 are controlled together as a gang or bank. An advantage of independent control of each light source 914, or independent readout from each of the multiple detectors 916 (e.g., obtaining independent signals based on the same or different wavelengths of light from each of the multiple detectors), is that a multiple light path approach can be used to improve estimation of HR and / or other physiological indices, as described herein.

[0092] The light detector 916 may include one or more sensors adapted to detect wavelengths of light emitted from the light source 914. A combination of a particular light source 914 and a particular detector may include a sensor, such as a PPG sensor. The first and second PPG sensors may share components, such as the same light source 914 and / or detector 916, or may have different components. Thus, the term "PPG sensor" may refer to any of such arrangements, in addition to having its ordinary meaning. Actual embodiments may use multiple components in implementing a PPG sensor. The term "PPG device" may refer to any device that includes a PPG sensor, in addition to having its ordinary meaning. The photodetector 916, in one embodiment, may include one or more detectors 916 for detecting each of the different wavelengths of light used by the light source 914. For example, a first detector 916 may be configured to detect light having a wavelength of 560 nm, a second detector 916 may be configured to detect light having a wavelength of 940 nm, and a third detector 916 may be configured to detect light having a wavelength of 528 nm. Examples include photodiodes made of semiconductor materials and having optical filters that allow only light of a particular wavelength or range of wavelengths to enter. The photodetector 916 may include any of the following: a photodiode, a phototransistor, a charge-coupled device ("CCD"), a thermopile detector, a microbolometer, or a complementary metal-oxide semiconductor ("CMOS") sensor. The photodetector 916 may include multiple detector elements, as further described herein. One or more of the detectors 916 may include a bandpass filter circuit.

[0093] In other embodiments, the detector 916 may include one or more detectors 916 configured to detect multiple wavelengths of light. For example, a single detector 916 may be configured to tune to different frequencies based on data received from an electronic digital microprocessor coupled to the detector. Alternatively, a single detector 916 may include multiple active areas, each active area sensitive to a given range of wavelengths. A single detector 916 may be configured to detect light having wavelengths in the red and IR frequencies, and a second detector 916 may be configured to detect light having wavelengths in the green frequencies. Additionally, each of the light sources 914 may use any of one or more different wavelengths of light, as previously described.

[0094] In one embodiment, the photodetector 916 may be mounted in a housing having one or more filters configured to filter out wavelengths of light other than those emitted by the light source 914. For example, a portion of the housing may be covered with a filter that filters out ambient light other than the wavelengths of light emitted by the light source 914. The signal from the light source 914 may be received by the photodetector 916 through an ambient light filter that filters out ambient light sources that produce ambient light having wavelengths different from those detected by the detector 916. Although LEDs and photodiodes are used as examples of the light source 914 and the photodetector 916, respectively, the techniques described herein may be extended to other types of light sources, such as edge-emitting lasers, surface-emitting lasers, and LED-pumped phosphors that produce broadband light. And the techniques may be extended to other combinations of light sources and detectors. For example, a PPG device may include (i) a single or multiple LEDs and a multi-element photodetector (e.g., a camera sensor), (ii) an LED array and single or multiple photodiodes, (iii) a broadband LED-excited phosphor and detector array with wavelength-selective filters on each detector, (iv) a spatial light modulator ("SLM") (e.g., a digital micromirror device ("DMD")), or (v) a liquid crystal on silicon ("LCoS") device and single or multiple LEDs, other combinations thereof, or other configurations of light sources and detectors.

[0095] Although certain flow descriptions and diagrams are provided herein to illustrate various methods that may be performed by the example embodiments, these are merely illustrative of example algorithms that may be programmed using any suitable programming environment or language to create machine code executable by the CPU or microcontroller of the PPG device. In other words, the flow diagrams, together with the written description herein, are algorithmic disclosures of aspects of the claimed subject matter and are presented at the same level of detail typically used to communicate this subject matter among those of ordinary skill in the art to which this disclosure pertains. Various embodiments may be coded using assembly, C, Objective C, C++, Java, or other human readable languages ​​and then The program may be compiled, assembled, or otherwise translated into machine code that may be loaded into a ROM, EPROM, or other recordable memory of a monitoring device that is coupled to a CPU or microcontroller and then executed by the CPU or microcontroller.

[0096] In one embodiment, a computer application or program may be used to process the PPG signals obtained from multiple optical paths to identify motion components of the signals by filtering or filtering out signal components associated with user motion, and to remove the identified motion components from the composite signal, leaving the cardiac components as a residual or final signal. PPG signals may be collected at various activities during the day or night, which may be associated with periods of walking, exercise, or sleep. Other on-device sensors, including accelerometers, gyroscopes, or altimeters, may be used to classify or detect activities, or human postures, as a basis for developing appropriate filters. These filters or signal processing methods may be used for targeted reduction of variability in PPG data with multiple optical paths. By way of example and not limitation, accelerometer data may be used to develop signal processing methods that filter the PPG data and filter out other body orientations to look for specific postures. This may help reduce noise in the PPG data and obtain a better assessment of the corresponding physiological variables of the corresponding optical paths.

[0097] In various embodiments, the approaches described herein may be performed by one or more of firmware running on a monitoring device or a secondary device, such as a mobile device, paired with the monitoring device, a server, a host computer, etc. For example, the monitoring device may perform operations related to generating a signal that is uploaded or otherwise communicated to a server that performs operations to remove the motion component and to create a final estimate of HR, SpO2, and / or other physiological data indicators. Alternatively, the monitoring device may perform operations related to generating a monitoring signal and removing the motion component to create a final estimate of HR, SpO2, and / or other physiological indicators local to the monitoring device. In this case, the final estimate may be uploaded or otherwise communicated to a server, such as a host computer, that performs other operations with the value.

[0098] An example monitoring or tracker device, such as a device having the components shown in FIG. 9, may collect one or more types of physiological and / or environmental data from one or more sensors and / or external devices and communicate or relay such information to another device (e.g., a host computer or another server) to enable viewing of the collected data, for example, using a web browser or a network-based application. For example, a monitoring device may perform biometric tracking by calculating and storing a user's step count using one or more sensors while worn by the user. The monitoring device may transmit data representing the user's step count to an account on a web service (e.g., www.fitbit.com), a computer, a mobile phone, and / or a health station, where the data may be stored, processed, and / or visualized by the user. As discussed above, a monitoring device may measure or calculate many other physiological data metrics in addition to or in lieu of a user's step count. Again, such physiological data may include energy expenditure (e.g., calories burned), floors climbed and / or descended, HR, heart rate waveform, HR variability, HR recovery, respiration, SpO2, blood volume, blood glucose, skin hydration, skin pigmentation level, location and / or orientation (e.g., via GPS, Global Navigation Satellite System (GLONASS), or similar systems), altitude, walking speed and / or distance traveled, swimming lap counts, swimming style and detection counts, cycling distance and / or speed, skin conductance, skin temperature and / or body temperature, and as measured by electromyography. The parameters may include, but are not limited to, muscle condition measured by electroencephalography, brain activity measured by electroencephalography, weight, body fat, caloric intake, nutrient intake from food, medication intake, sleep period (e.g., time of day, sleep phase, sleep quality, and / or duration), pH level, hydration level, respiratory rate, and / or other indicators.

[0099] An example device for monitoring or tracking may measure or calculate (e.g., with one or more environmental sensors) metrics related to the user's surrounding environment, such as, for example, air pressure, weather conditions (e.g., temperature, humidity, pollen count, air quality, rain / snow conditions, wind speed), exposure to light (e.g., ambient light, ultraviolet (UV) exposure, time, and / or duration spent in darkness), noise exposure, radiation exposure, and / or magnetic fields. Additionally, the monitoring device (and / or a host computer and / or another server) may collect data from one or more sensors of the device and calculate metrics derived from such data. For example, the monitoring device may calculate a user's stress or relaxation level based on a combination of HR variability, skin conductance, noise pollution, and / or sleep quality. In yet another example, the monitoring device may determine the effectiveness of a medical intervention, such as based on a combination of data related to medication intake, sleep, and / or activity. Again, the examples herein are provided for illustrative purposes only and are not intended to be limiting or exhaustive.

[0100] An example monitoring device may further include a computer-readable storage medium reader, a communication device (e.g., a modem, a network card (wireless or wired), and / or an infrared communication device), and a working memory 904 as described above. The computer-readable storage medium reader may be connected to or configured to receive computer-readable storage media representing remote, local, fixed and / or removable storage devices and storage media for temporarily and / or more permanently containing, storing, transmitting and retrieving computer-readable information. The monitoring system and various devices typically further include a number of software applications, modules, services, or other elements located in at least one working memory device 904, including operating systems and application programs such as client applications or web browsers. It should be recognized that alternative embodiments may have numerous variations from the above. For example, customized hardware may also be used and / or specific elements may be realized in hardware, software (including portable software such as applets), or both. Additionally, connections to other computing devices such as network input / output devices may be used.

[0101] Storage media and other non-transitory computer readable media for storing code or portions of code may include any suitable media known or used in the art, including, but not limited to, volatile and non-volatile media, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Volatile and non-volatile media, removable and non-removable media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks ("DVD") or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media that can be used to store the desired information and that can be accessed by a system device.

[0102] Based on the disclosure and teachings provided herein, one of ordinary skill in the art will be able to contemplate various embodiments. It will be appreciated that other means and / or methods of implementation are to be recognized. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be apparent that various modifications and changes can be made therein without departing from the broader spirit and scope of the invention as set forth in the appended claims.

Claims

1. 1. A method for determining a sleeper type of a user wearing a wearable computing device, the method comprising: one or more computing devices configured to communicate with the wearable computing device determine one of the plurality of different sleeper types as the sleeper type of the user based at least in part on the user's values ​​for each of a plurality of sleep metrics over a period of time and the values ​​of the plurality of sleep metrics for each of a plurality of different sleeper types; generating, by the one or more computing devices, one or more suggestions for the user based at least in part on the determined sleeper type of the user; causing the one or more computing devices to display a graphical object indicative of the determined sleeper type of the user; causing the one or more computing devices to display the one or more suggestions; said one or more computing devices collecting sleep-related data for a population of users; determining, by the one or more computing devices, the plurality of sleep metrics from the sleep-related data; and wherein the one or more computing devices perform clustering of the plurality of sleep metrics to generate a set of clusters corresponding to the plurality of different sleeper types.

2. The method of claim 1 , wherein the plurality of different sleeper types comprises six different sleeper types.

3. The method of claim 1 or 2, wherein the plurality of sleep metrics comprises eleven different sleep metrics.

4. The method according to any one of claims 1 to 3, wherein said graphical objects include sleeping animals.

5. 5. The method of claim 1, further comprising causing the one or more computing devices to perform a graphical comparison of the values ​​of the plurality of sleep indicators for the user with values ​​of the plurality of sleep indicators for an average user of another sleeper type among the plurality of different sleeper types.

6. 5. The method of claim 1, further comprising causing the one or more computing devices to perform a graphical comparison of the values ​​of the plurality of sleep indicators for the user with values ​​of the plurality of sleep indicators for an average user of all other sleeper types among the plurality of different sleeper types.

7. The method of any one of claims 1 to 6, wherein the one or more computing devices generating one or more suggestions to the user based at least in part on the determined sleeper type of the user further comprises the one or more computing devices communicating one or more commands to adjust one or more devices in the user's surrounding environment.

8. 1. A wearable computing device, comprising: A display device; one or more biometric sensors; at least one processor, the at least one processor comprising: and configured to obtain sleeper type data of a user wearing the wearable computing device, the sleeper type data indicative of one of a plurality of different sleeper types being determined at least in part based on the user's values ​​for each of a plurality of sleep indicators over a period of time and the values ​​of the plurality of sleep indicators for each of the plurality of different sleeper types, and the at least one processor further comprises: Obtaining one or more suggestions for the user based at least in part on the sleeper-type data of the user; causing the display device to display a graphical object indicative of the determined sleeper type of the user; configured to cause the display device, or a display device of an electronic device communicatively coupled to the wearable computing device, to display the one or more suggestions based at least in part on the sleeper-type data of the user; A wearable computing device, wherein the plurality of different sleeper types are determined by using a clustering algorithm to generate a set of clusters corresponding to the plurality of different sleeper types.

9. 9. The wearable computing device of claim 8, wherein the plurality of sleep metrics include at least one of: (i) a time spent asleep for each of a plurality of sleep events occurring during the period; (ii) a percentage of rapid eye movement (REM) occurring during each of the plurality of sleep events; or (iii) a duration of each of the plurality of sleep events.

10. The wearable computing device of claim 9 , wherein the period is 28 days.

11. The at least one processor further comprises: A wearable computing device as described in any one of claims 8 to 10, configured to provide a graphical comparison of the values ​​of the plurality of sleep indicators for the user with values ​​of the plurality of sleep indicators for an average user of another sleeper type among the plurality of different sleeper types.

12. The at least one processor further comprises: A wearable computing device as described in any one of claims 8 to 10, configured to provide a graphical comparison of the values ​​of the plurality of sleep indicators for the user with values ​​of the plurality of sleep indicators for an average user of all other sleeper types among the plurality of different sleeper types.

13. The wearable computing device of any one of claims 8 to 12, wherein the one or more biometric sensors include an optical sensor configured to acquire photoplethysmogram (PPG) data of the user.

14. A program comprising instructions executable by a computing device, the instructions comprising: determining one of the plurality of different sleeper types as the user's sleeper type based at least in part on the user's values ​​for each of a plurality of sleep metrics over a period of time and on the values ​​of the plurality of sleep metrics for each of a plurality of different sleeper types; generating one or more suggestions for the user based at least in part on the determined sleeper type of the user; displaying a graphical object indicative of the determined sleeper type of the user; displaying the one or more suggestions to the user based at least in part on the determined sleeper type of the user; and Collecting sleep-related data for a population of users; determining the plurality of sleep metrics from the sleep-related data; and and performing clustering of the plurality of sleep metrics to generate a set of clusters corresponding to the plurality of different sleeper types.

15. The instructions further include:

15. The program of claim 14, further comprising: providing a graphical comparison of the values ​​of the plurality of sleep metrics for the user with values ​​of the plurality of sleep metrics for an average user of another sleeper type among the plurality of different sleeper types.

Images