Head-mounted wearable devices providing indication of received and monitored sensor data and methods and systems for use thereof

Abstract

Methods and systems are disclosed for coordinating the display of biometric data at a head-mounted wearable device based on sensor data from a wrist wearable device. A method includes receiving an indication that a user of a head-mounted wearable device is performing a physical activity. The head-mounted wearable device includes a light emitting diode that is visible to a user when the user wears the head-mounted wearable device, and the head-mounted wearable device communicates with a wrist wearable device worn by the user. The wrist wearable device is configured to sense biometric data of a user during physical activity. The method includes, after receiving the indication and while the user is performing an activity, determining that the biometric data satisfies a physiology-based threshold indicating that information about the biometric data will assist the user in performing a physical activity, the head-mounted wearable device is caused to present information about the biometric data via the light emitting diode.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 341,390, filed on May 12, 2022, entitled “Head-Worn Wearable Device Providing Indications of Received and Monitored Sensor Data, and Methods and Systems of Use Thereof,” which is incorporated herein by reference. Technical Field

[0003] The present application generally relates to communicating information to a user via a head-mounted wearable device, and more particularly, to coordinating the display of sensor data to a user at a head-mounted wearable device based on sensor data from a wrist-mounted wearable device. Background Art

[0004] Users who perform physical activities usually carry many electronic devices to assist them in performing physical activities. For example, users may carry fitness trackers, smart phones, or other devices including biometric sensors that track the performance of users during exercise. In order to view the biometric data they sense, users are usually required to interrupt, pause, or otherwise end their exercise to view the collected data (e.g., by having to overlook the fitness tracker or search, and then unlock another device to view the data). In addition, traditional wearable devices including displays require users to lift their devices and / or physically interact with the wearable device to view the sensed data, which reduces the user's experience and may affect the user's ability to perform effective exercise while viewing biometric data. In addition, because traditional wearable devices require user interaction (e.g., input for unlocking the device, input for searching the device, then unlocking the device, and then accessing applications including biometric data and other interactions), users cannot easily access and use the sensed data to improve their physical activity performance.

[0005] Furthermore, the use of artificial reality devices and systems to assist in exercise activities is still in its early stages and has not yet been accepted or even tried by many consumers. Therefore, there is a need to explore how to present data (e.g., biometric data) from one device at an artificial reality device to assist in exercise activities, and to do so in a manner that promotes further adoption of such devices and systems.

[0006] Therefore, there is a need for a wearable device that coordinates the display of sensor data to a user without distracting the user or requiring user interaction. Summary of the invention

[0007] In order to circumvent one or more of the disadvantages or challenges discussed above, the following artificial reality system (e.g., including a head-mounted wearable device) is disclosed: the artificial reality system coordinates the display of sensor data received from one or more communicatively coupled devices (e.g., wrist wearable devices). The head-mounted wearable device (which may also be more simply referred to as a "head-mounted device") presents sensor data to a user without using a head-up display or an overhead display. More specifically, the head-mounted wearable device displays sensor data via an illumination source (e.g., a light-emitting diode (LED)) that does not obstruct the user's field of view. In some embodiments, the head-mounted wearable device detects when a user is performing a physical activity and requests sensor data (e.g., biometric data, location data, orientation data, movement data, etc.) related to the physical activity from the one or more communicatively coupled devices. Alternatively, in some embodiments, the head-mounted wearable device receives an indication from the one or more communicatively coupled devices that the user is performing a physical activity and receives sensor data from the one or more communicatively coupled devices. In some embodiments, the head-mounted wearable device continuously monitors the sensor data to determine whether the user is performing a physical activity. Alternatively, in some embodiments, the one or more communicatively coupled devices periodically provide sensor data to the head-mounted wearable device to determine whether the user is performing physical activity.

[0008] The head-mounted wearable device uses received and / or monitored sensor data to determine whether the sensor data meets a physiological-based threshold indicating that the representation of the sensor data will assist the user in performing physical activity. For example, the head-mounted wearable device may determine that the sensor data indicates that the user is running at a target pace, the user is within a target heart rate zone, the user has reached a target lactate threshold, etc. Based on determining that the sensor data meets the physiological-based threshold indicating that the representation of the sensor data will assist the user in performing physical activity, the head-mounted wearable device may present information about the biometric data that will assist the user in performing physical activity via at least one light-emitting diode of the head-mounted wearable device. The light-based electrodes may emit light in different colors, different frequencies, and different patterns, and may be used to convey different information to the user.

[0009] The head-mounted wearable device can be used in a real-world environment and / or an artificial reality (AR) environment, including but not limited to a virtual-reality (VR) environment (including a non-immersive VR environment, a semi-immersive VR environment, and a fully immersive VR environment), an augmented reality environment (including a marker-based augmented reality environment, a markerless augmented reality environment, a location-based augmented reality environment, and a projection-based augmented reality environment), a mixed reality, and other types of mixed reality environments. For example, the head-mounted wearable device can provide a variable light-based representation of changes in the user's physical activity performance when the user is performing an activity outdoors (e.g., running), or when the user is participating in an AR game (e.g., a virtual fitness game, a horror game, a role-playing game, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order that the present disclosure may be understood in more detail, a more specific description may be given by reference to the features of various embodiments, some of which are shown in the accompanying drawings. However, the accompanying drawings only show the relevant features of the present disclosure. As will be understood by those skilled in the art when reading the present disclosure, the description may admit other effective features.

[0011] Figures 1A to 1D Coordinating the display of biometric data at a head-mounted wearable device based on sensor data from another device is shown in accordance with some embodiments.

[0012] FIG. 2A to FIG. 2D Coordinating the display of exercise guidance information to a user of a head-mounted wearable device based on data other than biometric data is shown in accordance with some embodiments.

[0013] Figure 3 is a flow chart illustrating a method of coordinating display of data at a head mounted wearable device based on sensor data from another device in accordance with some embodiments.

[0014] Figure 4 is a detailed flow chart illustrating a method of coordinating the display of biometric data according to some embodiments.

[0015] FIG. 5A to FIG. 5E An example wrist wearable device is shown in accordance with some embodiments.

[0016] Fig. 6A and Figure 6B An example AR system is shown in accordance with some embodiments.

[0017] Figure 7A system of one or more devices for providing indications of biometric data and other sensor data is shown in accordance with some embodiments.

[0018] As is customary, the various features shown in the drawings are not drawn to scale. Therefore, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of these drawings may not depict all of the components of a given system, method, or apparatus. Finally, throughout the specification and drawings, like reference numerals denote like features. DETAILED DESCRIPTION

[0019] Numerous details are described herein in order to provide a thorough understanding of the example embodiments illustrated in the accompanying drawings. However, some embodiments may be practiced without many of these specific details, and the scope of the claims is limited only by those features and aspects specifically recited in the claims. In addition, well-known processes, components, and materials are not described in detail to avoid obscuring relevant aspects of the embodiments described herein.

[0020] The embodiments of the present disclosure may include various types of artificial reality systems or embodiments of artificial reality systems, or may be implemented in combination with various types of artificial reality systems or embodiments of artificial reality systems. As described herein, artificial reality (AR) is a presentation provided by an artificial reality system in the user's physical environment, any superimposed functions and / or senses detectable. Such artificial reality may include and / or represent virtual reality (VR), augmented reality, mixed artificial reality (MAR), or some combination and / or a variation of these. For example, a user may make an air sliding gesture to skip a song through a song providing an application programming interface (API), which provides playback at, for example, a home speaker. The AR environment described herein includes, but is not limited to, a VR environment (including a non-immersive VR environment, a semi-immersive VR environment, and a fully immersive VR environment); an augmented reality environment (including a marker-based augmented reality environment, a markerless augmented reality environment, a location-based augmented reality environment, and a projection-based augmented reality environment); mixed reality; and other types of mixed reality environments.

[0021] Artificial reality content may include fully generated content or generated content combined with captured (e.g., real-world) content. Artificial reality content may include video, audio, tactile events, or some combination thereof, any of which may be presented in a single channel or in multiple channels (e.g., stereoscopic video that produces a three-dimensional effect to the viewer). In addition, in some embodiments, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof, which are used, for example, to create content in artificial reality and / or otherwise be used in artificial reality (e.g., to perform activities in artificial reality).

[0022] Gestures as described herein may include air gestures, surface contact gestures, and / or other gestures that can be detected and determined based on the movement of a user's single hand or a combination of both hands. In some embodiments, air means that the user's hand does not touch a surface, object, or part of an electronic device (e.g., head wearable device 110 or other communication-coupled device (e.g., wrist wearable device 120)), in other words, the gesture is made in an open space in 3D space and does not touch a surface, object, or electronic device. More generally, surface contact gestures (contact at a surface, object, body part of a user, or electronic device) are also considered, where contact (or contact intention) is detected at the surface (e.g., a single-finger or double-finger tap on a table, on a user's hand or another finger, on a user's leg, sofa, steering wheel, etc.). The different gestures disclosed herein may be detected using image data and / or sensor data (e.g., neuromuscular signals sensed by one or more biopotential sensors (e.g., electromyography (EMG) sensors) or other types of data from other sensors (e.g., proximity sensors, time-of-flight sensors, sensors from inertial measurement units, etc.)), which are detected by a wearable device worn by a user and / or other electronic devices owned by the user (e.g., a smartphone, a laptop computer, an imaging device, an intermediate device, and / or other devices described herein).

[0023] Figures 1A to 1D 1 shows coordination of display of biometric data at a head-mounted wearable device based on sensor data from another device in accordance with some embodiments. In some embodiments, the head-mounted wearable device 110 is communicatively coupled to the wrist wearable device 188 and / or another device (e.g., see below). Figure 7In some embodiments, the head-mounted wearable device 110 is communicatively coupled to more than one device (e.g., a wrist wearable device 188, a smartphone 774b, a personal computer (PC) 774a, a tablet, a workout tracking device, a server 770, or other device described herein). In some embodiments, the head-mounted wearable device 110 is communicatively coupled to more than one device (e.g., a wrist wearable device 188, a smartphone 774b, an independent biometric sensor (or other health tracking device), and / or other devices). The head-mounted wearable device 110 includes a light emitting diode (LED) 127 for transmitting different messages to a user 130 of the head-mounted wearable device 110 and / or other persons near the user 130 (e.g., within at least 5 to 10 meters) via an active variable light-based representation. In some embodiments, the LED 127 is coupled to a housing of the head-mounted wearable device 110 and / or one or more lenses 111 of the head-mounted wearable device 110. In some embodiments, one or more LEDs 127 can be positioned at different locations along the head-mounted wearable device 110. For example, the LED 127 is one of a plurality of LEDs positioned along the head-mounted wearable device 110.

[0024] The head-mounted wearable device 110 is configured to receive an indication that the user 130 (while wearing the head-mounted wearable device 110) is performing a physical activity, and after receiving the indication, receive data (e.g., biometric data) sensed by a sensor (e.g., a biometric sensor) of the wrist wearable device 188 and / or a sensor of another device during the user's physical activity. Additionally or alternatively, in some embodiments, the head-mounted wearable device 110 monitors data (e.g., biometric data) via one or more sensors included in the head-mounted wearable device 110. The biometric data may include at least hydration data, oxygen level data (e.g., blood oxygen saturation (SpO2)), heart rate data (e.g., resting heart rate and heart rate variability (HRV)), pressure data, skin or body temperature data, ambient temperature data, and the like. In some embodiments, the head-mounted wearable device 110 is configured to monitor the following items or receive (for example, from the wrist wearable device 188 or other devices communicatively coupled to the head-mounted wearable device 110) the following items via one or more sensors: position data (for example, location, altitude, distance traveled, head position, distance between devices, etc.), movement data (for example, speed, acceleration, number of repetitions, number of steps, arm swings, etc.), orientation data (for example, device position and / or user position sensed by an inertial measurement unit), posture data (for example, the user's position relative to the head-mounted wearable device 110). In some embodiments, the position data, orientation data, and / or movement data are used to determine posture data (for example, whether the user is upright, hunched, hunched, etc.). Refer to the following Figure 7Additional examples of different devices communicatively coupled to the head mounted wearable device 110 and sensors for collecting data are described.

[0025] In some embodiments, the head-mounted wearable device 110 is configured to determine a variable light-based representation of changes in the user's physical activity performance based on sensor data monitored by the head-mounted wearable device 110 and / or received from the wrist wearable device 188 or other device. For example, the head-mounted wearable device 110 can determine a variable light-based representation of changes in the user's physical activity performance based on monitored and / or received biometric data, location data, movement data, posture data, etc. In some embodiments, the variable light-based representation of changes in the user's physical activity performance is based on satisfying one or more physiologically based thresholds.

[0026] Physiologically based thresholds include one or more of the following: hydration threshold (e.g., user 130 has consumed at least 64 ounces (oz) of water, the ratio of water intake to sweating of user 130 is within a predetermined range, etc.); speed / pace threshold; oxygen level threshold; heart rate zone threshold or cardiovascular zone threshold; stress threshold; posture threshold, etc. In some embodiments, one or more physiologically based thresholds are: determined based on the user 130's physical activity history; defined by the user 130; and / or dynamically adjusted while performing physical activity. For example, if the user 130 wants to be notified during their workout that they have reached a specific heart rate, the user 130 can define one or more physiologically based thresholds so that the head-mounted wearable device 110 provides a variable light-based representation via the LED 127 to notify the user 130 that they have reached their target heart rate. Additionally, if the user 130 wants to keep their heart rate within a particular zone, the head-mounted wearable device 110 can provide a variable light-based representation of the user's physical activity performance via LED 127 to notify the user 130 that the heart rate is within, above, or below a particular zone.

[0027] The variable light-based representation of changes in the user's physical activity performance is configured to help the user 130 complete physical activities, train, achieve target performance, break personal records, maintain correct and safe posture, etc. In some embodiments, the variable light-based representation of changes in the user's physical activity performance changes dynamically as the user performs activities, performs their workouts, performs different physical activities, etc. The variable light-based representation may include, but is not limited to, strobe light, steady light, changing colors, user-defined light patterns (e.g., three quick flashes, two quick flashes followed by a delayed third flash, etc.) or colors, communication modes such as Morse code, etc. The head-mounted wearable device 110 is configured to provide instructions to the LED 127 for generating a variable light-based representation in response to satisfying one or more physiologically based thresholds. As Figure 1A As shown, user 130 starts exercising while using wrist wearable device 188 and head-mounted wearable device 110. When user 130 starts exercising (e.g., running), head-mounted wearable device 110 receives biometric data from wrist wearable device 188 (including one or more biometric sensors for sensing biometric data) and causes LED 127 to emit light in different patterns, different colors, different frequencies, etc. (thereby providing a variable light-based representation) to convey different messages to the user when the user is performing physical activities.

[0028] For example, the head-mounted wearable device 110 can receive biometric data (including heart rate) from the wrist wearable device 188 or other device communicatively coupled to the head-mounted wearable device 110, and use the biometric data to determine the color, frequency, and / or pattern of the LED 127 emitting light. The head-mounted wearable device 110 can determine the color, frequency, and / or pattern of the LED 127 emitting light based on the heart rate (e.g., HR1, HR2, HR3) of the user 130 and one or more physiological-based thresholds of the user 130. More specifically, based on determining that the biometric data satisfies a physiological-based threshold indicating that the representation of the biometric data will assist the user in performing physical activity, the head-mounted wearable device 110 causes the LED 127 to provide a variable light-based representation of the user 130's physical activity performance corresponding to the physiological-based threshold that is met. The above examples are non-limiting. As will be understood by those skilled in the art upon reading the description provided herein, different biometric data or devices can be used to determine the one or more physiological-based thresholds that are met. For example, in some embodiments, duration (e.g., t1, t2, t3, t4), distance, altitude, oxygen (O2) statistics, skin temperature, ambient temperature, user posture or other ergonomic indicators, hydration, speed / pace, pressure, etc. can be used to determine whether one or more physiologically based thresholds are met.

[0029] In some embodiments, LEDs 127 are configured so that only user 130 can view (e.g., presented without obstructing user 130's field of view (e.g., field of view 135)) the variable light-based representation. This allows user 130 to monitor their performance in real time and make any needed adjustments to improve their performance, correct their posture to avoid injury, control their pace or energy level applied to assist user 130 in completing a workout, and / or make any other improvements to the performance of a physical activity. In some embodiments, LEDs 127 are configured so that user 130 and others in the vicinity of user 130 can view the variable light-based representation. This allows user 130 to share their current performance with friends and / or mentors (e.g., providing a visual representation of user 130's current physiological state to an instructor or personal trainer to guide a workout), as well as enabling the user to communicate with others (e.g., to notify others that the user does not want to be disturbed, or to notify others that user 130 is injured or needs help, etc.). In some embodiments, LEDs 127 are configured so that only others in the vicinity of user 130 can view the variable light-based representation. This allows user 130 to provide busy or do not disturb notifications to others, notify others that assistance is not needed, notify others that assistance is needed, etc. In some embodiments, user 130 can select whether the variable light-based representation should be visible only to them, visible to them and nearby people, or visible only to nearby people. Figures 1A to 1D Different examples of variable light based representations are provided.

[0030] exist Figure 1A , user 130 is shown starting their workout (or physical activity). The physical activity of user 130 is represented by a first heart rate graph 150a, which includes at least three physiological-based thresholds (HR Threshold 1, HR Threshold 2, and HR Threshold 3) of the heart rate (e.g., HR1, HR2, and HR3) of user 130. As user 130 engages in their workout, wrist wearable device 188 (or other device communicatively coupled to head-mounted wearable device 110) monitors biometric data of user 130 and provides the sensed biometric data to head-mounted wearable device 110. At t1, head-mounted wearable device 110 determines that the heart rate of user 130 meets the first physiological-based threshold (HR Threshold 1) and provides a first variable light-based representation of the changes in the user's physical activity performance (e.g., first variable light frequency 115). Specifically, LED 127 emits light at frequency f1, thereby prompting the user that they have reached the first target heart rate threshold (e.g., HR Threshold 1).

[0031] exist Figure 1B, user 130 is shown in the middle of their workout. The physical activity of user 130 is represented by a second heart rate graph 150b. At t2, the head-mounted wearable device 110 determines that the heart rate of user 130 meets a second physiological-based threshold (HR Threshold 2), and provides a second variable light-based representation of changes in the user's physical activity performance (e.g., second variable light frequency 117). Specifically, LED 127 emits light at frequency f2, thereby prompting the user that they have reached the second target threshold (e.g., HR Threshold 2). In some embodiments, the head-mounted wearable device 110 can have multiple LEDs (e.g., 127, 128, 129, and 131). In some embodiments, one or more of the multiple LEDs (e.g., 127, 128, 129, and 131) are located at the same position or different positions of the head-mounted wearable device 110. Each of the one or more of the multiple LEDs can be independently controlled so that each LED can provide a variable light-based representation with a corresponding color, frequency, pattern, etc.

[0032] In some embodiments, a corresponding illumination source such as an LED or a display may also be used to enable the wrist wearable device 188 (or other device communicatively coupled to the head-mounted wearable device 110, see below). Figure 7 In some embodiments, the wrist wearable device 188 (or other device communicatively coupled to the head-mounted wearable device 110) is configured to illuminate with the variable light-based representation when the user 130 focuses on the wrist wearable device 188 (or other device). For purposes of this disclosure, focusing means that the user 130 is interacting with the device and / or looking at the device. For example, as shown in FIG. Figure 1B As shown, when the user 130 looks at the wrist wearable device 188 (e.g., represented by the wrist wearable device 188 being within the field of view 135), the wrist wearable device 188 emits light at a frequency f2 (e.g., the watch variable light frequency 157), thereby prompting the user that they have reached a second target threshold (e.g., HR threshold 2). In some embodiments, the illumination source of the wrist wearable device 188 or other device communicatively coupled to the head-mounted wearable device 110 includes a display 720, an LED, or other light source of the corresponding device.

[0033] exist Figure 1C, user 130 is shown to be near the peak of his / her exercise. The physical activity of user 130 is represented by a third heart rate graph 150c. At t3, the head-mounted wearable device 110 determines that the heart rate of user 130 meets the third physiological-based threshold (HR threshold 3), and provides a third variable light-based representation of the changes in the user's physical activity performance (e.g., a third variable light frequency 119). Specifically, LED 127 emits light at a frequency f3, thereby prompting the user that they have reached the third physiological-based threshold. In some embodiments, the variable light-based representation provided by the wrist wearable device 188 (or other device communicatively coupled to the head-mounted wearable device 110) is also updated based on changes in the user's biometric data. When the user focuses on the wrist wearable device 188 (or other device), the wrist wearable device 188 (or other device) is caused to emit light using the updated variable light-based representation (e.g., at a frequency f3, as shown in the watch variable light frequency 159).

[0034] In some embodiments, LED 127 is caused to emit light toward other people near user 130. The user may need help; if the user does not need help, the signal can be used to signal to others that the user does not want to be disturbed; and / or the user shares their workout with friends. As an example, in Figure 1C In one embodiment, the user is deep into their workout and may be fatigued or dehydrated. The head mounted wearable device 110 may determine, based on the received or monitored biometric data, that a physiological-based threshold associated with fatigue and / or dehydration has been met, and in response to determining that the physiological-based threshold associated with fatigue and / or dehydration has been met, cause LED 127 to provide a variable light-based representation to those near the user indicating that the user needs assistance.

[0035] exist Figure 1D , user 130 is shown to be during a relaxation exercise period. The physical activity of user 130 is represented by a fourth heart rate graph 150d. At t4, the head-mounted wearable device 110 determines that the heart rate of user 130 has returned to a first physiological-based threshold (HR Threshold 1), and provides a first variable light-based representation of the changes in the user's physical activity performance (e.g., a first variable light frequency 115). Specifically, LED 127 emits light at frequency f1, thereby prompting the user that they have reached the HR1 threshold. In some embodiments, the wrist wearable device 188 (or other device communicatively coupled to the head-mounted wearable device 110) may also emit light at the same frequency (f1) or a similar frequency as described above.

[0036] FIG. 2A to FIG. 2DFIG. 1 shows coordination of display of exercise guidance information to a user of a head-mounted wearable device based on data other than biometric data according to some embodiments. The head-mounted wearable device 110 includes one or more components and is configured to perform the above referenced Figures 1A to 1D One or more functions described. Any data sensed by one or more sensors of the head-mounted wearable device 110 and / or an electronic device (e.g., wrist wearable device 188) communicatively coupled to the head-mounted wearable device 110 can be used to determine whether a physiological-based threshold is met. In some embodiments, the head-mounted wearable device 110 is also configured to provide guidance to the user based on the sensed, monitored and / or received sensor data. The guidance can include an LED emitting light using a specific color, pattern and / or frequency to assist the user in performing physical activities. In some embodiments, the guidance includes providing audio feedback (e.g., via a speaker of the head-mounted wearable device 110 or a speaker communicatively coupled to the head-mounted wearable device 110).

[0037] exist Figure 2A , user 130 is shown exercising and using sensory data displayed by a head-mounted wearable device to assist in performing physical activities (e.g., lifting a heavy weight 205). One or more sensors of the head-mounted wearable device and / or wrist wearable device 188 are used to acquire and / or monitor data while the user is performing physical activities. For example, when user 130 is exercising, wrist wearable device 188 (and / or as described below in reference Figure 7 The head-mounted wearable device 110 (and / or other communicatively coupled devices described herein) monitors position and orientation data of the user 130 and provides the sensed position and orientation data to the head-mounted wearable device 110. The head-mounted wearable device 110 (and / or other communicatively coupled devices) can determine whether the user's position and orientation data meets a physiological-based threshold (e.g., posture threshold 210). Although the above examples describe the use of position and orientation data sensed by one or more sensors of the wrist wearable device 188, upon reading the description provided herein, those skilled in the art will understand that the position and orientation data sensed by one or more sensors of the head-mounted wearable device can be used to determine whether the position and orientation data of the user 130 meets a physiological-based threshold.

[0038] The posture thresholds 210 may include one or more thresholds for providing guidance regarding the posture of the user 130. For example, the posture thresholds 210 may include a first posture threshold 217 for determining whether the user is upright, with a straight back, and / or otherwise maintaining a safe exercise posture, and a second posture threshold 219 for determining whether the user is hunched, slouched, and / or has other unsafe or high-risk (injury) postures.

[0039] Based on determining that one or more physiological-based thresholds are met, the head-mounted wearable device 110 is caused to illuminate the LED 127 to assist the user and / or provide guidance to the user when performing physical activities. For example, the head-mounted wearable device 110 can determine that the current posture 215 of the user 130 is within the first posture threshold 217 and cause the LED 127 to emit light at the first variable light frequency 115 (f1) to prompt the user 130 that they are maintaining a safe posture (e.g., not hunched or hunched).

[0040] exist Figure 2B , user 130 is shown further performing their exercise. User 130 continues to use the sensory data displayed by the head-mounted wearable device to assist in performing physical activities. Figure 2B As shown, the head-mounted wearable device 110 determines that the current posture 215 of the user 130 is still within the first posture threshold 217, and causes the LED 127 to continue to emit light at the first variable light frequency 115 (f1) to prompt the user that they are maintaining a safe posture. In some embodiments, if it is determined that the user 130 is focused on the wrist wearable device 188 (or other device), the illumination source of the wrist wearable device 188 is caused to emit light using the same variable light-based representation (e.g., at the frequency f1 as shown in the watch variable light frequency 257) or using a different variable light-based representation, thereby prompting the user that they are maintaining a safe posture.

[0041] Go to Figure 2C , showing that the user 130 is further performing their exercise. Specifically, it is shown that the user feels tired and / or fatigued when performing physical activities, which causes the posture or exercise posture of the user 130 to change from a safe posture to a high-risk posture. The change in the posture of the user 130 is detected by one or more sensors of the head-mounted wearable device 110 and / or a device communicatively coupled to the head-mounted wearable device 110. When it is determined that the current posture 215 of the user 130 meets the second posture threshold 219, the head-mounted wearable device is caused to illuminate the LED 127 with another variable light frequency 229 (f4) to prompt the user 130 that their posture has changed from a safe posture to a high-risk posture. As described above with reference to Figure 2B As described, in some embodiments, the illumination source of the wrist wearable device 188 is caused to emit light with the same variable light-based representation (e.g., watch variable light frequency 259 with frequency f4) or with a different variable light-based representation, thereby prompting the user 130 of their current posture 215.

[0042] Alternatively or additionally, in some embodiments, the head-mounted wearable device 110 (or other device communicatively coupled to the head-mounted wearable device) is caused to provide audio feedback via one or more speakers 223. The audio feedback provides guidance to the user 130 to assist the user 130 in performing physical activities. In some embodiments, the guidance provides suggestions to the user 130, such as suggesting that the user 130 take a break, reduce weight, replenish water, stand up straight, correct their posture, etc. In some embodiments, auditory feedback is provided for positioning guidance. In some embodiments, auditory feedback is presented to the user 130 only when the user 130 is wearing headphones communicatively coupled to the head-mounted wearable device 110. In this way, the privacy of the user 130 is protected by not providing audio feedback when other people are near the user 130 (for example, in the case where the user 130 does not want the exercise guidance to be announced using the speakers). Refer to the following Figure 4 Additional information about one or more speakers is provided.

[0043] exist Figure 2D , the user 130 corrects their posture according to the guidance provided by the head-mounted wearable device 110. In particular, the user has reduced the exercise weight (e.g., from the heavier weight 205 to the lighter weight 225). The reduction in the exercise weight allows the user 130 to correct their posture so that the current posture 215 of the user 130 is within the first posture threshold 217. As the user 130 corrects their posture, the LED 127 of the head-mounted wearable device emits light at the first variable light frequency 115 (f1), thereby prompting the user 130 that they are exercising in a safe posture (e.g., without hunching or slouching).

[0044] Above Figures 1A to 2D The example instructions described in are non-limiting. In some embodiments, LED 127 is illuminated to provide directions, instructions to the user, operate as a timer, and provide any other additional information to the user to assist in performing physical activities. For example, when a user is jogging outdoors, the LED can be configured to illuminate once to instruct the user 130 to turn left, illuminate twice to instruct the user 130 to turn right, illuminate at a steady color (e.g., blue) to instruct the user 130 to continue moving forward, and illuminate red to alert the user to unsafe events (e.g., a reported nearby robbery). In some embodiments, various LEDs 127 located on different portions of the head-mounted wearable device are illuminated (e.g., an LED located on the left side of the head-mounted wearable device 110 is illuminated to instruct the user to turn left, an LED located on the right side of the head-mounted wearable device 110 is illuminated to instruct the user to turn right, etc.). As described above in FIG. 2A to FIG. 2D As described in , in some embodiments, the head-mounted wearable device is configured to provide audio feedback in conjunction with illuminating LED 127, or to provide audio feedback instead of illuminating LED 127.

[0045] Figure 3 300 is a flowchart illustrating a method for coordinating the display of data at a head-mounted wearable device based on sensor data from another device according to some embodiments. The operations (eg, steps) of method 300 may be referred to below. Figure 7 In some embodiments, the head-mounted wearable device 110 is connected to one or more sensors (e.g., the head-mounted wearable device 110 is connected to one or more sensors (e.g., the head-mounted wearable device 110 is connected to the wrist-mounted wearable device 188, etc.) Figure 7 In some embodiments, the head-mounted wearable device 110 is communicatively coupled to the display 720 ( Figure 7 ). Figure 3 At least some of the operations shown in correspond to stored in a computer memory or computer-readable storage medium (e.g., memory 760; Figure 7 ). The operations of method 300 may be performed by a single device (e.g., head-mounted wearable device 110) or at least two devices communicatively coupled to each other (e.g., head-mounted wearable device 110 and wrist wearable device 188, a smartphone, a computer, a server 770, etc.). Additionally or alternatively, the operations of method 300 may be performed by instructions stored in a device communicatively coupled to the above referenced device. Figure 7 The instructions in the memory or computer-readable medium of one or more devices described herein are executed.

[0046] The method 300 includes receiving (310) an indication that a user of a head-mounted wearable device is performing a physical activity. The indication may be received from a wrist wearable device 188 or any other device communicatively coupled to the head-mounted wearable device 110. For example, as described above with reference to Figures 1A to 1D As described, the head-mounted wearable device 110 can receive instructions at different points in the user 130's exercise. In some embodiments, the head-mounted wearable device 110 continuously receives instructions while the user 130 is performing physical activity. In some embodiments, the head-mounted wearable device 110 receives instructions at predetermined intervals (e.g., 1 second, 10 seconds, 30 seconds, etc.).

[0047] The method 300 includes receiving (320) sensor data while the user is performing a physical activity. During the user's physical activity, sensor data from a communicatively coupled electronic device can be received at the head-mounted wearable device. For example, during Figures 1A to 1D, the wrist wearable device 188 provides biometric data from one or more sensors to the head-mounted wearable device 110. The head-mounted wearable device can receive any sensor data that can be used to determine whether one or more physiological-based thresholds are met, or any sensor data that can be used to assist the user in performing physical activities. The sensor data may include biometric data (e.g., oxygen level, heart rate, body temperature, etc.), position data (e.g., device (relative to the head-mounted wearable device 110) position data, positioning data, etc.), movement data (e.g., acceleration, speed, etc.), environmental data (e.g., temperature, altitude, etc.), and / or the following references Figure 7 Describes other sensor data.

[0048] The method 300 also includes determining (330) whether the information about the sensor data assists the user in performing physical activity. In response to determining that the information about the sensor data will assist the user in performing physical activity ("yes" at operation 330), the method 300 includes causing (340) the head-mounted wearable device to present the information about the sensor data via at least one LED. Figures 1A to 1D As described, the information about the sensor data can be LEDs 127 illuminated at different frequencies, different patterns, different colors, etc. In some embodiments, the head-mounted wearable device 110 does not include a display, a heads-up display, an overhead display, etc. In other words, the head-mounted wearable device 110 presents information without a display.

[0049] Information about sensor data can convey different messages to users, which users can use to improve the performance of their physical activities and / or assist users in performing physical activities. For example, information about sensor data can be light that emits in a green hue to inform users that they are within their target heart rate, within their target running pace, performing a target exercise type (e.g., aerobic vs. anaerobic), exercising with the correct posture (e.g., standing upright instead of hunching over), and are expected to break personal records, improved performance metrics (e.g., improved lactate threshold), etc. Similarly, information about sensor data can be light that emits in a yellow hue to inform users that they are no longer within their target heart rate, outside their target running pace, etc. In addition, information about sensor data can be light that emits in a red hue to inform users that they are dehydrated, at risk of injury (e.g., having an incorrect posture), at an unsafe heart rate for a long time, etc. The above examples are non-limiting. Variable light-based representations can be lights that emit in any color, at different frequencies (e.g., steady light, strobe, predetermined intervals, etc.), and in different modes (e.g., at least two flashes, at least three flashes, Morse code, etc.). Information about the sensor data may be generated to assist the user 130 in performing any physical activity, such as running, cycling, weight training, walking, yoga, etc. Similarly, information about the sensor data may be generated to assist the user 130 in daily activities, such as reminding the user 130 to drink water, reminding the user 130 to sit up straight, reminding the user 130 to stand up regularly, etc.

[0050] After causing the head-mounted wearable device to present information about the sensor data via the at least one LED, method 300 returns to operation (310) and awaits a new (or ongoing) indication that the user of the head-mounted wearable device is performing physical activity.

[0051] Returning to operation 330, in response to determining that the information about the sensor data will not assist the user in performing physical activity ("No" at operation 330), method 300 returns to operation (310) and waits for a new (or ongoing) indication that the user of the head-mounted wearable device is performing physical activity.

[0052] Figure 4 is a detailed flow chart illustrating a method for coordinating the display of biometric data according to some embodiments. Figure 3 The operations of method 300 and method 400 can be referred to as follows: Figure 7 The described one or more devices may be executed by one or more processors. Figure 4At least some of the operations shown in correspond to instructions stored in a computer memory or a computer-readable storage medium. The operations of method 400 may be performed by a single device or at least two devices communicatively coupled to each other. Additionally or alternatively, the operations of method 400 may be performed by instructions stored in a memory or a computer-readable medium of another device communicatively coupled to one or more devices described above with reference to FIG. 1. For simplicity, method 400 is described as being performed on a head-mounted wearable device 110 ( Figures 1A to 1D ) shall be executed at .

[0053] Method 400 includes receiving (410) an indication that a user of a head-mounted wearable device is performing a physical activity. In some embodiments, the head-mounted wearable device 110 includes at least one LED 127 that is visible to the user when the user is wearing the head-mounted wearable device. The head-mounted wearable device can communicate with an electronic device worn or carried by the user during the physical activity, the electronic device being configured to sense at least biometric data of the user during the physical activity. For example, in some embodiments, the electronic device communicatively coupled to the head-mounted wearable device 110 can be a wrist wearable device 188, a smartphone 774b, a fitness tracker, or a fitness tracker as described below. Figure 7 Other devices described herein can provide an indication that a user 130 of the head mounted wearable device 110 is performing physical activity based on the sensor data.

[0054] In some embodiments, determining that a user of a head-mounted wearable device is performing a physical activity is based on one or more of: position data; orientation data (e.g., electronic device orientation, user hand orientation, and / or other posture data measured by an inertial measurement unit); biometric data; and / or Figure 7 Other data sensed by one or more sensors of the communication-coupled device described above. Figures 1A to 2D As shown and described in , the wrist wearable device 188 can monitor sensed data (e.g., biometric data) and determine that the user is performing an activity. After the wrist wearable device 188 determines that the user is performing an activity, the wrist wearable device provides the sensed data to the head-mounted wearable device 110. In some embodiments, an electronic device communicatively coupled to the head-mounted wearable device 110 periodically sends the sensed data to the head-mounted wearable device 110. Figures 1A to 2D and Figure 7 Sensor data for determining that a user of a head-mounted wearable device is performing a physical activity is described.

[0055] Alternatively or additionally, in some embodiments, determining that the user of the head-mounted wearable device is performing physical activity is based on input commands provided at the communicatively coupled device by the user 130. The input commands may include gestures (detected by one or more cameras and / or one or more sensors), voice commands, touch input commands, activation of one or more buttons, etc. The above examples are non-limiting.

[0056] In some embodiments, method 400 includes determining, by the head-mounted wearable device 110, that the user is performing a physical activity based on sensor data monitored by the head-mounted wearable device 110. In some embodiments, determining that the user is performing a physical activity is based on one or more of the following: location data sensed by one or more sensors of the head-mounted wearable device 110 and / or biometric data sensed by a biometric sensor of the head-mounted wearable device 110. Alternatively or additionally, in some embodiments, determining that the user of the head-mounted wearable device 110 is performing a physical activity is based on an input command provided by the user 130 at a communicatively coupled device. Figures 1A to 2D and Figure 7 The additional sensor data described may be used to determine that the user of the head-mounted wearable device is performing physical activity. In some embodiments, in response to determining that the user of the head-mounted wearable device 110 is performing physical activity, the method 400 includes the head-mounted wearable device 110 requesting sensor data from a communicatively coupled device, such as the wrist wearable device 188, the smartphone 774b, the fitness tracker, or the device described below. Figure 7 The requested sensor data is used to determine whether a physiologically based threshold is satisfied, indicating that the representation of the sensor data will assist the user in performing a physical activity, as discussed below.

[0057] The method 400 includes, after receiving the indication and while the user is performing a physical activity (420), causing (430) the head-mounted wearable device to present information about the biometric data (and / or other sensed data) via at least one LED based on determining that the biometric data (and / or other sensed data) satisfies a physiological-based threshold indicating that information about the biometric data (and / or other sensed data) will assist the user in performing the physical activity. The physiological-based threshold is associated with the type of physical activity performed by the user 130. Different information may be provided to the user 130 based on the physiological-based threshold that is satisfied. For example, the biometric data sensed by the wrist-mounted wearable device may be a first type of biometric data, and at least one different physiological-based threshold that is different from the physiological-based threshold may be used to determine when to cause information about the first type of biometric data to be presented when the user is performing different physical activities. In other words, different thresholds may be used when performing different activities. In some embodiments, the physiological-based threshold includes one or more of the following: a hydration threshold; a speed / pace threshold; an oxygen level threshold; one or more heart rate zone thresholds; a pressure threshold; one or more cardiovascular zone thresholds; a posture threshold, etc. In some embodiments, the physiologically based threshold is a composite threshold based on two or more of the following: heart rate; blood oxygen saturation; respiratory rate; body temperature; position data; orientation data; and / or Figure 7 The physiologically based thresholds may be user defined, or physiologically based thresholds learned based on certain past physical activity performances of the user. The physiologically based thresholds may be stored on any (or all) of the following: a wrist wearable device; a head wearable device; and an intermediate device that facilitates communication between the wrist wearable device and the head wearable device, and the following references to FIG. 5A to FIG. 7 Other equipment described.

[0058] Determining that the biometric data (and / or other sensed data) meets the physiological-based threshold can be based on the sensed data provided to the head-mounted wearable device via the communicatively coupled electronic device. For example, the wrist wearable device 188 can monitor the biometric data and provide the biometric data to the head-mounted wearable device 110 to determine whether the physiological-based threshold is met. Alternatively, in some embodiments, the electronic device communicatively coupled to the head-mounted wearable device 110 provides instructions to the head-mounted wearable device 110, which instructions cause the LED of the head-mounted wearable device 110 to emit light in different patterns, frequencies, and / or colors based on determining that the physiological-based threshold is met. In some embodiments, the head-mounted wearable device 110 can monitor the biometric data sensed by the biometric sensor of the head-mounted wearable device 110 and / or the position data sensed by one or more sensors of the head-mounted wearable device 110 to determine whether the physiological-based threshold is met. In some embodiments, the biometric data (and / or other data) received from one or more communicatively coupled devices is analyzed and merged to determine whether the physiological-based threshold is met. For example, biometric data from the wrist wearable device 188 provided to the head-mounted wearable device 110 can be combined with biometric data sensed at the head-mounted wearable device 110 to determine whether a physiological-based threshold is met. Figures 1A to 7 Additional data is described that may be received and / or monitored by the head-mounted wearable device 110 and / or other devices communicatively coupled to the head-mounted wearable device 110. The illumination of the LED is as follows.

[0059] In some embodiments, method 400 includes receiving (450) location data sensed by one or more sensors of a wrist wearable device 188 while a user is performing (440) a physical activity, wherein the one or more sensors of the wrist wearable device 188 are different from the biometric sensors used to sense biometric data, and based on determining that the location data indicates that the user needs guidance when performing the physical activity, causing (460) a head-mounted wearable device to present guidance via at least one LED of the head-mounted wearable device to assist the user in performing the physical activity. In addition, in some embodiments, method 400 includes, based on determining that the location data sensed by one or more sensors of the head-mounted wearable device indicates that the user needs guidance when performing the physical activity, causing the head-mounted wearable device 110 to present guidance via at least one LED of the head-mounted wearable device to assist the user in performing the physical activity. When it is determined that the location data indicates that the user is performing the physical activity incorrectly, it is determined that the location data indicates that the user needs guidance. For example, FIG. 2A to FIG. 2D As shown, the head mounted wearable device 110 lights up the LED 127 to provide guidance to the user 130 on how to correct their posture.

[0060] In some embodiments, guidance that assists the user in performing physical activity is presented in conjunction with auditory feedback presented via a speaker of a head-mounted wearable device, which also assists the user in performing physical activity. Alternatively, in some embodiments, guidance that assists the user in performing physical activity is presented using only auditory feedback (e.g., the LED is not illuminated). In some embodiments, only audio feedback (e.g., instructions issued through a speaker of the head-mounted wearable device 110) is preferably used to provide guidance (e.g., positioning guidance, such as correcting posture), and the illumination of LED 127 may make it more difficult to interpret the guidance. In some embodiments, audio feedback can be provided to the user via one or more speakers (e.g., headphones) that are communicatively coupled and worn by user 130. In some embodiments, audio feedback is only available when the head-mounted wearable device 110 is communicatively coupled to a speaker worn by user 130 (e.g., when user 130 wants more privacy and does not want to announce the guidance to others nearby). Reference above Figures 1A to 2D Examples of different information provided to the user through the lighting of LED 127 are provided.

[0061] In some embodiments, the head-mounted wearable device 110 includes a single LED that is configured to emit light in any color, at different frequencies (e.g., steady light, flashing, predetermined intervals, etc.), and in different patterns (e.g., at least two flashes, at least three flashes, Morse code, etc.). In some embodiments, the different colors, patterns, and / or frequencies of the LED are associated with corresponding physiologically based thresholds and / or sensed data. For example, the LED can be configured to emit red light to notify the user 130 that they should pay attention to their posture, emit green light to notify the user that they are at a target heart rate, etc.

[0062] Alternatively, in some embodiments, the head-mounted wearable device 110 may include multiple LEDs. More than one LED may be used to provide information about biometric data (or other sensed data). In some embodiments, different LEDs are associated with corresponding physiological-based thresholds and / or sensed data. For example, multiple LEDs may include a first LED associated with the heart rate of the user 130 (e.g., HR LED), a second LED associated with the blood oxygen saturation of the user 130 (e.g., SpO2 LED), a third LED associated with the breathing rate of the user 130 (e.g., breathing rate LED), and a fourth LED associated with the posture of the user 130 (e.g., posture LED). Each LED can be configured to emit light in any color, at different frequencies, and in different modes. In some embodiments, each LED is individually controlled based on information about the sensor data (e.g., each corresponding LED is made to emit red light, yellow light, or green light to notify the user 130 that adjustment is required, caution is required, or no major problems are detected). Alternatively, in some embodiments, each LED is configured to emit light in a corresponding color, pattern, or frequency based on information about the sensor data (e.g., the posture LED can be made to emit red light to notify user 130 to pay attention to their posture, the breathing rate LED can be made to emit purple light to notify user 130 to pay attention to their breathing, etc.).

[0063] In some embodiments, the wrist wearable device 188 (or other communicatively coupled electronic device) is configured to monitor additional biometric data of the user 130 during the physical activity, the additional biometric data being sensed using an additional biometric sensor that is different from the biometric sensor used to provide the biometric data mentioned in operation 430. The method 400 also includes, when the user is performing the physical activity, in accordance with determining that the additional biometric data satisfies an additional physiological-based threshold that is different from the physiological-based threshold, indicating that the information about the additional biometric data will assist the user in performing the physical activity, thereby causing the head-mounted wearable device to present the information about the additional biometric data via an additional LED in the plurality of LEDs. In some embodiments, the information about the additional biometric data and the information about the biometric data are presented via the additional LED and the at least one LED, respectively, during an overlapping time period. In some embodiments, the wrist wearable device is configured to monitor additional biometric data of the user during the physical activity, the additional biometric data being sensed using another biometric sensor that is different from the biometric sensor and the additional biometric sensor. The method 400 also includes, when the user is performing a physical activity, in accordance with determining that the additional biometric data satisfies another physiologically based threshold different from the physiologically based threshold and the additional physiologically based threshold, indicating that the information about the additional biometric data will assist the user in performing the physical activity, so that the head-mounted wearable device presents the information about the additional biometric data via another LED of the plurality of LEDs. The information about the additional biometric data, the information about the biometric data, and the information about the additional biometric data are presented via the additional LED, the at least one LED, and the other LED, respectively, during the overlapping time period. In other words, as described above, different LEDs can be controlled separately to convey different information to the user 130.

[0064] In some embodiments, the LED is coupled to a housing of the head-mounted wearable device 110. Alternatively or additionally, in some embodiments, the LED is coupled to one or more lenses of the head-mounted wearable device 110. The head-mounted wearable device 110 is configured to provide information about the sensor data without using a heads-up display, a screen display, an overhead display, etc. In some embodiments, the head-mounted wearable device 110 does not include a heads-up display, a screen display, an overhead display, etc.

[0065] In some embodiments, information about sensor data and / or guidance to assist the user is a variable light-based representation that communicates to the user 130 how to improve their posture and / or technique. For example, the variable light-based representation can help the user perform yoga, high-intensity interval training (HIIT) routines, golf, jogging, and many other physical activities. In some embodiments, the variable light-based representation can be presented only to the user (without obstructing the user's view), presented to the user and others (e.g., presented to a workout instructor so that the workout instructor can train or coach the user), and / or presented only to others (e.g., a do not disturb indicator).

[0066] Example wrist wearable device

[0067] Figure 5A and Figure 5B An example wrist wearable device 550 is shown in accordance with some embodiments. Wrist wearable device 550 is an example of a wearable device described herein such that the wearable device should be understood to have features of wrist wearable device 550 and vice versa. Figure 5A A perspective view of a wrist wearable device 550 is shown, which includes a watch body 554 coupled to a watch band 562. The watch body 554 and the watch band 562 can have a generally rectangular or circular shape and can be configured to allow a user to wear the wrist wearable device 550 on a body part (e.g., a wrist). The wrist wearable device 550 can include a retaining mechanism 567 (e.g., a buckle, a hook and loop fastener, etc.) for securing the watch band 562 to the user's wrist. The wrist wearable device 550 can also include a coupling mechanism 560 (e.g., a bracket) for removably coupling a capsule or watch body 554 (via a coupling surface of the watch body 554) to the watch band 562.

[0068] The wrist wearable device 550 can perform various functions associated with navigating through the user interface and selectively opening applications, as well as the functions described above with reference to FIG. Figures 1A to 4As described in more detail below, the operations performed by the wrist wearable device 550 may include, but are not limited to, displaying visual content to a user (e.g., visual content displayed on the display 556), sensing user input (e.g., sensing touch to the peripheral button 568, sensing biometric data on the sensor 564, sensing neuromuscular signals on the neuromuscular sensor 565, etc.), delivering messages (e.g., text, voice, video, etc.), image acquisition, wireless communications (e.g., cellular, near field, Wi-Fi, personal area network, etc.), location determination, financial transactions, providing tactile feedback, alarms, notifications, biometric authentication, health monitoring, sleep monitoring, etc. These functions may be performed independently in the watch body 554, independently in the watch band 562, and / or in communication between the watch body 554 and the watch band 562. In some embodiments, these functions can be performed on the wrist wearable device 550 in conjunction with an artificial reality environment, including but not limited to a virtual reality (VR) environment (including a non-immersive VR environment, a semi-immersive VR environment, and a fully immersive VR environment), an augmented reality environment (including a marker-based augmented reality environment, a markerless augmented reality environment, a location-based augmented reality environment, and a projection-based augmented reality environment), a hybrid reality, and other types of mixed reality environments. As will be understood by those skilled in the art upon reading the description provided herein, the novel wearable device described herein can be used with any of these types of artificial reality environments.

[0069] The strap 562 can be configured to be worn by the user so that the inner surface of the strap 562 is in contact with the user's skin. When worn by the user, the sensor 564 is in contact with the user's skin. The sensor 564 can be a biosensor that senses the following items: the user's heart rate, saturated oxygen level, temperature, sweat level, muscle intention, or a combination thereof. The strap 562 may include a plurality of sensors 564, which may be distributed on the inner surface and / or outer surface of the strap 562. Additionally or alternatively, the watch body 554 may include sensors that are the same as or different from the sensors of the strap 562 (or in some embodiments, the strap 562 may not include any sensors at all). For example, a plurality of sensors may be distributed on the inner surface and / or outer surface of the watch body 554. As described below with reference to Figure 5B and / or Figure 5CAs described, the watch body 554 may include, but is not limited to, a front image sensor 525A and / or a rear image sensor 525B, a biometric sensor, an inertial measurement unit (IMU), a heart rate sensor, a saturated oxygen sensor, one or more neuromuscular sensors, an altimeter sensor, a temperature sensor, a bioimpedance sensor, a pedometer sensor, an optical sensor (e.g., an imaging sensor 5104), a touch sensor, a sweat sensor, etc. The sensor 564 may also include a sensor that provides data about the user's environment, including the user's motion (e.g., IMU), altitude, position, orientation, gait, or a combination thereof. The sensor 564 may also include a light sensor (e.g., an infrared light sensor, a visible light sensor) configured to track the position and / or motion of the watch body 554 and / or the strap 562. The strap 562 can use a wired communication method (e.g., a Universal Asynchronous Receiver / Transmitter (UART), a Universal Serial Bus (USB) transceiver, etc.) and / or a wireless communication method (e.g., near field communication, Bluetooth, etc.) to transmit data obtained by the sensor 564 to the watch body 554. The strap 562 can be configured to operate independently of whether the watch body 554 is coupled to the strap 562 or separated from the strap 562 (e.g., to collect data using the sensor 564).

[0070] In some examples, the strap 562 may include a neuromuscular sensor 565 (e.g., an EMG sensor, a mechanomyogram (MMG) sensor, a sonomyography (SMG) sensor, etc.). The neuromuscular sensor 565 may sense the user's intention to perform certain motor actions. The sensed muscle intention may be used to control certain user interfaces displayed on the display 556 of the wrist wearable device 550, and / or may be transmitted to a device responsible for rendering an artificial reality environment (e.g., a head-mounted display) to perform actions in an associated artificial reality environment, such as to control the movement of a virtual device displayed to the user.

[0071] Signals from neuromuscular sensors 565 may be used to provide the user with enhanced interaction with physical objects and / or virtual objects in an artificial reality application generated by the artificial reality system (e.g., user interface objects presented on display 556 or another computing device (e.g., a smartphone)). Signals from neuromuscular sensors 565 may be obtained (e.g., sensed and recorded) by one or more neuromuscular sensors 565 of strap 562. Figure 5AOne neuromuscular sensor 565 is shown, but the strap 562 may include multiple neuromuscular sensors 565 arranged circumferentially on the inner surface of the strap 562 so that the multiple neuromuscular sensors 565 contact the user's skin. The strap 562 may include multiple neuromuscular sensors 565 arranged circumferentially on the inner surface of the strap 562. The neuromuscular sensors 565 may sense and record neuromuscular signals from the user when the user performs muscle activation (e.g., movement, gestures, etc.). The muscle activation performed by the user may include: static gestures, such as the user's palm facing down on a table; dynamic gestures, such as grasping a physical or virtual object; and covert gestures that are not noticeable to another person, such as slightly tensing a joint by co-contracting opposing muscles or using sub-muscular activation. The muscle activation performed by the user may include symbolic gestures (e.g., gestures that are mapped to other gestures, interactions, or command gestures, such as based on a gesture vocabulary that specifies a mapping of gestures to commands).

[0072] The strap 562 and / or the watch body 554 may include a haptic device 563 (e.g., a vibrotactile actuator) configured to provide tactile feedback (e.g., skin sensation and / or kinesthetic sensation, etc.) to the user's skin. The sensors 564 and 565 and / or the haptic device 563 may be configured to operate in conjunction with a plurality of applications, including but not limited to health monitoring, social media, playing games, and artificial reality (e.g., applications associated with artificial reality).

[0073] The wrist wearable device 550 may include a coupling mechanism (also referred to as a cradle) for removably coupling the watch body 554 to the strap 562. The user may detach the watch body 554 from the strap 562 to reduce the burden of the wrist wearable device 550 on the user. The wrist wearable device 550 may include a coupling surface on the watch body 554, and / or one or more coupling mechanisms 560 (e.g., a cradle, a tracker band, a support base band, a buckle). The user may perform any type of action for coupling the watch body 554 to the strap 562 and detaching the watch body 554 from the strap 562. For example, the user may twist, slide, turn, push, pull, or rotate (or a combination thereof) the watch body 554 relative to the strap 562 to attach the watch body 554 to the strap 562 and detach the watch body 554 from the strap 562.

[0074] like Figure 5AAs shown in the example of , the coupling mechanism 560 of the watch band may include a type of frame or housing that allows the coupling surface of the watch body 554 to remain within the coupling mechanism 560 of the watch band. The watch body 554 may be removably coupled to the watch band 562 by friction fit, magnetic coupling, a rotation-based connector, a shear pin coupler, a retaining spring, one or more magnets, a clamp, a pin, a hook and loop fastener, or a combination thereof. In some examples, the watch body 554 can be separated from the watch band 562 by actuation of a release mechanism 570. The release mechanism 570 may include, but is not limited to, a button, a knob, a plug, a handle, a lever, a fastener, a buckle, a dial, a latch, or a combination thereof.

[0075] like Figure 5A and Figure 5B As shown, the coupling mechanism 560 can be configured to receive the coupling surface of the watch body 554 near the bottom side (e.g., the side opposite the front side of the watch body 554 where the display 556 is located) so that the user can push the watch body 554 downward into the coupling mechanism 560 to attach the watch body 554 to the coupling mechanism 560. In some embodiments, the coupling mechanism 560 can be configured to receive the top side of the watch body 554 (e.g., the side near the front side of the watch body 554 where the display 556 is located) and the watch body 554 is pushed upward into the bracket instead of being pushed downward into the coupling mechanism 560. In some embodiments, the coupling mechanism 560 is an integrated component of the strap 562 such that the strap 562 and the coupling mechanism 560 are a single unified structure.

[0076] The wrist wearable device 550 may include a single release mechanism 570 or multiple release mechanisms 570 (e.g., two release mechanisms 570, such as spring-loaded buttons, located on opposite sides of the wrist wearable device 550). Figure 5A As shown, the release mechanism 570 can be located on the watch body 554 and / or the coupling mechanism 560 of the watch band. Figure 5A The release mechanism 570 is shown to be located at a corner of the watch body 554 and a corner of the strap coupling mechanism 560, but the release mechanism 570 can be located at any position on the watch body 554 and / or the strap coupling mechanism 560 that is convenient for actuation by the user of the wrist wearable device 550. The user of the wrist wearable device 550 can actuate the release mechanism 570 by pushing, turning, lifting, pressing, moving, or performing other actions on the release mechanism 570. Actuation of the release mechanism 570 can release (e.g., separate) the watch body 554 from the strap coupling mechanism 560 and the strap 562, thereby allowing the user to use the watch body 554 independently of the strap 562. For example, separating the watch body 554 from the strap 562 can allow the user to use the rear image sensor 525B to capture images.

[0077] Figure 5BA top view of an example of a wrist wearable device 550 is included. Figure 5A and Figure 5B The example of a wrist wearable device 550 shown in FIG. 5 may include a coupling mechanism 560 (eg, Figure 5B , the shape of the coupling mechanism can correspond to the shape of the watch body 554 of the wrist wearable device 550). The watch body 554 can be removably coupled to the coupling mechanism 560 by: friction fit, magnetic coupling, rotation-based connector, shear pin coupler, retaining spring, one or more magnets, clamps, pins, hook and loop fasteners, or a combination thereof.

[0078] In some examples, the watch body 554 can be separated from the coupling mechanism 560 by actuating the release mechanism 570. The release mechanism 570 can include, but is not limited to, a button, a knob, a plug, a handle, a lever, a fastener, a buckle, a dial, a latch, or a combination thereof. In some examples, the wristband system functions can be performed independently in the watch body 554, independently in the coupling mechanism 560, and / or performed in communication between the watch body 554 and the coupling mechanism 560. The coupling mechanism 560 can be configured to operate independently of the watch body 554 (e.g., independently perform functions). Additionally or alternatively, the watch body 554 can be configured to operate independently of the coupling mechanism 560 (e.g., independently perform functions). As described below with reference Figure 5A As described in the block diagram of , the coupling mechanism 560 and / or the body 554 may each include independent resources required to independently perform functions. For example, the coupling mechanism 560 and / or the body 554 may each include a power source (e.g., a battery), a memory, a data storage library, a processor (e.g., a central processing unit (CPU)), a communication, a light source, and / or an input device / output device.

[0079] The wrist wearable device 550 may have various peripheral buttons 572, 574, and 576 for performing various operations on the wrist wearable device 550. In addition, various sensors (including one or both of the sensors 564 and 565) may be located at the bottom of the watch body 554 and may be selectively used even when the watch body 554 is detached from the strap 562.

[0080] Figure 5C 5 is a block diagram of a computing system 5000 according to at least one embodiment of the present disclosure. The computing system 5000 includes an electronic device 5002, which may be, for example, a wrist wearable device. Figure 5A and Figure 5BThe wrist wearable device 550 described in detail is an example of the electronic device 5002, and thus the electronic device 5002 will be understood to include the components shown and described below for the computing system 5000. In some embodiments, all or most of the components of the computing system 5000 are included in a single integrated circuit. In some embodiments, the computing system 5000 can be on a watch body (e.g., Figure 5A and Figure 5B The watch body 554) and the watch band (e.g., Figure 5A and Figure 5B The electronic device 5002 may include a processor (e.g., a central processing unit 5004); a controller 5010; a peripheral interface 5014 including one or more sensors 5100 and various peripheral devices; a power source (e.g., a power system 5300); and a memory (e.g., a memory 5400) including an operating system (e.g., an operating system 5402), data (e.g., data 5410), and one or more applications (e.g., applications 5430).

[0081] In some embodiments, computing system 5000 includes a power system 5300 including a charger input 5302 , a power-management integrated circuit (PMIC) 5304 , and a battery 5306 .

[0082] In some embodiments, the watch body and the watch band can each be an electronic device 5002, each having a corresponding battery (e.g., battery 5306), and can share power with each other. The watch body and the watch band can receive the charge using various technologies. In some embodiments, the watch body and the watch band can receive the charge using a wired charging component (e.g., a power line). Alternatively or additionally, the watch body and / or the watch band can be configured for wireless charging. For example, a portable charging device can be designed to match a portion of the watch body and / or the watch band and wirelessly transmit available power to the battery of the watch body and / or the battery of the watch band.

[0083] The watch body and the watch band can have independent power systems 5300 to enable each to operate independently. The watch body and the watch band can also share power (e.g., one can charge the other) via their own PMIC 5304, which can share power through electrical and ground conductors and / or wireless charging antennas.

[0084] In some embodiments, the peripheral interface 5014 may include one or more sensors 5100. The sensor 5100 may include a coupling sensor 5102 for detecting when the electronic device 5002 is coupled to another electronic device 5002 (e.g., the watch body can detect when it is coupled to the watch band and vice versa). The sensor 5100 may include an imaging sensor 5104 for collecting imaging data, which may optionally be the same device as one or more cameras 5218. In some embodiments, the imaging sensor 5104 may be separate from the camera 5218. In some embodiments, the sensor includes an SpO2 sensor 5106. In some embodiments, the sensor 5100 includes an EMG sensor 5108 for detecting muscle movement of, for example, a user of the electronic device 5002. In some embodiments, the sensor 5100 includes a capacitive sensor 5110 for detecting changes in electrical potential of a part of the user's body. In some embodiments, the sensor 5100 includes a heart rate sensor 5112. In some embodiments, the sensor 5100 includes an inertial measurement unit (IMU) sensor 5114, which is used to detect changes in acceleration of a user's hand, for example.

[0085] In some embodiments, the peripheral interface 5014 includes a near-field communication (NFC) component 5202 , a global-position system (GPS) component 5204 , a long-term evolution (LTE) component 5206 , and / or a Wi-Fi or Bluetooth communication component 5208 .

[0086] In some embodiments, the peripheral interface includes one or more buttons (e.g., Figure 5B Peripheral buttons 576, 574 and 5572 in the electronic device 5002), which when selected by the user, causes an operation to be performed at the electronic device 5002.

[0087] The electronic device 5002 may include at least one display 5212 for displaying visual affordances to the user, including user interface elements and / or three-dimensional virtual objects. The display may also include a touch screen for inputting user inputs (e.g., touch gestures and sliding gestures, etc.).

[0088] The electronic device 5002 may include at least one speaker 5214 for providing audio signals to the user and at least one microphone 5216 for receiving audio input from the user. The user may provide user input through the microphone 5216 and may also receive audio output from the speaker 5214 as part of the tactile event provided by the tactile controller 5012.

[0089] The electronic device 5002 may include at least one camera 5218, including a front camera 5220 and a rear camera 5222. In some embodiments, the electronic device 5002 may be a head wearable device, and one of the cameras 5218 may be integrated with a lens assembly of the head wearable device.

[0090] One or more of the electronic devices 5002 may include one or more haptic controllers 5012 and associated components for providing haptic events (e.g., a vibration sensation or audio output in response to an event at the electronic device 5002) at one or more of the electronic devices 5002. The haptic controller 5012 may communicate with one or more electroacoustic devices, including one of the one or more speakers 5214 and / or other audio components and / or an electromechanical device that converts energy into linear motion (e.g., a motor, an electromagnetic coil, an electroactive polymer, a piezoelectric actuator, an electrostatic actuator, or other haptic output generating component (e.g., a component that converts an electrical signal into a haptic output on the device)). The haptic controller 5012 may provide haptic events that can be sensed by a user of the electronic device 5002. In some embodiments, the one or more haptic controllers 5012 may receive an input signal from an application in the applications 5430.

[0091] The memory 5400 optionally includes a high-speed random access memory, and optionally also includes a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices. The access of other components of the electronic device 5002 (e.g., one or more processors of the central processing unit 5004) and the peripheral interface 5014 to the memory 5400 is optionally controlled by the storage controller of the controller 5010.

[0092] In some embodiments, the software components stored in the memory 5400 may include one or more operating systems 5402 (e.g., a Linux-based operating system, an Android operating system, etc.). The memory 5400 may also include data 5410, which includes structured data (e.g., a SQL database, a MongoDB database, GraphQL data, JSON data, etc.). The data 5410 may include profile data 5412, sensor data 5414, media file data 5416, and image storage 5418.

[0093] In some embodiments, the software components stored in the memory 5400 include one or more applications 5430 configured to perform operations at the electronic device 5002. In some embodiments, the memory 5400 includes one or more communication interface modules 5432, one or more graphics modules 5434, and (see below) Figure 7 The AR processing module 745 described above. In some embodiments, multiple applications 5430 can work in conjunction with each other to perform various tasks at one or more electronic devices in each electronic device 5002.

[0094] It should be understood that electronic devices 5002 are only some examples of electronic devices 5002 within computing system 5000, and that other electronic devices 5002 that are part of computing system 5000 may have more or fewer components than those shown, optionally combine two or more components, or optionally have a different configuration or arrangement of components. Figure 5C The various components shown in the drawings are implemented in hardware, software, firmware, or a combination thereof (including one or more signal processing circuits and / or application specific integrated circuits).

[0095] like Figure 5C As shown in the lower part of the wrist wearable device, various independent components of the wrist wearable device can be examples of electronic device 5002. For example, some or all of the components shown in electronic device 5002 can be housed or otherwise provided in combined watch device 5002A, or provided in independent components of capsule device watch body 5002B, bracket portion 5002C and / or strap.

[0096] Figure 5DA wearable device 5170 according to some embodiments is shown. In some embodiments, the wearable device 5170 is used to generate control information (e.g., sensed data about neuromuscular signals or instructions for executing a specific command after sensing the data), which is used to cause a computing device to execute one or more input commands. In some embodiments, the wearable device 5170 includes a plurality of neuromuscular sensors 5176. In some embodiments, the plurality of neuromuscular sensors 5176 includes a predetermined number (e.g., 16) of neuromuscular sensors (e.g., EMG sensors) arranged circumferentially around the elastic band 5174. The plurality of neuromuscular sensors 5176 may include any suitable number of neuromuscular sensors. In some embodiments, the number and arrangement of the neuromuscular sensors 5176 depends on the specific application in which the wearable device 5170 is used. For example, a wearable device 5170 configured as an armband, wristband, or chest band may include a plurality of neuromuscular sensors 5176 having a different number of neuromuscular sensors and a different arrangement for each use case (e.g., a medical use case compared to a game or a general daily use case). For example, at least 16 neuromuscular sensors 5176 may be arranged circumferentially around the elastic band 5174 .

[0097] In some embodiments, the elastic band 5174 is configured to be worn on the user's forearm or wrist. The elastic band 5174 may include a flexible electronic connector 5172. In some embodiments, the flexible electronic connector 5172 interconnects independent sensors and electronic circuits encapsulated in one or more sensor housings. Alternatively, in some embodiments, the flexible electronic connector 5172 interconnects independent sensors and electronic circuits outside one or more sensor housings. Each neuromuscular sensor in the plurality of neuromuscular sensors 5176 may include a skin contact surface, which includes one or more electrodes. One or more of the plurality of neuromuscular sensors 5176 may be coupled together using a flexible electronic device incorporated into the wearable device 5170. In some embodiments, one or more of the plurality of neuromuscular sensors 5176 may be incorporated into a woven fabric, wherein the fabric sews one or more of the plurality of neuromuscular sensors 5176 into the fabric and simulates the flexibility of the fabric (e.g., one or more of the plurality of neuromuscular sensors 5176 may be composed of a series of woven fabric strands). In some embodiments, the sensor is flush with the surface of the textile and cannot be distinguished from the textile when worn by the user.

[0098] Figure 5EA wearable device 5179 is shown according to some embodiments. The wearable device 5179 includes paired sensor channels 5185a to 5185f along the inner surface of the wearable structure 5175, and the sensor channels are configured to detect neuromuscular signals. Different numbers of paired sensor channels (e.g., one pair of sensors, three pairs of sensors, four pairs of sensors, or six pairs of sensors) can be used. The wearable structure 5175 may include a band portion 5190, a cabin portion 5195, and a bracket portion (not shown) that is coupled to the band portion 5190 to allow the cabin portion 5195 to be detachably coupled to the band portion 5190. For embodiments in which the cabin portion 5195 is detachable, the cabin portion 5195 may be referred to as a detachable structure, so that in these embodiments, the wearable device includes a wearable portion (e.g., a band portion 5190 and a bracket portion) and a detachable structure (a detachable cabin portion that can be removed from the bracket). In some embodiments, the cabin portion 5195 includes the above reference Figure 7 One or more processors and / or other components of the wearable device 188 described. The wearable structure 5175 is configured to be worn by the user 130. More specifically, the wearable structure 5175 is configured to couple the wearable device 5179 to the user's wrist, arm, forearm, or other part of the body. Each paired sensor channel 5185a to 5185f includes two electrodes 5180 (e.g., electrodes 5180a to 5180h) that are used to sense neuromuscular signals based on differential sensing within each corresponding sensor channel. According to some embodiments, the wearable device 5170 also includes an electrical ground electrode and a shield electrode.

[0099] The above technology can be used with any device for sensing neuromuscular signals (including FIG. 5A to FIG. 5C The invention is intended for use with an arm wearable device such as an arm wearable device in a computer program product, but may also be used with other types of wearable devices for sensing neuromuscular signals (e.g., a body wearable device or a head wearable device that may have neuromuscular sensors closer to the brain or spine).

[0100] In some embodiments, the wrist wearable device can be used in conjunction with the head wearable device described below, and the wrist wearable device can also be configured to allow the user to control aspects of the artificial reality (e.g., by using EMG-based gestures to control user interface objects in the artificial reality, and / or by allowing the user to interact with a touch screen on the wrist wearable device to also control aspects of the artificial reality). Having thus described an example wrist wearable device, attention is now turned to example head wearable devices, such as AR glasses and VR headsets.

[0101] Example head wearable device

[0102] Fig. 6A An example AR system 600 is shown in accordance with some embodiments. Fig. 6A , the AR system 600 includes an eyewear device having a frame 602, which is configured to hold a left display device 606-1 and a right display device 606-2 in front of the user's eyes. The display device 606-1 and the display device 606-2 can work together or independently to present an image or a series of images to the user. Although the AR system 600 includes two displays, the embodiments of the present disclosure can be implemented in an AR system having a single near-eye display (NED) or more than two NEDs.

[0103] In some embodiments, the AR system 600 includes one or more sensors, such as an acoustic sensor 604. For example, the acoustic sensor 604 can generate a measurement signal in response to the movement of the AR system 600, and the acoustic sensor 604 can be located on substantially any portion of the frame 602. Any of the sensors can be a position sensor, an IMU, a depth camera assembly, or any combination thereof. In some embodiments, the AR system 600 includes a sensor such as an acoustic sensor 604. Fig. 6A The components shown in the figure may be more or less than the components shown in the figure. In embodiments where the sensors include an IMU, the IMU may generate calibration data based on measurement signals from each sensor. Examples of sensors may include, but are not limited to, accelerometers, gyroscopes, magnetometers, other suitable types of sensors that detect motion, sensors for error correction of the IMU, or some combination thereof.

[0104] In some embodiments, the AR system 600 includes a microphone array having a plurality of acoustic sensors 604-1 to 604-8, collectively referred to as acoustic sensors 604. Acoustic sensors 604 may be transducers that detect changes in air pressure caused by sound waves. In some embodiments, each acoustic sensor 604 is configured to detect sound and convert the detected sound into an electronic format (e.g., an analog format or a digital format). In some embodiments, the microphone array includes ten acoustic sensors: 604-1 and 604-2 designed to be placed in corresponding ears of the user, acoustic sensors 604-3, 604-4, 604-5, 604-6, 604-7, and 604-8 located at different positions on the frame 602, and acoustic sensors located on corresponding neckbands, wherein the neckband is an optional component of the system, and the neckband is not present in some embodiments of the artificial reality system discussed herein.

[0105] The configuration of the acoustic sensor 604 of the microphone array may vary. Fig. 6A604, but the number of acoustic sensors 604 can be more or less than ten. In some cases, the use of a greater number of acoustic sensors 604 increases the amount of audio information collected and / or the sensitivity and accuracy of the audio information. In contrast, in some cases, the use of a smaller number of acoustic sensors 604 reduces the computing power required for the controller to process the collected audio information. In addition, the position of each acoustic sensor 604 of the microphone array can vary. For example, the position of the acoustic sensor 604 can include a defined position on the user, a defined coordinate on the frame 602, an orientation associated with each acoustic sensor, or some combination thereof.

[0106] Acoustic sensors 604-1 and 604-2 can be located at different parts of the user's ear. In some embodiments, in addition to the acoustic sensor 604 inside the ear canal, additional acoustic sensors may be present on or around the ear. In some cases, positioning the acoustic sensor next to the user's ear canal enables the microphone array to collect information about how the sound reaches the ear canal. By positioning at least two of the acoustic sensors 604 on both sides of the user's head (e.g., as binaural microphones), the AR device 600 can simulate binaural hearing and collect a 3D stereo sound field around the user's head. In some embodiments, the acoustic sensors 604-1 and 604-2 are connected to the AR system 600 via a wired connection, while in other embodiments, the acoustic sensors 604-1 and 604-2 are connected to the AR system 600 via a wireless connection (e.g., a Bluetooth connection). In some embodiments, the AR system 600 does not include acoustic sensors 604-1 and 604-2.

[0107] The acoustic sensors 604 on the frame 602 may be positioned along the length of the temples, across the nose bridge, above or below the display device 606, or some combination thereof. The acoustic sensors 604 may be oriented so that the microphone array can detect sounds in a wide range of directions around the user wearing the AR system 600. In some embodiments, a calibration process may be performed during the manufacture of the AR system 600 to determine the relative positioning of the acoustic sensors 604 in the microphone array.

[0108] In some embodiments, the eyeglass device also includes or is communicatively coupled to an external device (e.g., a pairing device), such as the optional neckband discussed above. In some embodiments, the optional neckband is coupled to the eyeglass device via one or more connectors. The connector may be a wired connector or a wireless connector, and may include electrical components and / or non-electrical (e.g., structural) components. In some embodiments, the eyeglass device and the neckband operate independently without any wired or wireless connection between them. In some embodiments, the components of the eyeglass device and the components of the neckband are located on one or more additional peripheral devices that are paired with the eyeglass device, the neckband, or some combination thereof. In addition, the neckband is intended to represent any suitable type or form of paired device. Therefore, the following discussion of the neckband may also be applicable to various other paired devices, such as smart watches, smart phones, wristbands, other wearable devices, handheld controllers, tablet computers, or laptop computers.

[0109] In some cases, pairing an external device (e.g., an optional neckband) with the AR eyewear device enables the AR eyewear device to achieve the form factor of a pair of glasses while still providing sufficient battery and computing power for the expanded capabilities. Some or all of the battery power, computing resources, and / or additional features of the AR system 600 can be provided by a paired device, or shared between a paired device and the eyewear device, thereby reducing the weight, heat distribution, and form factor of the eyewear device as a whole, while still maintaining the desired functionality. For example, a neckband can allow components that were originally included on the eyewear device to be included in the neckband, thereby shifting the weight load from the user's head to the user's shoulders. In some embodiments, the neckband has a larger surface area over which to diffuse and disperse heat to the surrounding environment. Therefore, the neckband can allow for greater battery power and greater computing power than would otherwise be possible on a standalone eyewear device. Because the weight carried in the neckband may be less intrusive to the user than the weight carried in the eyeglass device, the user may be able to tolerate wearing the lighter eyeglass device and carrying or wearing a paired device for longer periods of time than the user would tolerate wearing a heavier separate eyeglass device, thereby allowing the artificial reality environment to be more fully integrated into the user's daily activities.

[0110] In some embodiments, the optional neckband is communicatively coupled to the eyewear device and / or communicatively coupled to other devices. These other devices can provide certain functions (e.g., tracking, positioning, depth mapping, processing, storage, etc.) to the AR system 600. In some embodiments, the neckband includes a controller and a power supply. In some embodiments, the acoustic sensor of the neckband is configured to detect sound and convert the detected sound into an electronic format (analog or digital).

[0111] The controller of the neckband processes information generated by sensors on the neckband and / or AR system 600. For example, the controller can process information from the acoustic sensor 604. For each detected sound, the controller can perform a direction-of-arrival (DOA) estimate to estimate the direction in which the detected sound arrives at the microphone array. When the microphone array detects a sound, the controller can fill the audio data set with this information. In an embodiment where the AR system 600 includes an IMU, the controller can calculate all inertial and spatial calculation results from the IMU located on the eyeglass device. The connector can transfer information between the eyeglass device and the neckband and between the eyeglass device and the controller. The information can be in the form of optical data, electrical data, wireless data, or any other transmittable data form. Transferring the processing of information generated by the eyeglass device to the neckband can reduce the weight and heat of the eyeglass device, thereby making the eyeglass device more comfortable and safer for the user.

[0112] In some embodiments, the power supply in the neckband provides power to the eyeglass device and the neckband. The power supply may include, but is not limited to, a lithium-ion battery, a lithium-polymer battery, a disposable lithium battery, an alkaline battery, or any other form of power storage. In some embodiments, the power supply is a wired power supply.

[0113] As described, some artificial reality systems may substantially replace one or more of a user's sensory perceptions of the real world with a virtual experience, rather than blending the artificial reality with actual reality. An example of this type of system is a head-mounted display system that mostly or completely covers the user's field of view, e.g. Figure 6B VR system 650 in.

[0114] Figure 6B A VR system 650 (e.g., also referred to herein as a VR head-mounted viewer or VR head-mounted viewer) according to some embodiments is shown. VR system 650 includes a head-mounted display (HMD) 652. HMD 652 includes a front body 656 and a frame 654 (e.g., a strip or band) shaped to fit a user's head. In some embodiments, HMD 652 includes output audio transducers 658-1 and 658-2 (e.g., transducers), such as Figure 6B In some embodiments, the front body 656 and / or the frame 654 include one or more electronic components, including one or more electronic displays, one or more IMUs, one or more tracking transmitters or detectors, and / or any other suitable device or sensor for generating an artificial reality experience.

[0115] The artificial reality system may include various types of visual feedback mechanisms. For example, the display device in the AR system 600 and / or the display device in the VR system 650 may include one or more liquid crystal displays (LCD), one or more light emitting diode (LED) displays, one or more organic LED (OLED) displays, and / or any other suitable type of display screen. The artificial reality system may include a single display screen for both eyes or may provide a display screen for each eye, which may allow additional flexibility for zoom adjustment or for correcting refractive errors associated with the user's vision. Some artificial reality systems also include an optical subsystem having one or more lenses (e.g., a conventional concave or convex lens, a Fresnel lens, or an adjustable liquid lens), through which the user can view the display screen.

[0116] In addition to or instead of using a display screen, some artificial reality systems include one or more projection systems. For example, the display device in the AR system 600 and / or the display device in the VR system 650 may include micro LED projectors that project light (e.g., using a waveguide) into a display device (e.g., a transparent combination lens that allows ambient light to pass through). The display device can refract the projected light toward the user's pupil and can enable the user to view both the artificial reality content and the real world at the same time. The artificial reality system can also be configured with any other suitable type or form of image projection system.

[0117] The artificial reality system may also include various types of computer vision components and subsystems. For example, the AR system 600 and / or the VR system 650 may include one or more optical sensors, such as two-dimensional (2D) or three-dimensional (3D) cameras, time-of-flight depth sensors, single-beam or scanning laser rangefinders, 3D laser radar (LiDAR) sensors, and / or any other suitable type or form of optical sensor. The artificial reality system may process data from one or more of these sensors to identify the user's location, map the real world, provide the user with context about the real-world surroundings, and / or perform various other functions. For example, Figure 6B A VR system 650 is shown with cameras 660 - 1 and 660 - 2 , which can be used to provide depth information, which is used to create a voxel field and a two-dimensional grid to provide object information to the user for collision avoidance. Figure 6BThe VR system is also shown to include one or more additional cameras 662 configured to enhance cameras 660-1 and 660-2 by providing more information. For example, additional camera 662 can be used to provide color information that is not recognized by cameras 660-1 and 660-2. In some embodiments, cameras 660-1 and 660-2 and additional camera 662 can include an optional IR cut filter configured to remove IR light from light received at the corresponding camera sensor.

[0118] In some embodiments, the AR system 600 and / or the VR system 650 may include a haptic (tactile) feedback system that can be incorporated into headwear, gloves, jumpsuits, handheld controllers, environmental devices (e.g., chairs or foot pads), and / or any other type of device or system (e.g., wearable devices discussed herein). The haptic feedback system can provide various types of skin feedback, including vibration, force, tension, shear, texture, and / or temperature. The haptic feedback system can also provide various types of kinesthetic feedback, such as motion and compliance. Tactile feedback can be implemented using motors, piezoelectric actuators, fluid systems, and / or various other types of feedback mechanisms. The haptic feedback system can be implemented independently of other artificial reality devices, implemented within other artificial reality devices, and / or implemented in combination with other artificial reality devices.

[0119] The above techniques can be used with any device used to interact with an artificial reality environment (including Fig. 6A and Figure 6B The AR system 600 and / or the VR system 650 may be used with the head wearable device of the present invention, but may also be used with other types of wearable devices for sensing neuromuscular signals (e.g., body wearable devices or head wearable devices that may have neuromuscular sensors closer to the brain or spine). In addition, the AR system 600 and / or the VR system 650 may perform the above referenced Figures 1A to 4 Having thus described example wrist wearable devices and head wearable devices, attention is now turned to example feedback systems, which may be integrated into the above devices or may be separate devices.

[0120] Example System

[0121] Figure 7A system 700 of one or more devices for providing indications of biometric data and other sensor data is shown in accordance with some embodiments. For example, the head-mounted wearable device 110 can be configured to provide indications of activities that the user 130 is performing via an illumination source based on received and / or monitored sensor data. These indications are used to convey different messages to the user 130 and / or other people near the user 130 (e.g., within at least 5 meters to 10 meters). The system 700 can include one or more of the following: a server 770; an electronic device 774 (e.g., a computer 774a, a smartphone 774b, a controller 774c, and / or other devices); a head-mounted wearable device 110; and / or a wrist wearable device 188. In some embodiments, one or more of the following are communicatively coupled via a network 772: a server 770; an electronic device 774; a head-mounted wearable device 110; and / or a wrist wearable device 188. In some embodiments, the indication is provided via a head-mounted wearable device 110 that is communicatively coupled to at least one other device (e.g., a wrist wearable device 188, a smartphone 774b, a controller 774c, or other device). The indication may be provided by multiple devices in conjunction with the head-mounted wearable device 110. For example, in some embodiments, the indication is also provided via a wrist wearable device 188 that is communicatively coupled to at least one other device. In some embodiments, the indication is controlled via an artificial reality (AR) processing module 745. The AR processing module 745 may be implemented in one or more devices, such as one or more of the following: a server 770; an electronic device 774; a head-mounted wearable device 110; and / or a wrist wearable device 188. In some embodiments, the one or more devices perform the operations of the AR processing module 745 using one or more corresponding processors, alone or in conjunction with at least one other device described herein.

[0122] In some embodiments, the head-mounted wearable device 110 includes one or more components, such as a communication interface 715b, one or more sensors 725b, an illumination source 727, an AR processing module 745b, one or more imaging devices 755b (e.g., a camera), one or more processors 750b, and a memory 760b (including sensor data 762b and AR processing data 764b). In addition, in some embodiments, the head-mounted wearable device 110 includes a display 720b and one or more applications 735b. In some embodiments, the memory 760b is configured to store sensor data 762b and AR processing data 764b. Although not shown, in some embodiments, the memory 760b may include application data, device data (e.g., device hardware, device model, etc.), image data, and / or user data (e.g., data collected by using a device, data collected by using an application, user preferences, or other information stored by a user). In some embodiments, the head-mounted wearable device 110 includes smart glasses (e.g., augmented reality glasses), artificial reality head-mounted viewers (e.g., VR / AR head-mounted viewers), or other head-mounted devices. In some embodiments, one or more components of the head-mounted wearable device 110 are housed in the body of the head-mounted wearable device 110 (e.g., the frame of smart glasses, the body of an AR head-mounted viewer, etc.). In addition, in some embodiments, one or more components of the head-mounted wearable device 110 are stored in the lens of the head-mounted wearable device 110 or coupled to the lens of the head-mounted wearable device 110.

[0123] In some embodiments, the communication interface 715 is configured to communicatively couple the head-mounted wearable device 110 to one or more other devices, such as the wrist wearable device 188, the electronic device 774 (e.g., a computer 774a, a smartphone 774b, a controller 774c, a tablet computer, etc.), and / or one or more servers 770. The communication interface 715 is used to establish a wired connection or a wireless connection between the wrist wearable device 188 and other devices. In some embodiments, the communication interface 715 includes hardware capable of communicating data using any of the following protocols: a custom or standard wireless protocol (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Smart Bluetooth, ISA100.11a, Wireless Addressable Remote Sensor Highway (WirelessHART) or MiWi), a custom or standard wired protocol (e.g., Ethernet or HomePlug), and / or any other suitable communication protocol.

[0124] Optional display 720 is configured to present information, such as one or more user interfaces, one or more messages, one or more notifications (e.g., prompts, alarms, etc.), one or more images, and videos to user 130. In some embodiments, display 720 is an overhead display that displays information to the user without obstructing the user's 130 field of view.

[0125] One or more sensors 725 may include a heart rate sensor, an electromyography (EMG) sensor, an SpO2 sensor, an altimeter, a thermal sensor or thermocouple, an ambient light sensor, an ambient noise sensor, and / or an inertial measurement unit (IMU). Additional non-limiting examples of one or more sensors 725 include, for example, infrared sensors, pyroelectric sensors, ultrasonic sensors, microphone sensors, laser sensors, optical sensors, Doppler sensors, gyroscope sensors, accelerometer sensors, resonant LC sensors, capacitive sensors, acoustic sensors, and / or inductive sensors. In some embodiments, one or more sensors 725 are configured to collect additional data about the user (e.g., the impedance of the user's body). Examples of sensor data output by these sensors include body temperature data, infrared rangefinder data, location information, motion data, activity recognition data, contour detection and recognition data, posture data, heart rate data, and other wearable device data (e.g., biometric readings and outputs, accelerometer data). The one or more sensors 725 may include position sensing devices (e.g., GPS) that are configured to provide location information. In some embodiments, data measured or sensed by one or more sensors 725 is stored in memory 760. In some embodiments, the head mounted wearable device 110 uses the sensor data to communicate different messages via indications generated by the illumination source 727 that will assist the user 130 in performing activities as discussed below.

[0126] The illumination source 727 may include one or more LEDs 127 ( Figures 1A to 2D ) or similar light sources. The illumination source 727 can be caused to emit light in different patterns, emit light in different colors, emit light at different frequencies, etc., to convey different messages to the user when the user is performing an activity (e.g., exercising). More specifically, the illumination source 727 can be caused to provide a variable light-based representation of the activity that the user 130 is performing. As discussed herein, the variable light-based representation can be based on sensor data, activity applications, device data, and / or other data streams. In some embodiments, one or more illumination sources 727 can be coupled to the housing of the head-mounted wearable device 110 and / or located within one or more lenses of the head-mounted wearable device 110. One or more illumination sources 727 can be positioned at different locations of the head-mounted wearable device 110, as described above with reference to Figures 1A to 1D As shown and described. In some embodiments, one or more illumination sources 727 are positioned so that only user 130 can view the variable light-based representation, user 130 and other people near user 130 (e.g., within at least 5 meters to 10 meters) can view the variable light-based representation, and / or only other people near user 130 can view the variable light-based representation. Each illumination source 727 can be controlled individually so that each illumination source 727 can provide the same variable light-based representation or a different variable light-based representation. Transmitting different messages to the user based on the variable light-based representation is discussed below with reference to the AR processing module 745.

[0127] In some embodiments, one or more applications 735 include social media applications, banking applications, health applications, messaging applications, web browsers, gaming applications, streaming applications, media applications, imaging applications, productivity applications, social applications, etc. In some embodiments, one or more applications 735 include artificial reality applications. One or more applications 735 can be configured to provide data to the head-mounted wearable device 110, which can be used to determine a variable light-based representation. In some embodiments, one or more applications 735 can be displayed via an optional display of the head-mounted wearable device 110.

[0128] In some embodiments, the AR processing module 745 is configured to dynamically determine one or more indications that will assist the user 130 in performing an activity based at least on the sensor data. For example, the head-mounted wearable device 110 can receive biometric data from one or more biometric sensors of the wrist wearable device 188 worn by the user 130 and communicatively coupled to the head-mounted wearable device 110, and provide the received biometric data to the AR processing module 745. The AR processing module 745 uses the biometric data to determine one or more indications that will assist the user 130 in performing an activity. These indications (or variable light-based representations) can be used to convey to the user 130 that one or more physiologically based thresholds are met, such as a hydration threshold, an oxygen level threshold, a cardiovascular region threshold, a posture threshold, and / or other thresholds discussed in detail below. More specifically, the AR processing module 745 determines whether the sensor data satisfies one or more physiologically based thresholds, and determines the indication to be provided to the user 130 for conveying that the physiologically based thresholds are met. In some embodiments, the AR processing module 745 determines one or more physiological-based thresholds based on sensor data stored within a predetermined time period (e.g., 1 week, 30 days, 3 months, etc.). In some embodiments, the AR processing module 745 dynamically determines one or more physiological-based thresholds based on the activity performance of the user 130 (e.g., the physiological-based thresholds are adjusted when the user 130 is sick or performs recovery exercises). An instruction for generating the determined indication is provided to the illumination source 727 so that the illumination source 727 emits light according to the determined indication.

[0129] In some embodiments, one or more imaging devices 755 may include an ultra-wide camera, a wide camera, a telephoto camera, a depth sensing camera, or other types of cameras. In some embodiments, one or more imaging devices 755 are used to collect image data and / or video data via a wrist wearable device 188. The collected image data can be processed and stored in a memory and then presented to the user for viewing. One or more imaging devices 755 may include one or more modes for collecting image data or video data. For example, these modes may include a high-dynamic range (HDR) image acquisition mode, a low-light image acquisition mode, a continuous image acquisition mode, and other modes. In some embodiments, a specific mode is automatically selected based on the environment (e.g., lighting, movement, etc. of the device). For example, a wrist wearable device with an HDR image acquisition mode and an activated low-light image acquisition mode can automatically select an appropriate mode based on the environment (e.g., dark lighting may cause the use of a low-light image acquisition mode instead of an HDR image acquisition mode). In some embodiments, the user can select a mode. Image data and / or video data captured by one or more imaging devices 755 are stored in a memory 760 (which may include volatile memory and non-volatile memory so that image data and / or video data may be temporarily or permanently stored as needed depending on the circumstances).

[0130] One or more processors 750 may be implemented as any type of computing device, such as an integrated system on chip, a microcontroller, a fixed programmable gate array (FPGA), a microprocessor, and / or other application specific integrated circuits (ASIC). The processor may operate in conjunction with a memory 760. The memory 760 may be or include a random access memory (RAM), a read-only memory (ROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), and a magnetoresistive random access memory (MRAM), and the processor 760 may include firmware, such as static data or fixed instructions, a basic input / output system (BIOS), system functions, configuration data, and other routines used during the operation of the wrist wearable device 188 and the processor 750. The memory 760 also provides a storage area for data and instructions associated with applications and data processed by the processor 750.

[0131] In some embodiments, the memory 760 stores at least sensor data 762 and AR processing data 764. The sensor data 762 includes sensor data monitored by one or more sensors 725 of the head-mounted wearable device 110 and / or sensor data received from one or more devices communicatively coupled to the head-mounted wearable device 110, such as a wrist wearable device 188, a smartphone 774b, etc. The sensor data 562 may include sensor data collected within a predetermined time period that can be used by the AR processing module 745. The AR processing data 764 may include one or more indications previously determined by the AR processing module 745, user preferences when customizing or determining indications (e.g., color preferences, mode preferences, frequency preferences, etc.), one or more predefined physiological-based thresholds, and one or more physiological-based thresholds determined by the AR processing module 745.

[0132] The wrist wearable device 188 may include a communication interface 715a, a display 720, one or more sensors 725a, an illumination source 727a, one or more imaging devices 755a (e.g., a camera), one or more applications 735a, one or more processors 750a, and a memory 760a (including sensor data 762a and AR processing data 764a). In some embodiments, the display 720a of the wrist wearable device 188 is used as an illumination source 727a. In some embodiments, one or more components of the wrist wearable device 188 are housed in a compartment (or body) and / or strap of the wrist wearable device 188. The wrist wearable device 188 is configured to be communicatively coupled to the head-mounted wearable device 110 (or other devices (e.g., electronic device 774)) using the communication interface 715a. In some embodiments, the wrist wearable device 188 is configured to be communicatively coupled to the head-mounted wearable device 110 (or other devices (e.g., electronic device 774)) via an application programming interface (API). In some embodiments, the wrist wearable device 188 operates in conjunction with the head-mounted wearable device 110 to determine a variable light-based representation of an activity performed by the user 130. Similar to the head-mounted wearable device 110, the wrist wearable device 188 can use the AR processing module 745a to generate instructions that cause the illumination source 727a to emit light according to the determined variable light-based representation.

[0133] The electronic device 774 may also include a communication interface 715d, a display 720d, one or more sensors 725d, one or more applications 735d, an AR processing module 745d, one or more processors 750d, and a memory 760d (including sensor data 762d and AR processing data 764d). Although not shown, in some embodiments, the electronic device 774 may include an illumination source 727d and / or one or more imaging devices 755d. The electronic device 774 is configured to be communicatively coupled to the head-mounted wearable device 110 (or other device) using the communication interface 715d. In some embodiments, the electronic device 774 is configured to be communicatively coupled to the head-mounted wearable device 110 (or other device) via an application programming interface (API). In some embodiments, the electronic device 774 operates in conjunction with the head-mounted wearable device 110 to determine a variable light-based representation of an activity performed by the user 130. Similar to the head-mounted wearable device 110, the electronic device 774 may use the AR processing module 745d to generate instructions that cause the illumination source 727 to emit light according to the determined variable light-based representation.

[0134] The server 770 includes a communication interface 715c, one or more applications 735c, an AR processing module 745c, one or more processors 750c, and a memory 760c (including sensor data 762c and AR processing data 764c). In some embodiments, the server 770 is configured to receive sensor data from one or more devices (e.g., head-mounted wearable device 110, wrist wearable device 188, and / or electronic device 774) and use the received sensor data (using AR processing module 745c) to determine a variable light-based representation. The server 770 can generate instructions that cause the illumination source 727c to emit light according to the determined variable light-based representation, and provide the generated instructions to one or more communication-coupled devices (e.g., head-mounted wearable device 110).

[0135] Further embodiments also include various subsets of the above embodiments, including reference Figures 1A to 7 Embodiments as described, combined, or otherwise rearranged.

[0136] Example aspects

[0137] Several example aspects will now be briefly described.

[0138] (A1) According to some embodiments, a method for coordinating the display of biometric data at a head-mounted wearable device based on sensor data from a wrist wearable device is disclosed. The method includes: receiving an indication that a user of the head-mounted wearable device is performing a physical activity. The head-mounted wearable device includes at least one light emitting diode that is visible to the user when the user is wearing the head-mounted wearable device, and the head-mounted wearable device communicates with a wrist wearable device worn by the user during the physical activity, and the wrist wearable device is configured to sense the user's biometric data during the physical activity. The method includes: after receiving the indication and while the user is performing the physical activity, based on determining that the biometric data meets a physiologically-based threshold indicating that information about the biometric data will assist the user in performing the physical activity, causing the head-mounted wearable device to present information about the biometric data via the at least one light emitting diode of the head-mounted wearable device.

[0139] (A2) In some embodiments of A1, the method further includes: receiving position data sensed by one or more sensors of a wrist-mounted wearable device when the user performs physical activity, the one or more sensors being different from biometric sensors used to sense biometric data; and based on determining that the position data indicates that the user needs guidance when performing the physical activity, causing the head-mounted wearable device to present guidance via at least one light emitting diode of the head-mounted wearable device to assist the user in performing the physical activity.

[0140] (A3) In some embodiments of A2, guidance for assisting a user in performing physical activity is presented in conjunction with audio feedback presented via a speaker of a head-mounted wearable device, which audio feedback also assists the user in performing the physical activity.

[0141] (A4) In some embodiments of A2, when it is determined that the position data indicates that the user is performing the physical activity incorrectly, it is determined that the position data indicates that the user needs guidance.

[0142] (A5) In some embodiments of any one of A1 to A4, the head-mounted wearable device includes a plurality of light-emitting diodes, the plurality of light-emitting diodes including the at least one light-emitting diode, and information about the biometric data is provided using one or more of the plurality of light-emitting diodes.

[0143] (A6) In some embodiments of any one of A1 to A4, the head-mounted wearable device includes a plurality of light-emitting diodes, the plurality of light-emitting diodes including the at least one light-emitting diode, the information about the biometric data is provided using only the at least one light-emitting diode, and the wrist-mounted wearable device is configured to monitor additional biometric data of the user during physical activity, the additional biometric data being sensed using an additional biometric sensor that is different from the biometric sensor used to send the biometric data. The method also includes: while the user is performing physical activity, based on determining that the additional biometric data satisfies an additional physiological-based threshold indicating that the information about the additional biometric data will assist the user in performing physical activity, causing the head-mounted wearable device to present the information about the additional biometric data via an additional light-emitting diode in the plurality of light-emitting diodes, the additional physiological-based threshold being different from the physiological-based threshold.

[0144] (A7) In some embodiments of A6, the information about the additional biometric data and the information about the biometric data are caused to be presented via the additional light emitting diode and the at least one light emitting diode, respectively, during an overlapping time period.

[0145] (A8) In some embodiments of A6 or A7, the wrist-mounted wearable device is configured to monitor additional biometric data of the user during physical activity, the additional biometric data being sensed using another biometric sensor that is different from the biometric sensor and the additional biometric sensor. The method further includes: while the user is performing the physical activity, based on determining that the additional biometric data satisfies another physiological-based threshold indicating that information about the additional biometric data will assist the user in performing the physical activity, causing the head-mounted wearable device to present information about the additional biometric data via another light-emitting diode among the plurality of light-emitting diodes, the other physiological-based threshold being different from the physiological-based threshold and the additional physiological-based threshold.

[0146] (A9) In some embodiments of A8, information about the additional biometric data, information about the biometric data, and information about the further biometric data are caused to be presented via the additional light-emitting diode, the at least one light-emitting diode, and the further light-emitting diode, respectively, during overlapping time periods.

[0147] (A10) In some embodiments of any one of A1 to A9, the physiological-based threshold is associated with a type of physical activity performed by the user, the biometric data sensed by the wrist wearable device is a first type of biometric data, and at least one different physiological-based threshold that is different from the physiological-based threshold is used to determine when to present information about the first type of biometric data when the user performs different physical activities.

[0148] (A11) In some embodiments of any one of A1 to A10, the method further includes: in response to determining that a user of the head-mounted wearable device is performing physical activity, monitoring, by the head-mounted wearable device, biometric data sensed by a biometric sensor of the head-mounted wearable device, and, based on determining that the biometric data sensed by the biometric sensor of the head-mounted wearable device satisfies a physiological-based threshold indicating that information about the biometric data will assist the user in performing physical activity, causing the head-mounted wearable device to present information about the biometric data sensed by the biometric sensor of the head-mounted wearable device via another light emitting diode of the head-mounted wearable device.

[0149] (A12) In some embodiments of A11, the other light-emitting diode is the same as the at least one light-emitting diode, biometric data sensed by the biometric sensor of the head-mounted wearable device and the biometric data of the user sensed by the wrist wearable device during physical activity are analyzed together to generate merged biometric data, and information about the biometric data sensed by the biometric sensor of the head-mounted wearable device and information about the biometric data are presented by causing the information about the merged biometric data to be presented.

[0150] (A13) In some embodiments of any one of A1 to A12, the method further includes: in response to determining that a user of the head-mounted wearable device is performing a physical activity, monitoring, by the head-mounted wearable device, position data sensed by one or more sensors of the head-mounted wearable device. The method further includes: based on determining that the position data sensed by one or more sensors of the head-mounted wearable device indicates that the user needs guidance when performing the physical activity, causing the head-mounted wearable device to present guidance via at least one light emitting diode of the head-mounted wearable device to assist the user in performing the physical activity.

[0151] (A14) In some embodiments of any one of A1 to A13, determining that a user of a head-mounted wearable device is performing physical activity is based on one or more of: location data sensed by one or more sensors of the head-mounted wearable device; biometric data sensed by a biometric sensor of the head-mounted wearable device; and input commands provided by the user at the head-mounted wearable device.

[0152] (A15) In some embodiments of A14, determining that a user of the head-mounted wearable device is performing physical activity is also based on one or more of the following: location data sensed by one or more sensors of the wrist-mounted wearable device; biometric data sensed by a biometric sensor of the wrist-mounted wearable device; and input commands provided by the user at the wrist-mounted wearable device.

[0153] (A16) In some embodiments of any one of A1 to A15, the head-mounted wearable device includes a housing, and the at least one light-emitting diode is coupled to the housing of the head-mounted wearable device.

[0154] (A17) In some embodiments of any one of A1 to A16, the head-mounted wearable device includes one or more lenses, and the at least one light-emitting diode is coupled to the one or more lenses of the head-mounted wearable device.

[0155] (A18) In some embodiments of any one of A1 to A17, the physiological-based threshold includes one or more of: a hydration threshold; a speed threshold; an oxygen level threshold; a heart rate zone threshold; a stress threshold; and a posture threshold.

[0156] (A19) In some embodiments of any one of A1 to A18, the head-mounted wearable device does not include a heads-up display.

[0157] (A20) In some embodiments of any one of A1 to A19, causing information about biometric data to be presented via at least one light emitting diode of a head-mounted wearable device includes causing the light emitting diode to emit light using one or more of multiple modes, multiple frequencies, and multiple colors.

[0158] (B1) According to some embodiments, a head-mounted wearable device for coordinating the display of biometric data, the head-mounted wearable device being configured to perform or cause the performance of any of the methods A1 to A20.

[0159] (C1) According to some embodiments, a system for coordinating the display of biometric data at a head-mounted wearable device based on sensor data from a wrist-mounted wearable device is configured to perform or cause the performance of any of the methods A1 to A20.

[0160] (D1) According to some embodiments, a non-transitory computer-readable storage medium includes instructions that, when executed by a head-mounted wearable device, a wrist-mounted wearable device, or an intermediate device that communicates with the head-mounted wearable device and the wrist-mounted wearable device, causes the execution of any one of methods A1 to A20.

[0161] (E1) On the other hand, a device on a wrist wearable device, a head wearable device and / or an intermediate device is used to perform or cause the method of any one of claims A1 to A20 to be performed.

[0162] (F1) According to some embodiments, a wrist wearable device for coordinating the display of biometric data, the wrist wearable device being configured to perform or cause the performance of the method of any one of claims A1 to A20.

[0163] Any data collection performed by the various devices described herein and / or any device configured to perform or cause the performance of the various embodiments described above with reference to any of the figures (hereinafter referred to as a "device") is performed with the user's consent and in a manner that complies with all applicable privacy laws. The user is provided with the option to allow the device to collect data and the option to limit or deny the device from collecting data. The user can choose to enable or disable any data collection at any time. In addition, the user is provided with the option to request deletion of any collected data.

[0164] It will be understood that, although the terms "first", "second", etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0165] The terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the claims. As used in the description of the embodiments and the appended claims, unless the context clearly indicates otherwise, the singular forms "one", "an" and "the" are also intended to include plural forms. It will also be understood that the terms "and / or" used herein refer to and encompass any and all possible combinations of one or more items in the associated listed items. It will also be understood that the terms "include" and / or "comprising" when used in this specification specify the presence of stated features, wholes, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, parts and / or their combinations.

[0166] As used herein, the term “if” may be interpreted as: “when the stated conditional precedent is true”; or “when the stated conditional precedent is true”; or “in response to determining” that the stated conditional precedent is true; or “upon determining” that the stated conditional precedent is true; or “in response to detecting” that the stated conditional precedent is true, depending on the context. Similarly, the phrase “if it is determined that [the stated conditional precedent is true]” or “if [the stated conditional precedent is true]” or “when [the stated conditional precedent is true]” may be interpreted as: “upon determining that the stated conditional precedent is true”; or “in response to determining” that the stated conditional precedent is true; or “upon determining” that the stated conditional precedent is true; or “upon detecting” that the stated conditional precedent is true”; or “in response to detecting” that the stated conditional precedent is true, depending on the context.

[0167] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the claims to the precise forms disclosed. In view of the above teachings, many modifications and variations are possible. The various embodiments are selected and described in order to best explain the principles of operation and practical application, thereby enabling others skilled in the art to realize.

Claims

1. A method for coordinating display of biometric data at a head-mounted wearable device based on sensor data from a wrist-mounted wearable device, the method comprising: Receiving an indication that a user of a head mounted wearable device is performing a physical activity, wherein: The head-mounted wearable device includes at least one light emitting diode, and the at least one light emitting diode is visible to the user when the user wears the head-mounted wearable device, and The head-mounted wearable device communicates with a wrist-mounted wearable device worn by the user during the physical activity, the wrist-mounted wearable device being configured to sense biometric data of the user during the physical activity; and After receiving the indication, and while the user is performing the physical activity: Based on determining that the biometric data satisfies a physiologically-based threshold indicating that information about the biometric data will assist the user in performing the physical activity, causing the head-mounted wearable device to present information about the biometric data via the at least one light emitting diode of the head-mounted wearable device.

2. The method according to claim 1, further comprising: While the user is performing the physical activity: receiving position data sensed by one or more sensors of the wrist wearable device, the one or more sensors being different from a biometric sensor used to sense the biometric data; and Based on determining that the position data indicates that the user needs guidance when performing the physical activity, the head-mounted wearable device presents guidance via the at least one light emitting diode of the head-mounted wearable device to assist the user in performing the physical activity.

3. The method according to claim 2, wherein: The guidance that assists the user in performing the physical activity is caused to be presented in conjunction with audio feedback presented via a speaker of the head-mounted wearable device, the audio feedback also assisting the user in performing the physical activity.

4. The method according to claim 2, wherein: When it is determined that the position data indicates that the user is performing the physical activity incorrectly, it is determined that the position data indicates that the user needs guidance.

5. The method according to claim 1, wherein: The head-mounted wearable device includes a plurality of light emitting diodes, the plurality of light emitting diodes including the at least one light emitting diode, and Information regarding the biometric data is caused to be provided using one or more light emitting diodes of the plurality of light emitting diodes.

6. The method according to claim 1, wherein: The head-mounted wearable device includes a plurality of light emitting diodes, wherein the plurality of light emitting diodes includes the at least one light emitting diode, information about said biometric data is caused to be provided using only said at least one light emitting diode, The wrist wearable device is configured to monitor additional biometric data of the user during the physical activity, the additional biometric data being sensed using an additional biometric sensor that is different from the biometric sensor used to transmit the biometric data, and The method further comprises: While the user is performing the physical activity: Based on determining that the additional biometric data satisfies an additional physiological-based threshold indicating that information about the additional biometric data will assist the user in performing the physical activity, the head-mounted wearable device presents information about the additional biometric data via an additional light-emitting diode among the multiple light-emitting diodes, and the additional physiological-based threshold is different from the physiological-based threshold.

7. The method according to claim 6, wherein: The information regarding the additional biometric data and the information regarding the biometric data are caused to be presented via the additional light emitting diode and the at least one light emitting diode, respectively, during an overlapping time period.

8. The method according to claim 6, wherein: The wrist wearable device is configured to monitor additional biometric data of the user during the physical activity, the additional biometric data being sensed using another biometric sensor that is different from the biometric sensor and the additional biometric sensor, and The method further comprises: While the user is performing the physical activity: Based on determining that the additional biometric data satisfies another physiological-based threshold indicating that information about the additional biometric data will assist the user in performing the physical activity, the head-mounted wearable device presents information about the additional biometric data via another light-emitting diode among the multiple light-emitting diodes, and the other physiological-based threshold is different from the physiological-based threshold and the additional physiological-based threshold.

9. The method according to claim 8, wherein: The information about the additional biometric data, the information about the biometric data and the information about the further biometric data are caused to be presented during an overlapping time period via the additional light emitting diode, the at least one light emitting diode and the further light emitting diode, respectively.

10. The method according to claim 1, wherein: The physiologically based threshold is associated with a type of physical activity performed by the user, The biometric data sensed by the wrist wearable device is a first type of biometric data, and At least one different physiological-based threshold value, different from the physiological-based threshold value, is used to determine when to cause presentation of information regarding the first type of biometric data when the user performs a different physical activity.

11. The method according to claim 1, further comprising: In response to determining that the user of the head-mounted wearable device is performing the physical activity, monitoring, by the head-mounted wearable device, biometric data sensed by a biometric sensor of the head-mounted wearable device; and Based on determining that the biometric data sensed by the biometric sensor of the head-mounted wearable device satisfies the physiological-based threshold indicating that information about the biometric data will assist the user in performing the physical activity, the head-mounted wearable device presents information about the biometric data sensed by the biometric sensor of the head-mounted wearable device via another light emitting diode of the head-mounted wearable device.

12. The method according to claim 11, wherein: The another light emitting diode is identical to the at least one light emitting diode, Biometric data sensed by the biometric sensor of the head-mounted wearable device and biometric data of the user sensed by the wrist-mounted wearable device during the physical activity are analyzed together to generate combined biometric data, and The information about the biometric data sensed by the biometric sensor of the head mounted wearable device and the information about the biometric data are caused to be presented by causing the information about the merged biometric data to be presented.

13. The method according to claim 1, further comprising: In response to determining that the user of the head-mounted wearable device is performing the physical activity, monitoring, by the head-mounted wearable device, position data sensed by one or more sensors of the head-mounted wearable device; as well as Based on determining that the position data sensed by the one or more sensors of the head-mounted wearable device indicates that the user needs guidance when performing the physical activity, the head-mounted wearable device presents guidance via the at least one light emitting diode of the head-mounted wearable device to assist the user in performing the physical activity.

14. The method according to claim 1, wherein: Determining that the user of the head-mounted wearable device is performing the physical activity is based on one or more of: position data sensed by one or more sensors of the head-mounted wearable device; biometric data sensed by a biometric sensor of the head-mounted wearable device; and input commands provided by the user at the head-mounted wearable device.

15. The method according to claim 14, wherein: Determining that the user of the head-mounted wearable device is performing the physical activity is also based on one or more of the following: position data sensed by one or more sensors of the wrist-mounted wearable device; biometric data sensed by a biometric sensor of the wrist-mounted wearable device; and input commands provided by the user at the wrist-mounted wearable device.

16. The method according to claim 1, wherein: The head-mounted wearable device includes a housing, and the at least one light emitting diode is coupled to the housing of the head-mounted wearable device.

17. The method according to claim 1, wherein: The head-mounted wearable device includes one or more lenses, and the at least one light emitting diode is coupled to the one or more lenses of the head-mounted wearable device.

18. The method according to claim 1, wherein: The physiologically based thresholds include one or more of: a hydration threshold; a speed threshold; an oxygen level threshold; a heart rate zone threshold; a stress threshold; and a posture threshold.

19. A head mounted wearable device for coordinating the display of biometric data, the head mounted wearable device being configured to perform or cause to be performed: Receiving an indication that a user of a head mounted wearable device is performing a physical activity, wherein: The head-mounted wearable device includes at least one light emitting diode, and the at least one light emitting diode is visible to the user when the user wears the head-mounted wearable device, and The head-mounted wearable device communicates with a wrist-mounted wearable device worn by the user during the physical activity, the wrist-mounted wearable device being configured to sense biometric data of the user during the physical activity; and After receiving the indication, and while the user is performing the physical activity: Based on determining that the biometric data satisfies a physiologically-based threshold indicating that information about the biometric data will assist the user in performing the physical activity, causing the head-mounted wearable device to present information about the biometric data via at least one light emitting diode of the head-mounted wearable device.

20. A non-transitory computer-readable storage medium comprising instructions that, when executed by one or more processors of a head-mounted wearable device, cause the head-mounted wearable device to: Receiving an indication that a user of a head mounted wearable device is performing a physical activity, wherein: The head-mounted wearable device includes at least one light emitting diode, and the at least one light emitting diode is visible to the user when the user wears the head-mounted wearable device, and The head-mounted wearable device communicates with a wrist-mounted wearable device worn by the user during the physical activity, the wrist-mounted wearable device being configured to sense biometric data of the user during the physical activity; and After receiving the indication, and while the user is performing the physical activity: Based on determining that the biometric data satisfies a physiologically-based threshold indicating that information about the biometric data will assist the user in performing the physical activity, causing the head-mounted wearable device to present information about the biometric data via at least one light emitting diode of the head-mounted wearable device.

Images