Detecting neuromuscular signals at wearable device to facilitate performance of physical activity and methods and systems thereof

Abstract

Systems and methods are described for using such signals detected by a neuromuscular signal sensor of a wearable electronic device while a user is performing an activity. A method for adaptively adjusting the rate of physical activity performed by a user while the user wears a wearable electronic device is described. An example method includes, in conjunction with detecting that a user is performing physical activity at a particular activity rate using one or more sensors located at a wearable electronic device: (i) detecting a degree of force of the user using a neuromuscular signal sensor located at the wearable electronic device, and (ii) based on determining that the degree of force differs from the reference degree of force by at least a threshold amount, determining an adjustment to the particular rate of activity while the user is performing the physical activity.

Description

Technical Field

[0001] The present application generally relates to systems including wearable electronic devices (e.g., wrist-wearable devices, head-wearable devices, leg-wearable devices), the wearable electronic devices including but not limited to wearable electronic devices configured to detect neuromuscular signals of a user, such as neuromuscular signals corresponding to: (i) the exertion of the user during physical activity, (ii) the repetition of a physical activity performed by the user, and / or (iii) an alert gesture performed by the user. Background Art

[0002] Wearable electronic devices (e.g., smartwatches and smartbands) are becoming increasingly popular, and wearable electronic devices can be configured to detect certain aspects of a user's movement (e.g., an estimate of the number of steps taken), thereby allowing the user to receive data regarding their health and performance. However, current wearable electronic devices are unable to accurately detect and facilitate the execution of physical activity by the user. For example, some wearable electronic devices have sensors (e.g., inertial measurement unit (IMU) sensors) that can be configured to detect user movement, but cannot detect other aspects of the user's execution (e.g., the degree of exertion). This deficiency means that these devices are prone to false alarms, cannot guide the user's execution, and have limited ability to optimize sensing for a higher level of accuracy and / or battery life. In particular, handheld sensors (e.g., handheld heart rate sensors) are vulnerable to false readings, depending on the user's physiology and exercise habits, including how the sensor is gripped and / or the degree to which the user's hand is sweating. Summary of the Invention

[0003] According to a first aspect of the present disclosure, there is provided a method for determining an adaptive adjustment to a physical activity being performed by a user wearing a wearable electronic device, the method comprising: detecting, using one or more sensors located at the wearable electronic device, that the user is performing a physical activity at a particular activity rate; detecting, using a neuromuscular signal sensor located at the wearable electronic device, the degree of exertion of the user; and determining an adjustment to the particular activity rate at which the user is performing the physical activity based on determining that the degree of exertion differs from a reference degree of exertion by at least a threshold amount.

[0004] In some embodiments, the method further comprises: prior to determining that the degree of exertion differs from the reference degree of exertion by at least the threshold amount: detecting, separately from detecting the degree of exertion, another degree of exertion using the neuromuscular signal sensor; and providing a notification to the user based on determining that the another degree of exertion differs from the reference degree of exertion by at least another threshold amount.

[0005] In some embodiments, a notification is provided at an electronic exercise device; the notification includes selectable user interface elements; and in response to a user selecting the selectable user interface elements, the electronic exercise device adjusts an activity rate.

[0006] In some embodiments, the method further includes: after determining that the other exertion level differs from a baseline exertion level by at least another threshold amount and before determining that the exertion level differs from the baseline exertion level by at least a threshold amount: separately from detecting the exertion level and detecting the other exertion level, detecting yet another exertion level using a neuromuscular signal sensor of the one or more sensors located at the wearable electronic device; and based on determining that the yet another exertion level differs from the baseline exertion level by yet another threshold amount, determining another adjustment to a specific activity rate, the another adjustment being different from the adjustment.

[0007] In some embodiments, another adjustment to a specific activity rate is to decrease the activity rate; and the adjustment to a specific activity rate is to stop body activity.

[0008] In some embodiments, a user is performing body activity at an electronic exercise device; and determining that the exertion level differs from the baseline exertion level is further based on data from the electronic exercise device.

[0009] In some embodiments, data from the electronic exercise device is generated by one or more of the following: (i) a photoplethysmography (PPG) sensing device; (ii) an electrocardiogram (ECG) sensing device; and (iii) a gyroscope sensor.

[0010] In some embodiments, the one or more sensors at the wearable electronic device include an inertial measurement unit (IMU) sensor; and determining that the exertion level differs from the baseline exertion level is further based on data from the IMU sensor.

[0011] In some embodiments, determining that the exertion level differs from the baseline exertion level further includes: applying a first weight to data from the neuromuscular signal sensor; and applying a second weight to data from the IMU sensor.

[0012] In some embodiments, the first weight and the second weight are based on a calibration performed by the user before performing body activity at the activity rate.

[0013] In some embodiments, the first weight and the second weight are based on the type of body activity being performed.

[0014] In some embodiments, based on determining that the neuromuscular signal sensor is configured to sense a primary muscle group associated with a body activity, a first weight is higher than a second weight.

[0015] In some embodiments, the activity rate is a first activity rate; the exertion level is lower than a reference exertion level by a threshold amount; and the first activity rate is automatically increased to a second activity rate, where the second activity rate is higher than the first activity rate.

[0016] In some embodiments, the exertion level is higher than a reference exertion level by a threshold amount; and the adjustment causes the electronic exercise device to stop operating to cause the body activity.

[0017] In some embodiments, the user wears a head-worn electronic device; the head-worn electronic device presents a user interface to the user, the user interface corresponding to an interactive exercise; and based on determining that the exertion level differs from a reference exertion level by a threshold amount, the adjustment to the body activity includes: causing an adjustment to the user interface presented by the head-worn electronic device.

[0018] In some embodiments, determining that the exertion level differs from a reference exertion level by a threshold amount is further based on the number of identified repetitions of the body activity that the user has performed at the activity rate.

[0019] In some embodiments, determining that the exertion level differs from a reference exertion level by a threshold amount is further based on the number of identified repetitions of another body activity that the user has performed, where the another body activity is different from the body activity that the user is currently performing.

[0020] In some embodiments, the threshold amount is based on a plurality of context criteria associated with the user; and the threshold amount is determined based on inputting one or more of the plurality of context criteria into a machine learning model.

[0021] According to a second aspect of the present disclosure, there is provided a wrist-worn device including: a display; one or more processors; and a memory including instructions that, when executed by the wrist-worn device, cause the following operations to be performed: detecting, in combination with one or more sensors located at the wrist-worn device, that the user is performing a body activity at a particular activity rate; detecting, using a neuromuscular signal sensor located at the wrist-worn device, the user's exertion level; and determining an adjustment to the particular activity rate at which the user is performing the body activity based on determining that the exertion level differs from a reference exertion level by at least a threshold amount.

[0022] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium including instructions that, when executed by a computing device, cause the computing device to perform the following operations: detecting, in combination with one or more sensors located at a wearable electronic device, that a user is performing a physical activity at a particular activity rate; detecting, using a neuromuscular signal sensor located at the wearable electronic device, the user's exertion level; and determining an adjustment to the particular activity rate at which the user is performing the physical activity based on determining that the exertion level differs from a baseline exertion level by at least a threshold amount.

[0023] The embodiments discussed herein address one or more drawbacks of current wearable electronic devices, for example, by providing sensors (e.g., electromyography (EMG) sensors, inertial measurement unit sensors) and other means (e.g., machine learning and / or other forms of artificial intelligence) for determining aspects of a user's health and / or aspects of a physical activity the user is performing, and means (e.g., user interfaces, gesture spaces) for interacting with the wearable electronic device and / or another electronic device (e.g., an electronic fitness apparatus) that communicates with the wearable electronic device. The systems, devices, and methods described herein facilitate improved human-machine interfaces by providing convenient, efficient, and intuitive control and analysis of the physical activities the user is performing.

[0024] As a non-limiting example, a wrist-worn device (e.g., a smartwatch) detects that a user is performing a physical activity at a particular exertion level. Based on the type of activity detected that the user is performing, power is supplied to one or more sensors (e.g., a subset of sensor channels of a particular type of sensor) of the wrist-worn device, the wrist-worn device being configured to detect neuromuscular signals in a subset of the user's muscles (e.g., including primary and / or non-primary muscles) of the user's body. For example, a particular subset of channels of a corresponding type of neuromuscular signal sensor (e.g., an EMG sensor) is configured to detect bicep curls, and a different subset of channels is configured to detect bench presses. Thus, if the user is performing bicep curls, the particular subset is powered and / or data is collected from the particular subset. If the user is performing a bench press, the different subset is powered and / or data is collected from the different subset. Each subset may include sensor channels that are not configured to detect the primary muscle groups for a particular physical activity, but are selected to improve the accuracy of detecting the performance of one or more repetitions of a particular physical activity via non-primary muscle activity.

[0025] Continuing with this example, data from the powered sensor can be used to determine whether the user of the wrist-worn device is exerting more (or less) force than the user normally exerts during exercise. In such a case, the wrist-worn device or another electronic device in electronic communication with the wrist-worn device can notify the user of the higher-than-normal level of exertion and / or can actively adjust the activity rate (e.g., resistance level) associated with the physical activity. One potential technical advantage of this example is to provide an efficient human-machine interface; this is particularly useful when the user is performing an activity that requires their full attention and engagement in the physical activity.

[0026] Continuing with this example, the data from the sensor can also be used in combination with other sensor data (e.g., IMU sensor data) to determine the number of repetitions and / or sets of the corresponding physical activity that the user has performed. This determination can have increased accuracy when it includes detecting the exertion of a particular muscle group of the user during a particular part of the corresponding physical activity.

[0027] Continuing with this example, the user may want to know if they are relaxed enough before starting a physical activity (e.g., the tension level is below a threshold tension level). For example, a user playing golf (e.g., on an actual golf course or in a virtual golf course) may want to have a low tension level before swinging a golf club (e.g., performing a repetition of the corresponding physical activity). The user may also want to know that they have a relatively high tension level when performing other activities (including other physical activities) and / or other parts of the golf activity.

[0028] As another example, the wearable electronic device can be configured so that the user can quickly send an alert (e.g., when injured or in danger). The alert can include context information about the user (e.g., the user's physical location, imaging data of the user's surroundings, and / or an audio recording of the user or their surroundings). The alert can be sent to a remote electronic device in response to the user performing an alert gesture. The alert gesture can be a combination of individual sub-gestures (e.g., a pinch sub-gesture accompanied by an eye movement sub-gesture).

[0029] According to some embodiments, a method for determining an adaptive adjustment to a physical activity being performed by a user wearing a wearable electronic device is provided. The method includes: detecting, using one or more sensors located at the wearable electronic device, that the user is performing a physical activity at a particular activity rate: (i) detecting, using a neuromuscular signal sensor located at the wearable electronic device, the level of exertion of the user; and (ii) determining an adjustment to the particular activity rate at which the user is performing the physical activity based on determining that the level of exertion differs from a baseline level of exertion by at least a threshold amount.

[0030] According to some embodiments, a method for determining a repetition of a physical activity being performed by a user is provided. The method includes: at a wearable electronic device having one or more sensors including a neuromuscular signal sensor: (i) detecting, using the one or more sensors, the physical activity being performed by the user; (ii) identifying, based on the detected physical activity being performed, one or more repetitions of the physical activity using data from the neuromuscular signal sensor, wherein identifying the one or more repetitions includes identifying at least one of: (a) a first level associated with a local maximum exertion of the physical activity, or (b) a second level associated with a local minimum exertion of the physical activity; and (iii) obtaining a count of the one or more repetitions to produce a count of the identified repetitions of the physical activity being performed.

[0031] According to some embodiments, a method for transmitting an alert to a remote device is provided. The method includes: at a wearable electronic device in communication with a neuromuscular signal sensor, determining that a user is performing an alert gesture based on data from the neuromuscular signal sensor. The method further includes: in response to the alert gesture and without further user input, (i) obtaining context information of the alert gesture, and (ii) causing a notification and the context information to be sent to the remote device.

[0032] According to some embodiments, a method for detecting a relaxation state of a user is provided. The method includes: determining, using data from a neuromuscular signal sensor, a tension level of a particular muscle group of the user (e.g., a level corresponding to a predefined signal-to-noise ratio). The method further includes: causing a notification indicating that the user is in a relaxation state to be provided to the user based on determining that the tension level meets one or more predefined criteria.

[0033] In some embodiments, a computing system (e.g., an artificial-reality (AR) system including a wrist wearable device and / or a head wearable device) includes: one or more processors; a memory; one or more programs stored in the memory; and optionally one or more means for presenting a user interface (e.g., a display or a projector). The one or more programs are configured to be executed by the one or more processors. The one or more programs include instructions for performing any of the methods described herein (e.g., method 900, method 1000, method 1100, and method 1200 described below).

[0034] In some embodiments, a non-transitory computer-readable storage medium stores one or more programs configured to be executed by a computing device (e.g., a wrist-wearable device or a head-wearable device, or another connected device such as a smartphone, a desktop computer, or a laptop computer configured to coordinate operations at the wrist-wearable device and the head-wearable device), the computing device having one or more processors, a memory, and an optional display. The one or more programs include instructions for performing (or causing to be performed) any of the methods described herein (e.g., method 900, method 1000, method 1100, and method 1200 described below).

[0035] Accordingly, systems and methods are provided for detecting and / or facilitating a user's performance of physical activities. Such methods and systems can supplement or replace conventional methods and systems for detecting and / or facilitating a user's performance of physical activities.

[0036] The features and advantages described in the specification are not necessarily all inclusive, and in particular, given the figures, specification, and claims provided in this disclosure, some additional features and advantages will be apparent to those of ordinary skill in the art. Further, it should be noted that the terminology used in this specification is primarily selected for readability and guidance purposes and is not necessarily selected to depict or limit the subject matter described herein.

[0037] It will be recognized that any feature described herein as being suitable for incorporation into one or more aspects or embodiments of the present disclosure is intended to be generalizable to any and all aspects and embodiments of the present disclosure. Those skilled in the art can understand other aspects of the present disclosure based on the specification, claims, and figures of the present disclosure. The foregoing general description and the following detailed description are merely exemplary and explanatory and are not limiting of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] For a more detailed understanding of the present disclosure, a more specific description can be obtained by reference to the features of the various embodiments, some of which are illustrated in the figures. However, the figures only illustrate the relevant features of the present disclosure and are not necessarily considered limiting, as those skilled in the art will recognize when reading this disclosure, since the present specification can be applied to other effective features.

[0039] Figures 1A to 1M An example user scenario of performing a physical activity while wearing a wearable device is shown in accordance with some embodiments.

[0040] Figures 2A to 2E Another example user scenario of performing a physical activity while wearing a wearable device is shown in accordance with some embodiments.

[0041] Figures 3A to 3I Shows another example user scenario of performing a physical activity while wearing a wearable device according to some embodiments.

[0042] Figures 4A to 4E Shows another example user scenario of performing a physical activity while wearing a wearable device according to some embodiments.

[0043] Figure 5A 、 Figure 5B 、 Figure 5C-1 、 Figure 5C-2 、 Figure 5D-1 and Figure 5D-2 Shows an example AR system according to some embodiments.

[0044] Figure 6A and Figure 6B Shows an example wrist wearable device according to some embodiments.

[0045] Figure 7A 、 Figure 7B-1 、 Figure 7B-2 and Figure 7C Shows an example AR system according to some embodiments.

[0046] Figure 8A and Figure 8B Shows an example handheld device according to some embodiments.

[0047] Figures 9A to 9C Shows an example wearable glove according to some embodiments.

[0048] Figure 10 Shows a flowchart of an example method for adaptive adjustment of physical activity according to some embodiments.

[0049] Figure 11 Shows a flowchart of an example method for determining the repetition of a physical activity according to some embodiments.

[0050] Figure 12 Shows a flowchart of an example method for providing an alert to a remote device according to some embodiments.

[0051] Figure 13 Shows a flowchart of an example method for detecting a user's relaxation state according to some embodiments.

[0052] By convention, the various features shown in the drawings are not necessarily drawn to scale, and like reference numerals may be used throughout the specification and drawings to indicate like features. Detailed Description

[0053] Users may have different performance levels when performing the same or similar physical activities (e.g., exercises such as bicep curls, bench presses, rows, jogging, golf, chess). For example, the performance level can vary based on diet, rest, previous exercise, and injuries. It is beneficial for users to be able to monitor their fatigue level / effort and adjust their exercise accordingly, or otherwise facilitate their goals regarding the corresponding activity. As described herein, the effort can be monitored via one or more neuromuscular signal sensors of a wearable device. Based on the effort, parameters of the exercise (e.g., speed, weight, and / or number of repetitions) can be adjusted. In this way, users can maximize their workout while avoiding injury.

[0054] In addition to detecting the effort during exercise, the one or more neuromuscular signal sensors can also detect whether the user is relaxed. For example, the wearable device can determine whether the user's stress level is below a specific threshold based on the neuromuscular signals detected by one or more sensors. In this way, the wearable device can notify the user whether they are relaxed enough (e.g., for meditation purposes or to prepare for exercise). For example, a chess player can associate a specific relaxation level with a desired level of mental clarity. The chess player may want to know whether they have reached a specific relaxation level in a timely manner (e.g., 30 seconds, 15 seconds).

[0055] The effort monitored by the one or more neuromuscular signal sensors can be combined with motion data obtained by the wearable device (or an intermediate device communicating with the wearable device) (e.g., detected via an IMU or a vision tracking sensor). In this way, repetitions of an exercise (e.g., running laps around a track, squats, lifting a barbell) can be tracked and counted. Additionally, exercise repetitions can be distinguished from other similar movements to ensure accurate counting. For example, lifting a barbell can be distinguished from the user lifting a water bottle or a towel to their face.

[0056] By monitoring muscle signals, a health condition can be detected and the user can be alerted. For example, the one or more neuromuscular signal sensors are capable of detecting and distinguishing different types of muscle contractions. In this way, the wearable device can detect involuntary movements associated with certain health risks and alert the user to seek medical attention. In some embodiments, an AR-based gesture model can be utilized to determine whether the user has performed an involuntary movement.

[0057] Similarly as described herein, a user suffering a medical emergency or encountering a dangerous situation can send a distress signal via an alert gesture. For example, the one or more neuromuscular signal sensors can detect signals corresponding to minute hand movements, and the wearable device can send an alert to one or more remote persons. Additionally, the wearable device can send context information to the one or more remote persons so that these persons can better assist the user. The context information can include images, audio data, location data, health data, etc. In this way, the user can receive help faster and more efficiently.

[0058] Turning now to the drawings, Figures 1A to 1M illustrates an example user scenario of performing a physical activity while wearing a wearable device according to some embodiments. In some embodiments, the wearable device is a component of a computing system (e.g., a constellation of devices) that includes multiple electronic devices or communicates with such a computing system, which can include additional wearable devices and / or their constituent components (e.g., a holding structure, a band portion, a companion device, a portable computing unit, a remote server). In some embodiments, notifications can be provided to the user based on which devices in the device constellation are being used in conjunction with the execution of a particular physical activity.

[0059] Figure 1A illustrates user 101 sitting on a fitness bench, holding dumbbell 111. According to some embodiments, Figures 1A to 1M neither the illustrated fitness bench nor the dumbbell includes any electronic components. User 101 is wearing a wrist wearable device 102 (e.g., a smartwatch). In some embodiments, the wrist wearable device 102 includes Figure 6A and Figure 6B some or all of the multiple components of the illustrated example wrist wearable device. Figures 1A to 1M The wrist wearable device 102 in [ ] includes a display 104 (e.g., a touch-sensitive display) for displaying a user interface and corresponding user interface elements. In some embodiments, the corresponding user interface elements can be presented on one or more of the multiple electronic devices.

[0060] In Figure 1AIn this case, the display 104 presents a user interface 107 that includes a clock user interface element 106 for displaying the current time (e.g., "6:30 PM"). In some embodiments, in addition to or instead of displaying the user interface 107 at the wrist-worn device, a similar user interface is displayed on another display (e.g., a display of a television and / or fitness equipment). In addition to the clock user interface element 106, the user interface 107 further includes a plurality of user interface elements. For example, a notification user interface element 108 that includes a text indicator (e.g., the statement: "Body activity detected - bicep curl") indicating the type of body activity that the user 101 is performing. In some embodiments, the wrist-worn device 102 determines the type of body activity based on signals from one or more sensors of the wrist-worn device 102 and / or another electronic device in communication with the wrist-worn device 102. The notification user interface element 108 further includes another text indication (e.g., the statement: "Your device was calibrated for this exercise two weeks ago.") indicating the amount of time since the user 101 calibrated the sensor for that body activity.

[0061] The user interface 107 further includes a repetition indicator 116, which is represented as a circular user interface element, for example. The repetition indicator 116 includes the number of repetitions of bicep curls that the user 101 has performed during a certain part of the activity (e.g., in Figure 1A this case, the count is one repetition). The repetition indicator 116 includes a prediction accuracy of the repetitions, which can be based on combining data from one or more sensor channels of the wrist-worn device 102 with an AR model. The user interface 107 further includes an effort indicator 118, which is represented as another circular user interface element, for example. The effort indicator 118 is displaying the current level of effort (e.g., in Figure 1A this case, stating that the user 101 is exerting effort at a baseline level of effort). The user interface 107 further includes an adaptability indicator element 120, which is also represented as a circular user interface element, for example. In some embodiments, the adaptability indicator element 120 provides the user 101 with information about the device and / or sensors used to monitor the body activity. For example, in Figure 1A this case, the adaptability indicator element 120 provides the user with a suggestion to wear an armband to improve accuracy. In some embodiments, the adaptability indicator element 120 provides the user 101 with adaptability indications on how to improve their performance and / or how to enhance aspects of the body activity that the user is performing.

[0062] Figure 1B1 shows a series of time points corresponding to reference numerals t0 to t3 while user 101 is performing a set of biceps curls. While user 101 is performing the set of biceps curls, sensor 190 in electronic communication with wrist wearable device 102 (e.g., located at wrist wearable device 102, wirelessly connected to wrist wearable device 102) detects signals (including neuromuscular signals) and / or other contextual data related to user 101 performing the physical activity. According to some embodiments, sensor 190 includes neuromuscular signal sensor 192 (e.g., Figure 6A ), and optionally includes an IMU sensor 194, an imaging sensor 196, a time-of-flight sensor 197, and a photoplethysmography (PPG) sensor 198.

[0063] exist Figure 1B In the example of , the wrist wearable device 102 includes multiple neuromuscular signal sensors 192 located on different parts of the band portion (e.g., arranged into different sensor channels). Each channel (e.g., channel 199-1, channel 199-4) can correspond to a discrete location and / or a combination of discrete locations on the wrist wearable device 102. In some embodiments, each channel is arranged to sense signals at different muscles and / or muscle groups. Figure 1B As shown, channels 199-1 and 199-4 output corresponding voltages during physical activity. In some embodiments, spectral power density is determined for each channel. In some embodiments, the voltage data and / or the generated power data are processed (e.g., via an artificial intelligence (AI) model or a machine learning (ML) model) to determine the minimum and maximum effort levels corresponding to various aspects of the repetition of the physical activity being performed by the user.

[0064] like Figure 1B As shown, user 101 performs a biceps curl while wearing wrist wearable device 102. At time t0, user 101's arm is straightened (e.g., corresponding to a minimum amount of effort in the muscles of user 101's arm). According to some embodiments, as indicated by graph 193, channel 199-1 monitors the muscles (e.g., triceps) engaged when user 101's arm is straightened. As indicated by notification 123, Figure 1B The wrist wearable device 102 is shown to detect a local minimum at t0. In some embodiments, the notification 123 is an audio notification, a tactile notification, and / or a visual notification. In some embodiments, the wrist wearable device 102 detects a local minimum but does not issue a notification. In some embodiments, whether the wrist wearable device 102 issues a notification depends on user settings or device settings.

[0065] At time t1, the arm of user 101 bends (flexes) and the wrist-worn device 102 detects a local maximum force. According to some embodiments, the wrist-worn device 102 updates a repetition count (e.g., corresponding to the repetition indicator 116) based on the detected local maximum. According to some embodiments, as indicated by the graph 193, channel 199-4 monitors the muscle (e.g., biceps) involved when the arm of user 101 bends. As indicated by the notification 124, Figure 1B shows that the wrist-worn device 102 detected a local maximum at t1. In some embodiments, the notification 124 is an audio notification, a tactile notification, and / or a visual notification. In some embodiments, the wrist-worn device 102 detects a local maximum but does not issue a notification.

[0066] At time t2, the arm of user 101 straightens again. According to some embodiments, as indicated by the graph 193, channel 199-1 outputs a voltage corresponding to the straightening of the arm of user 101. At time t3, the arm of user 101 bends and the wrist-worn device 102 detects a local maximum force. According to some embodiments, as indicated by the graph 193, channel 199-4 outputs a voltage corresponding to the bending of the arm of user 101. In some embodiments, compared to a previous repetition, based on the increased force exerted by the user in performing a subsequent repetition, the voltage detected by one or more neuromuscular signal sensor channels of the wrist-worn device is higher for the subsequent repetition. In some embodiments, an AR model can be used to determine a significant increase in the force exerted by user 101 from a specific repetition to a subsequent repetition based on the previous repetitions performed by user 101.

[0067] Figure 1CShows a user interface 129 (e.g., an activity settings user interface) displayed at a display 104 of a wrist-wearable device 102 (e.g., after a user 101 has performed a set of repetitions of bicep curls). The user interface 129 includes a user interface element 130 that displays the number of sets of bicep curls (e.g., one set) that the user 101 has performed. In some embodiments, the wrist-wearable device 102 determines that the user 101 has stopped performing a physical activity (e.g., bicep curls) for at least a threshold amount of time, and may also determine that the user has performed one set of the physical activity based on the user stopping the performance of the physical activity for the threshold amount of time. According to some embodiments, the user interface element 130 also includes an adaptive text prompt (e.g., "Great job on your first set! 10 repetitions with low effort.") based on data detected during the performance of the physical activity. In some embodiments, determining that the user 101 has stopped performing a physical activity is based on determining that the relaxation level of the user 101 is within a preset threshold of a baseline exertion level. In some embodiments, determining that the relaxation level of the user 101 is within a preset threshold of the baseline exertion level is based on power spectral density measurements calculated based on voltage measurements from one or more neuromuscular signal channels of the wrist-wearable device 102. The user interface element 130 also includes a button element 132 (stated: "Incorrect repetition count?") that is configured to allow the user 101 to calibrate one or more sensors of the wrist-wearable device 102 (e.g., to improve the detection of recognized repetitions of the physical activity) when selected by the user (e.g., via an air gesture). The user interface 129 also includes a repetition indicator 116 and an exertion indicator 118 that have updated values corresponding to updated repetition detection and exertion detection (e.g., detected via different subsets of neuromuscular signal sensor channels). In some embodiments, a notification is provided if the corresponding accuracy shown by the repetition indicator 116 and / or the exertion indicator 118 has changed by a threshold amount. The user interface 129 also includes an adaptive indicator 134 that provides a notification to the user 101 related to the performance of a previous set of physical activity (e.g., feedback related to the user's form during the activity).

[0068] Figure 1D Shows a user interface 133 that is identical to the user interface 129, except that the user interface 133 includes different text and buttons within the user interface element 130 and different text within the adaptive indicator 134. In Figure 1DIn it, the user interface element 130 includes a text prompt that asks, "Do you want to increase the difficulty of the exercise?" In some embodiments, the text prompt provided within the user interface element 130 is based on the detected exertion level of user 101 compared to the normal exertion level of the user for the physical activity being performed by user 101, and the text prompt includes the number of sets and / or repetitions of the physical activity that user 101 has performed. Figure 1D The user interface element 130 in it further includes selectable buttons 138, 140, and 142 for adaptively increasing the rate of the physical activity. In some embodiments, the adaptive adjustment recommended to user 101 is based on applying an AR model to data from one or more sensors and / or one or more sensor channels of the wrist wearable device 102. Figure 1D The adaptive indicator 134 in it states: "Pause between repetitions for a better posture." According to some embodiments, the user interface 133 provides an alternative to the user interface 129.

[0069] Figure 1E Shows a series of time points corresponding to reference numerals t4 to t7 when user 101 is performing another set of bicep curls. Similar components and elements are referenced Figure 1B to the components and elements in it. In some embodiments, the neuromuscular signal sensor 192 and the IMU sensor 194 of the wrist wearable device 102 are used in combination to determine that the user has started performing a physical activity. In some embodiments, for example, the imaging sensor 196 is used according to the sensor calibration model of the wrist wearable device 102, and the sensor calibration model indicates that it will improve the accuracy of detecting repetitions. In some embodiments, when user 101 starts performing a new set of physical activities, the wrist wearable device 102 calculates a new local minimum exertion value and / or local maximum exertion value because the exertion level may increase or decrease as the user performs the physical activity.

[0070] As Figure 1E shown, user 101 performs bicep curls while wearing the wrist wearable device 102. At time t4, the user's arm is straight (e.g., corresponding to the minimum amount of force exerted by the muscles of the user's arm). According to some embodiments, as indicated by the chart 195, channel 199-1 monitors the muscles (e.g., triceps) involved when the user's arm is straight. As indicated by the notification 148, Figure 1EIt is shown that the wrist-wearable device 102 detects that a set is being executed, and the notification 148 states, for example, "A new set is detected. Calibration is being performed for the new local minimum." In some embodiments, the notification 148 is an audio notification, a tactile notification, and / or a visual notification. In some embodiments, the wrist-wearable device 102 does not issue a notification based on detecting a new set and / or detecting a local minimum for the set.

[0071] At time t5, the user's arm bends (flexes), and the wrist-wearable device 102 provides a corresponding notification 150 (e.g., stating: "Good posture! Calibration is being performed for the new local maximum"). According to some embodiments, as indicated by the graph 195, channel 199-4 monitors the muscles (e.g., biceps) involved when the user's arm bends. At time t6, the user's arm straightens again. According to some embodiments, as indicated by the graph 195, channel 199-1 outputs a voltage corresponding to the straightening of the user's arm. At time t7, the user's arm bends, and the wrist-wearable device 102 provides a corresponding notification 154, which states, for example: "Good posture!" According to some embodiments, as indicated by the graph 193, channel 199-4 outputs a voltage corresponding to the bending of the user's arm. In some embodiments, when the user is performing a physical activity, the notifications 150 and 154 provide feedback to the user based on an analysis of the user's posture.

[0072] Figure 1F It is shown an active user interface 163 displayed at the display 104 of the wrist-wearable device 102, for example, after the user 101 has performed a set of repetitions of bicep curls. Figure 1F The user interface element 130 therein indicates that the user 101 has completed two sets of repetitions of bicep curls. The user interface element 130 also includes a text element that states: "13 repetitions of medium effort. Nice!" The user interface element 130 also includes a selectable button 160 that states: "Incorrect repetition count?" In some embodiments, the text and the button presented within the user interface element 130 are based on an assessment of the user's effort and / or posture when performing a physical activity. The active user interface 163 also includes an adaptability indicator element 162 that states: "View overview." In some embodiments, the adaptability indicator element 162 is displayed after the wrist-wearable device 102 determines that the user 101 has completed a set. In some embodiments, selecting the adaptability indicator element 162 causes an overview user interface to be presented (e.g., instead of the user interface 163 or overlaid on the user interface 163).

[0073] Figure 1GShows an active user interface 165 displayed at the display 104 of the wrist-wearable device 102, for example, after the user 101 has performed a set of repetitions of bicep curls. In the user interface 165, the user interface element 130 includes a text prompt that states: "Accelerate?" (e.g., asking the user 101 if they want to increase the difficulty of the physical activity they are performing). In some embodiments, the user interface 165 is presented to the user 101 based on determining that the user's exertion is below a preset threshold. In some embodiments, the user interface 163 is presented to the user 101 based on determining that the user's exertion is above a preset threshold. Based on the exertion data from one or more neuromuscular signal sensors, the active user interface 165 further includes an adaptability indicator element 164 that states "Closer to normal exertion".

[0074] Figure 1H Shows a series of time points corresponding to reference numerals t8 to t11 while the user 101 is performing another set of bicep curls. These components and elements refer Figure 1B and Figure 1D to similar components and elements in. Based on an assessment of the user's posture while performing bicep curls, the wrist-wearable device 102 provides notifications 170 and 172 (e.g., indicating good posture).

[0075] Figure 1I Shows Figure 1F different text elements and buttons displayed at the user interface element 130 compared to the user interface element 130 shown in. Figure 1I The user interface element 130 in indicates that the user 101 has completed three sets of repetitions of bicep curls. Figure 1I The user interface element 130 in further includes a selectable button element 132 that prompts the user 101 to indicate whether the sensors of the wrist-wearable device 102 have detected an incorrect repetition count. In some embodiments, the button element 132 is presented based on determining that the previous set had a different number of repetitions than expected (e.g., a different number than specified in the user-set exercise routine and / or a different number than the previous set).

[0076] Figure 1J Shows the user interface 167, which includes elements related to sharing information about physical activity. The user interface 167 includes information 178 about physical activity. In some embodiments, in response to the user's Figure 1IThe selection of user interface element 176 in [description] is used to display user interface 167. For example, user interface 167 includes user interface element 173, which shows multiple data items collected during the execution of a physical activity. In some embodiments, the wrist-wearable device 102 is configured to capture an image based on criteria associated with data from sensors detected during the execution of a physical activity (e.g., when the neuromuscular signal sensor detects that user 101 is in a relaxed state, notify user 101 to capture a self-portrait image). User interface 167 also includes user interface element 171, which is used to share information with other people. In Figure 1J In an example, the selection of user interface element 171 causes a message with information about the user's exercise to be sent to a remote person named "Molly".

[0077] Figure 1K Shows that the display 104 of the wrist-wearable device 102 displays an avatar user interface element 175. The avatar user interface element 175 includes an avatar accessory item 177 (e.g., a virtual hat with an identifier based on context data related to the execution of a physical activity). For example, the avatar accessory item 177 includes a bent arm according to the arm-related exercise that user 101 has performed. According to some embodiments, the accessory item 177 includes text information, such as stating: "San Francisco", to indicate the geographical location associated with the user's execution of the physical activity. In some embodiments, the avatar user interface element is provided in combination with an option to share the avatar user interface element 175, and the avatar user interface element includes and / or alternatively is an overview of the physical activity performed by user 101.

[0078] Figure 1L Shows the user holding a towel up to their face. Figure 1L The movement trajectory of the action in [description] is similar to that of a bicep curl. Figure 1L The IMU sensor 194 in [description] detects characteristic movements corresponding to the repetition of a bicep curl, but the neuromuscular signal sensor 192 does not recognize a force curve similar to the repetition of a bicep curl (e.g., the voltage output of the corresponding neuromuscular signal sensor is lower than the voltage threshold of a bicep curl). In some embodiments, the force curve is based on EMG data generated during multiple sets of physical activities (e.g., as shown in Figure 1B 、 Figure 1E and Figure 1H shown in). Based on the determination that user 101 is not performing a bicep curl, Figure 1LThe wrist-worn device 102 therein presents a notification 187 (stating: "No repetition detected."). In some embodiments, the notification 187 is an audio notification and / or a visual notification. In some embodiments, the wrist-worn device 102 does not present a notification based on determining that the user is not performing a repetition. In some embodiments, a notification is provided to prompt the user 101 to indicate whether a repetition of a physical activity has been performed, and / or whether the user 101 wants to calibrate one or more sensors of the wrist-worn device for the performance of a particular physical activity.

[0079] Figure 1M Shows a user performing repetitions of a bicep curl with a barbell 183. Figure 1M The IMU sensor 194 therein detects a movement similar to the movement detected when the user holds a towel up to their face as Figure 1L shown. However, in Figure 1M the neuromuscular signal sensor 192 detects the degree of exertion associated with lifting the barbell 183 (e.g., identifying local minimum exertion and / or local maximum exertion). Figure 1M Also shown is the wrist-worn device 102 presenting a notification 189 (stating: "Repetition detected."). In some embodiments, the notification 189 is an audio notification and / or a visual notification. In some embodiments, the wrist-worn device 102 does not present a notification based on determining that the user is performing a repetition. For example, the wrist-worn device 102 may wait for a threshold period to determine whether the user 101 pauses for a sufficient amount of time between repetitions so as to provide the notification 189 without interrupting the user's performance of the physical activity.

[0080] Figures 2A to 2E Shows another example user scenario of performing a physical activity while wearing a wearable device according to some embodiments. Figure 2A Shows the user 101 holding a barbell 211 on a fitness bench. The user 101 is wearing the wrist-worn device 102. The display 104 of the wrist-worn device 102 shows a user interface element 208 including context data. Figure 2A The context data therein includes a text prompt stating: "Your device was calibrated for this workout five weeks ago." Figure 2A The context data therein further includes a physical activity indicator stating: "Physical activity detected - bench press." In some embodiments, the physical activity is determined based on data from one or more sensors of the wrist-worn device (e.g., IMU sensor data). In some embodiments, the user 101 inputs information about the physical activity being performed (e.g., via a touch-sensitive portion of the display of the wrist-worn device 102). In some embodiments, data about the physical activity is received from fitness equipment or other electronic devices. Figure 2AThe user interface element 208 in also includes a selectable button 214 that states, "Recommend recalibration." In some embodiments, the wrist-wearable device 102 prompts the user 101 to recalibrate the sensors of the wrist-wearable device 102 based on a threshold calibration time since the sensors of the wrist-wearable device 102 were previously calibrated. In some embodiments, an indication to calibrate the sensors of the wrist-wearable device 102 is provided to the user 101 based on a detected change in the user's 101 body composition during a physical activity and / or a change in values detected by one or more sensors of the wrist-wearable device 102 during a physical activity. In some embodiments, one or more sensors of the wrist-wearable device 102 are powered based on the type of physical activity being performed. For example, the wrist-wearable device 102 determines which muscles are involved in a particular physical activity and activates sensors positioned to monitor the activity of those muscles. In some embodiments, the wrist-wearable device causes an AR model to perform an operation to determine which sensors and / or sensor channels to activate at the wrist-wearable device 102 based on the type of physical activity being performed. The display 104 also shows Figures 1A to 1M the repeat indicator 116 and the exertion indicator 118 shown Figure 2A . In an example of

[0081] Figure 2B , a series of time points corresponding to reference numerals t0 through t3 are shown while the user is performing a set of bench press lifts. Similar components and elements refer to those in a similar Figures 1A to 1M sequence. According to some embodiments, a different subset of sensor channels (e.g., sensor channels 199-5 and sensor channels 199-6) than the subset of sensor channels used in Figures 1A to 1M is used to detect the repetitions of the bench press being performed. In some embodiments, the exertion of the user is determined based on a particular subset of neuromuscular signal sensor channels for detecting a particular physical activity (e.g., bench press).

[0082] At time t0, the user's arm is bent (e.g., corresponding to the minimum muscle effort of the user's arm). According to some embodiments, as indicated by chart 213, channels 199-5 and 199-6 monitor the muscles involved when the user's arm is straight. At time t1, the user's arm is straight, and as indicated by notification 222, the wrist wearable device 102 detects high effort. According to some embodiments, as indicated by chart 213, channels 199-5 and 199-6 output corresponding voltages (e.g., local maxima) corresponding to the straightening of the user's arm. In some embodiments, the indication that the user is exerting high effort is based on a specific sensor that is being used to measure a specific activity (e.g., rather than the user's overall effort). For example, the same user 101 may have below-normal effort detected for one specific activity (e.g., bicep curls), while above-normal effort is detected for another body activity (e.g., bench press). In some embodiments, notification 222 is an audio notification and / or a visual notification. In some embodiments, the wrist wearable device 102 detects high effort but does not issue a notification. At time t2, the user's arm is bent again, and the wrist wearable device 102 generates notification 226 (e.g., an audio notification, a visual notification, and / or a tactile notification). In some embodiments, the wrist wearable device 102 generates notification 226 based on detecting a local minimum and / or based on determining that the user's arm is fully bent for a bench press exercise. In some embodiments, the wrist wearable device 102 generates notification 226 based on determining that the user has completed a bench press repetition. At time t3, the user's arm is straight, and the wrist wearable device 102 generates notification 228, which, for example, indicates that the user's current repetition has an incorrect posture (e.g., the user straightens one arm faster than the other arm without keeping the barbell 211 level). In some embodiments, for the execution of the same repetition of a body activity, an indication of the user's posture can be provided based on detecting above-normal effort by one corresponding sensor channel (e.g., channel 199-5), in combination with detecting below-normal effort by another corresponding sensor channel (e.g., channel 199-6). That is, the posture indication can be determined based on the user 101 overexerting one or more muscle groups associated with the first sensor channel and underexerting one or more muscle groups associated with the second sensor channel.

[0083] Figure 2CShows an active user interface 229 displayed at the display 104 of the wrist-wearable device 102, for example, after user 101 has performed a set of bench press repetitions. The user interface 229 includes a user interface element 230 that shows context data regarding the user's performance of the set of bench presses. The context data includes a set indicator that, for example, states: "Bench press - 1 set." The user interface element 230 also includes a text prompt (e.g., stating: "Posture quality has declined since the last activity, and the exertion is above normal."). In this way, the wrist-wearable device 102 can use data from different sensors to determine whether the user is generally performing well in all physical activities, and additionally or alternatively, to determine whether user 101 is performing a particular physical activity (e.g., bench press) effectively and / or whether the user is exerting above-normal effort for that particular physical activity (e.g., after exerting below-normal effort for a bicep curl, the user is exerting above-normal effort for a bench press). In Figure 2C the example of, the user interface element 230 also includes selectable buttons 232 that user 101 can activate to select different physical activities. The user interface 229 also includes Figures 1A to 1M the repetition indicator 116 and exertion indicator 118 shown in, but with values corresponding to the new physical activity (e.g., bench press) that user 101 is performing. The user interface 229 also includes a user interface element 234 that is configured to provide an adaptation notification to the user. In some embodiments, the adaptation notification includes feedback to the user regarding posture and / or exertion, such as stating: "Try to pause in the middle of a set."

[0084] Figure 2D Shows an active user interface 231 displayed at the display 104. In the user interface 231, the user interface element 230 is presenting text information related to the particular physical activity that user 101 is performing to user 101. In Figure 2D the example of, the user interface element 230 includes a text prompt that states: "Want to reduce the difficulty of the workout?" The user interface element 230 also includes multiple selectable buttons 238, 240, and 242 to allow user 101 to adjust the difficulty of the workout (e.g., adjust the weight on the barbell).

[0085] Figure 2EIllustrated is that when the barbell 211 is positioned at the user's chest (e.g., the user 101 is unable to lift the barbell 211), the user 101 performs an alert gesture 252. In some embodiments, in response to detecting the alert gesture 252, the wrist-worn device 102 sends a notification to a remote device to inform remote personnel that the user 101 is in a dangerous situation. In some embodiments, the notification is sent to a remote device previously selected by the user 101 (e.g., a remote device selected as an emergency contact). In some embodiments, the notification is sent to nearby remote devices (e.g., broadcasting the notification to nearby devices via Bluetooth and / or Wi-Fi protocols). In some embodiments, the notification is sent to emergency responders (e.g., police departments, gym security departments, fire departments, and / or medical staff). In some embodiments, the notification includes context information of the user 101, such as the user's biometric data, the user's location, images captured by the user's wrist-worn device or another device, and / or audio data captured by the user's wrist-worn device or another device. In some embodiments, based on the user 101 performing the alert gesture 252 and in combination with the context information that the user is located in a public gym facility, a notification may be provided to another user located in the same public gym facility (e.g., a user who has subscribed to an alert service associated with the alert gesture). According to some embodiments, Figure 2E the wrist-worn device 102 in [the figure] is presenting a notification 254 to the user 101, and the notification 254 indicates, for example, that no repetition has been detected and / or an alert gesture has been recognized. In some embodiments, based on the detection of the alert gesture, user interface elements related to a specific physical activity are suppressed.

[0086] Figures 3A to 3I Illustrated is another example user scenario of performing a physical activity while wearing the wrist-worn device 102. Figure 3A Illustrated is that the user 301 is preparing to perform a physical activity at an electronic fitness equipment 306 (e.g., a rowing machine). Specifically, the user 301 is performing a rowing machine exercise at the electronic fitness equipment 306. The user 301 is wearing the wrist-worn device 102 and a head-worn device 304 (e.g., AR glasses).

[0087] In Figure 3AIn the example, the head-worn device 304 is presenting a user interface 310 related to the execution of a physical activity to the user 301. The user interface 310 includes a text prompt 312 that states: "Configure the exercise at the electronic fitness equipment from your AR glasses using hand or eye movements!" The user interface 310 also includes a selectable exertion indicator 314 that displays and allows the user 301 to adjust the level of exertion corresponding to their execution of the physical activity. The user interface 310 also includes an activity duration indicator 316 that indicates the activity duration and allows the user 301 to adjust the desired duration of their execution of the physical activity. The user interface 310 includes a selectable settings button 318 that is configured to allow the user 301 to adjust one or more settings for executing the physical activity. These settings can include options for calibrating the sensors of the user's respective electronic devices. For example, the sensors at the head-worn device 304 can be detected during calibration and can cause an adjustment to the detection settings for the physical activity. The user interface 310 includes a selectable button 320 that allows the user to set and / or adjust the exercise, such as indicating the start of the physical activity. In some embodiments, any user interface element among the selectable button 320 and / or other user interface elements displayed at the user interface 310 is selected based solely on the tracked eye movements of the user, or based on the tracked eye movements of the user in combination with gestures detected by the sensors of the head-worn device 304, the wrist-worn device 102, and / or the electronic fitness equipment 306.

[0088] Figure 3B shows, for example, the user 301 at the electronic fitness equipment 306 after the user has selected the Figure 3A selectable button 320 shown in. Figure 3B The user interface 310 in includes a text prompt 322 that states: "Above baseline resting exertion detected. Do you want to do a warm-up?" For example, depending on the user's selection of the Figure 3A selectable button 320 shown in, Figure 3B the user interface 310 in is updated. In some embodiments, the text prompt 322 is based on data from one or more neuromuscular signal sensors. For example, depending on data from one or more neuromuscular signal sensors indicating that the level of exertion of the user is below a preset threshold, the text prompt can state "Low exertion detected. Do you want to increase the difficulty?"

[0089] The user interface 310 also includes a desired exertion indicator 324 that, for example, corresponds to the level of exertion set via the Figure 3A selectable button element 314 in. Figure 3BThe user interface 310 therein further includes a detected exertion indicator 323, which corresponds, for example, to the degree of exertion detected by one or more neuromuscular signal sensors of the wrist-wearable device 102. Figure 3B The user interface 310 therein further includes an activity duration indicator 316 and selectable setting buttons 318. The user interface 310 further includes a selectable button 326 for starting a physical activity (e.g., a rowing machine workout), and a selectable button 328 for selecting a fitness device (e.g., the electronic fitness device 306) (e.g., identifying the fitness device and / or being communicatively coupled with the fitness device).

[0090] As Figure 3C shown, the user 301 is performing a rowing motion at the electronic fitness device 306 while wearing the wrist-wearable device 102. At time t0, the user is in a fully extended position (e.g., corresponding to the maximum amount of force exerted by at least one muscle group detected by the wrist-wearable device 102). According to some embodiments, channel 199-2 monitors the muscles involved when the user is in the fully extended position of the rowing motion. As indicated by the graph 395, Figure 3C it shows that at time t0, the wrist-wearable device 102 detects a local maximum on channel 199-2. According to some embodiments, a notification 330 is provided to the user 301, for example, via the electronic fitness device 306, which is in electronic communication with the wrist-wearable device 102. The notification 330 states: "A very low pose score has been detected. Use eye pose to confirm that the counted repetitions are incorrect", which can be based on the signals detected from one or more of the sensors 190.

[0091] At time t1, the user 301 is in a contracted position as part of the rowing motion (e.g., corresponding to the minimum amount of force exerted by at least one muscle group detected by the wrist-wearable device 102). According to some embodiments, as indicated by the graph 395, channel 199-4 monitors the muscles involved when the user is in the contracted position of the rowing motion. A notification 332 is provided to the user 301 via the electronic fitness device 306. The notification 332 states: "Enable additional sensors at the wrist-wearable device to ensure accuracy", which can be based on the signals detected from one or more of the sensors 190, including one or more neuromuscular signals associated with sensor channel 199-4. In some embodiments, the wrist-wearable device 102 automatically activates (enables) additional sensors based on determining that the accuracy is below a preset threshold. In some embodiments, the wrist-wearable device activates additional sensors based on an input from the user.

[0092] Figure 3DShows user 301 after the user has completed the execution of a set of repetitions of rowing at the electronic fitness equipment 306. Figure 3D The user interface 310 in Figure 3D is displaying several user interface elements related to the execution of the physical activity. A display notification 332 (e.g., a text prompt) is shown, which states: "You're exerting too much force! Be careful, exercise should be healthy! Wait 30 seconds to relax yourself." In some embodiments, the text prompt 332 is shown based on determining that the user's exertion level is higher than a preset threshold. Selectable button elements 314, 334, and 336 are included in Figure 3D the user interface 310 of Figure 3D and provide the user 301 with the ability to adjust the exercise parameters and / or settings at the electronic fitness equipment 306. In

[0093] Figure 3E and Figure 3F also presents an activity duration indicator 342 that indicates the time the user has spent performing the physical activity at the electronic fitness equipment 306. Selectable buttons 338 and 340 are presented at Figure 3E the user interface 310 in Figure 3F which provide the user 301 with the ability to rest or start another set of repetitions of the physical activity, respectively. In some embodiments, in addition to or instead of the user interface elements based on the detection from the sensors communicating with the wrist-wearable device 102, the visual aspects and / or visual elements of the user interface 310 are based on the electronic fitness equipment 306 that communicates electronically with the wrist-wearable device 102.

[0094] Figure 3GShows a portion of another set of repetitions of a user performing rowing at the electronic fitness equipment 306. At time t2, a notification 350 is presented via the electronic fitness equipment, and the notification 350 states: "An unusually high muscle exertion is detected", and the notification 350 has a selectable button 351, and the button 351 states: "Reduce resistance?" In some embodiments, the notification is presented based on determining that the user's exertion level is higher than a preset threshold (e.g., the exertion level corresponding to channel 199-2). At time t3, a notification 352 is presented via the electronic fitness equipment (e.g., output visually and / or auditorily), and the notification 352 states: "An unusually high muscle exertion is detected. Ensure that the wearable strap is properly fastened." In some embodiments, the notification 352 is presented based on determining that the accuracy of the device does not meet one or more criteria and / or the user's exertion level is higher than a preset threshold.

[0095] Figure 3H Shows that the user continues to perform physical activity when the reminder element 362 is displayed on the user interface 310, and the reminder element 362 indicates to the user that the detected exertion is higher than a second (higher) exertion threshold. Figure 3H The user interface 310 in also displays a user interface element 364, and the user interface element 364 states: "You are dangerously overexerting yourself! For your safety, the equipment will be shut down." In some embodiments, based on determining that the exertion of the user 301 is higher than the second exertion level, the wrist-worn device 102 and / or another electronic device in electronic communication with the electronic fitness device causes a shutdown command to be provided at the electronic fitness equipment 306. In some embodiments, based on determining that the user's exertion is higher than the exertion threshold and / or based on other sensors (e.g., a heart rate sensor) indicating that the user's body is uncomfortable, the Figure 3H user interface 310 in is displayed.

[0096] Figure 3IIllustrated is user 301 performing an alert gesture 372 that causes a corresponding alert element to be presented within user interface 310. In some embodiments, performing alert gesture 372 causes a notification with context information to be sent to a remote device. In some embodiments, the context information includes an image captured by wrist-worn device 102 or head-worn device 304. In some embodiments, the context information includes audio data and / or health data captured by wrist-worn device 102 and / or head-worn device 304. As an example, a reminder element 362 is presented at user interface 310, and notification user interface element 368 states: "You are performing an alert gesture! Move your eyes to the right to cancel the alert!" In some embodiments, once user 301 performs alert gesture 372, an operation corresponding to the gesture is automatically performed without further instruction from user 301. In some embodiments, user 301 may cause the operation corresponding to alert gesture 372 to be cancelled, or otherwise adjusted, based on another gesture (e.g., tracked eye movement). User interface 310 includes a slidable user interface element 370 that allows user 301 to cancel the operation corresponding to alert gesture 372.

[0097] Figures 4A to 4E Illustrated is another example user scenario of performing a physical activity while wearing a wearable device. Figures 4A to 4E The illustrated sequence may be combined with Figures 1A to 3I any of the sequences illustrated, or in lieu of Figures 1A to 3I any of the sequences illustrated.

[0098] Figure 4AIllustrated is a user 401 wearing a wrist-wearable device 102 (e.g., a smartwatch), a head-wearable device 304, and an ankle-wearable device 406. The field of view 408 of the user 401 shows a user interface 410 presented to the user 401 (e.g., via the head-wearable device 304), the user interface 410 having settings related to performing a physical activity. Specifically, the user interface 410 includes a text element 412 having an application name "Fitness Buddy App" corresponding to an application to be used when the user 401 performs a physical activity. In some embodiments, the user interface 410 is displayed based on actions performed by the user while wearing the wrist-wearable device 102, the head-wearable device 304, and / or the ankle-wearable device 406. For example, the application may be displayed based on detecting at the ankle-wearable device 406 a force indicating that the user 401 is performing a leg stretch and / or jogging. For example, the user interface 410 may be displayed based on recognizing that the user 401 is performing a physical activity (e.g., running). In some embodiments, the user interface 410 is displayed based on a user input (e.g., for starting the wearable device and / or the fitness application).

[0099] The user interface 410 further includes a drop-down user interface element 414 that allows the user 401 to select (e.g., via a gesture detected by the wrist-wearable device 102 and / or the head-wearable device 304) a physical activity to perform. The user interface 410 further includes a selectable force setting element 416 that indicates the desired force of the user 401 for the physical activity. The user interface 410 further includes a selectable activity time user interface element 418 that the user 401 can use to adjust the duration of the physical activity, and a selectable force interface element 421 that the user 401 can use to adjust the desired force of the physical activity. The user interface 410 further includes a selectable button input 420 that the user 401 can select to initialize the physical activity. In some embodiments, based on the user 401 selecting the selectable button input 420, the power of one or more sensors of the wrist-wearable device 102, the head-wearable device 304, and / or the ankle-wearable device 406 is increased and / or decreased. The user interface 410 further includes a selectable settings user interface element 422 that the user 401 can select to adjust global settings at one or more of the wearable electronic devices (e.g., the wearable electronic devices currently worn by the user 401). In some embodiments, the user can calibrate one or more sensors based on an input provided after selecting the selectable settings user interface element 422.

[0100] Figure 4B Illustrated is, for example, when selecting Figure 4AAfter the selectable button input 420 shown, user 401 engages in a physical activity (running). User interface element 424 is presented, which indicates various aspects related to the user performing the physical activity (e.g., pace, mileage, repetitions), while another user interface element 426 indicates the level of exertion of the user while performing the physical activity (e.g., "Pick up the pace! Your level of exertion is lower than usual!"), which indicates a comparison between the user's current level of exertion and the typical level of exertion of user 401 while the user is performing the physical activity (e.g., the exertion detected by one or more neuromuscular signal sensors).

[0101] Figure 4C It is shown that when the user interface element is displayed in the field of view 408 of user 401, user 401 continues to engage in the physical activity. For example, Figure 4C the field of view 408 in includes user interface element 424, which indicates aspects related to performing the physical activity, and these aspects can be updated based on data from the ankle wearable device 406 in combination with data from the wrist wearable device 102.

[0102] Figure 4D It is shown that when the user interface element is displayed in the field of view 408 of user 401, user 401 continues to engage in the physical activity. When the user interface element including user interface element 424 and other user interface elements 426 is displayed within the field of view 408 of user 401, a tornado disaster 430 appears at a visible distance from user 401. According to some embodiments, the field of view 408 includes a notification 436, which indicates how the user can perform an alert gesture. In some embodiments, in response to recognizing the tornado disaster 430, the notification 436 is displayed. Figure 4D User 401 in is performing an alert gesture 434, which can be detected by the neuromuscular signal sensor (e.g., EMG sensor) of the wrist wearable device 102. In some embodiments, in response to detecting the alert gesture 434, the wrist wearable device 102 sends a notification with the user's context information (e.g., location, image data, audio data, and / or health data) to a remote device.

[0103] Figure 4E It is shown in response to user 401 performing Figure 4DUpdated user interface elements that are within the field of view 408 of the user 401 in response to the alert gesture 434. The user's field of view includes the user interface element 442, which states: "Your image and location have been sent to the authorities. Follow the navigation path presented by the head-mounted device to find the fastest route to a safe location!" Corresponding user interface elements 438a and 438b are also presented, which indicate the respective locations of places where the user 401 may go for safety (e.g., locations determined to be safe from dangerous weather). Based on the user 401 performing the alert gesture 434, the navigation user interface element 440 is displayed to appear on the ground in front of the user 401 (e.g., to suggest that the user turn around). In some embodiments, one or more navigation user interface elements are displayed while the user 401 is performing a physical activity, and after the user 401 has performed the alert gesture 434, these navigation user interface elements are caused to change (e.g., change color, change animation style) (e.g., to guide the user away from a dangerous situation). In some embodiments, the navigation user interface element 440 has a physical appearance that is adjusted based on the user 401 exerting a force level higher than a force threshold (e.g., a force level that is considered unhealthy for the user).

[0104] In some embodiments, Figures 1A to 1M 、 Figures 2A to 2E 、 Figures 3A to 3I and Figures 4A to 4E the notifications and alerts shown have one or more of an audio component, a visual component, and a tactile component. For example, the wrist-worn device displays notifications, generates audio alerts (e.g., text-to-speech output), and / or vibrates.

[0105] After describing example sequences and methods of operation using these example sequences, attention will now be turned to a system-level description of the hardware and software on which (or with which) these methods may be implemented.

[0106] Example systems

[0107] Figure 5A 、 Figure 5B 、 Figure 5C-1 、 Figure 5C-2 、 Figure 5D-1 and Figure 5D-2 illustrate example AR systems according to some embodiments. Figure 5A Illustrates the AR system 5000a, and a first example user interaction using a wrist-worn device 6000, a head-mounted device (e.g., AR system 7000), and / or a handheld intermediary processing device (HIPD) 8000. Figure 5BShows the AR system 5000b, as well as a second example of user interaction using the wrist-wearable device 6000, the AR system 7000, and / or the HIPD 8000. Figure 5C-1 And Figure 5C-2 Shows the AR system 5000c, as well as a third example of user interaction using the wrist-wearable device 6000, the head-wearable device (e.g., a virtual-reality (VR) headset 7010), and / or the HIPD 8000. Figure 5D-1 And Figure 5D-2 Shows a fourth AR system 5000d, as well as a fourth example of user interaction using the wrist-wearable device 6000, the VR headset 7010, and / or the device 9000 (e.g., a wearable haptic glove). The example AR systems described above (detailed below) may perform the various functions and / or operations referred to above with reference to Figures 1A to 4E described.

[0108] Below with reference to Figure 6A And Figure 6B describe the wrist-wearable device 6000 and its components; below with reference to Figure 7A to FIGS. 7D describe the head-wearable device and its components; and below with reference to Figure 8A And Figure 8B describe the HIPD 8000 and its components. Below with reference to Figures 9A to 9C describe the wearable glove and its components. As Figure 5A shown, the wrist-wearable device 6000, the head-wearable device, and / or the HIPD 8000 may be communicatively coupled via a network 5025 (e.g., a cellular network, a near-field network, Wi-Fi, a personal area network, or a wireless local area network (LAN)). Additionally, the wrist-wearable device 6000, the head-wearable device, and / or the HIPD 8000 may also be communicatively coupled via the network 5025 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless LAN) to one or more servers 5030, computers 5040 (e.g., laptop computers, computers), mobile devices 5050 (e.g., smartphones, tablets), and / or other electronic devices. Similarly, the device 9000 may also be communicatively coupled via the network 5025 to the wrist-wearable device 6000, the head-wearable device, the HIPD 8000, one or more servers 5030, computers 5040, mobile devices 5050, and / or other electronic devices.

[0109] Go to Figure 5A, shows user 5002 wearing a wrist-wearable device 6000 and an AR system 7000, and placing the HIPD 8000 on their table. The wrist-wearable device 6000, AR system 7000, and HIPD 8000 facilitate the user's interaction with the AR environment. In particular, as shown in AR system 5000a, the wrist-wearable device 6000, AR system 7000, and / or HIPD 8000 cause one or more avatars 5004, digital representations 5006 of one or more contacts, and one or more virtual objects 5008 to be presented. As described below, user 5002 can interact with one or more avatars 5004, digital representations 5006 of one or more contacts, and one or more virtual objects 5008 via the wrist-wearable device 6000, AR system 7000, and / or HIPD 8000.

[0110] User 5002 can use any one of the wrist-wearable device 6000, AR system 7000, and / or HIPD 8000 to provide user input. For example, user 5002 can make one or more gestures that are detected by the wrist-wearable device 6000 (e.g., using one or more EMG sensors and / or IMUs described with reference to Figure 6A and Figure 6B and / or the AR system 7000 (e.g., using one or more image sensors or cameras described with reference to Figure 7A and FIG. 7B) to provide user input. Alternatively or additionally, user 5002 can provide user input via one or more touch surfaces of the wrist-wearable device 6000, AR system 7000, and / or HIPD 8000; and / or voice commands captured by a microphone of the wrist-wearable device 6000, AR system 7000, and / or HIPD 8000. In some embodiments, the wrist-wearable device 6000, AR system 7000, and / or HIPD 8000 includes a digital assistant to assist the user in providing user input (e.g., completing an operation sequence, suggesting different operations or commands, providing reminders, or confirming commands). In some embodiments, user 5002 provides user input via one or more facial gestures and / or facial expressions. For example, a camera of the wrist-wearable device 6000, AR system 7000, and / or HIPD 8000 can track the eyes of user 5002 to navigate the user interface.

[0111] The wrist-wearable device 6000, the AR system 7000, and / or the HIPD 8000 can be operated individually or in combination to allow the user 5002 to interact with the AR environment. In some embodiments, the HIPD 8000 is configured to operate as a central hub or control center for the following devices: the wrist-wearable device 6000; the AR system 7000; and / or another communicatively coupled device. For example, the user 5002 can provide input at any one of the wrist-wearable device 6000, the AR system 7000, and / or the HIPD 8000 to interact with the AR environment, and the HIPD 8000 can identify one or more back-end tasks and front-end tasks to effect the requested interaction, and can distribute instructions to effect the execution of the one or more back-end tasks and front-end tasks at the wrist-wearable device 6000, the AR system 7000, and / or the HIPD 8000. In some embodiments, the back-end tasks are background processing tasks that are not perceptible to the user (e.g., rendering content, decompressing or compressing), while the front-end tasks are user-perceptible user-facing tasks (e.g., presenting information to the user or providing feedback to the user). As described below with reference to Figure 8A and Figure 8B the HIPD 8000 can perform back-end tasks and provide operation data corresponding to the performed back-end tasks to the wrist-wearable device 6000 and / or the AR system 7000, such that the wrist-wearable device 6000 and / or the AR system 7000 can perform front-end tasks. In this way, the HIPD 8000 (which may have more computing resources and a greater thermal headroom than the wrist-wearable device 6000 and / or the AR system 7000) performs computationally intensive tasks and reduces the computer resource utilization and / or power usage of the wrist-wearable device 6000 and / or the AR system 7000.

[0112] In the example shown in the AR system 5000a, the HIPD 8000 identifies one or more back-end tasks and front-end tasks associated with the following user request: the user request to initiate an AR video call with one or more other users (represented by the avatar 5004 and the digital representation 5006 of the contact); and the HIPD 8000 distributes instructions to effect the execution of the one or more back-end tasks and front-end tasks. In particular, the HIPD 8000 performs back-end tasks for processing and / or rendering image data (and other data) associated with the AR video call, and provides operation data associated with the performed back-end tasks to the AR system 7000, such that the AR system 7000 performs front-end tasks for presenting the AR video call (e.g., presenting the avatar 5004 and the digital representation 5006 of the contact).

[0113] In some embodiments, the HIPD 8000 is used as a focus or anchor for information presentation. This allows the user 5002 to generally know where the information is presented. For example, as shown in the AR system 5000a, the avatar 5004 and the digital representation 5006 of the contact are presented on top of the HIPD 8000. In particular, the HIPD 8000 and the AR system 7000 operate in conjunction to determine the location for presenting the avatar 5004 and the digital representation 5006 of the contact. In some embodiments, information can be presented within a predetermined distance from the HIPD 8000 (e.g., within 5 meters). For example, as shown in the AR system 5000a, the virtual object 5008 is presented on a table at a certain distance from the HIPD 8000. Similar to the above example, the HIPD 8000 and the AR system 7000 can operate in conjunction to determine the location for presenting the virtual object 5008. Alternatively, in some embodiments, information presentation is not constrained by the HIPD 8000. More specifically, the avatar 5004, the digital representation 5006 of the contact, and the virtual object 5008 do not have to be presented within a predetermined distance from the HIPD 8000.

[0114] Coordinate user inputs provided at the wrist-worn device 6000, the AR system 7000, and / or the HIPD 8000 so that the user can use any device to initiate, continue, and / or complete an operation. For example, the user 5002 can provide a user input to the AR system 7000 to cause the AR system 7000 to present the virtual object 5008, and while the AR system 7000 is presenting the virtual object 5008, the user 5002 can provide one or more gestures via the wrist-worn device 6000 to interact with and / or manipulate the virtual object 5008.

[0115] Figure 5B The user 5002 is shown wearing the wrist-worn device 6000 and the AR system 7000 and holding the HIPD 8000. In the AR system 5000b, the wrist-worn device 6000, the AR system 7000, and / or the HIPD 8000 are used to receive one or more messages and / or provide one or more messages to the contacts of the user 5002. In particular, the wrist-worn device 6000, the AR system 7000, and / or the HIPD 8000 detect and coordinate one or more user inputs to initiate a messaging application and prepare a response to messages received via the messaging application.

[0116] In some embodiments, user 5002 launches an application on the following items via user input: wrist-wearable device 6000; AR system 7000; and / or HIPD 8000, and the user input causes the application to be launched on at least one device. For example, in AR system 5000b, user 5002 makes a gesture associated with a command for launching a messaging application (represented by messaging user interface 5012); wrist-wearable device 6000 detects the gesture; and based on determining that user 5002 is wearing AR system 7000, AR system 7000 presents messaging user interface 5012 of the messaging application. AR system 7000 can present messaging user interface 5012 to user 5002 via its display (e.g., as shown in user 5002's field of view 5010). In some embodiments, the application is launched and can run on the device that detects the user input for launching the application (e.g., wrist-wearable device 6000, AR system 7000, and / or HIPD 8000), and the device provides operation data to another device to cause the messaging application to be presented. For example, wrist-wearable device 6000 can detect user input to launch the messaging application, launch and run the messaging application, and provide operation data to AR system 7000 and / or HIPD 8000 to cause the messaging application to be presented. Alternatively, the application can be launched and run on a device other than the device that detects the user input. For example, wrist-wearable device 6000 can detect a gesture associated with launching the messaging application, and can cause HIPD 8000 to run the messaging application and coordinate the presentation of the messaging application.

[0117] In addition, user 5002 can provide user input at wrist-wearable device 6000, AR system 7000, and / or HIPD 8000 to continue and / or complete an operation launched at another device. For example, after launching the messaging application via wrist-wearable device 6000 and when AR system 7000 presents messaging user interface 5012, user 5002 can provide input at HIPD 8000 to prepare a response (e.g., as shown by a swiping gesture made on HIPD 8000). The gesture made by user 5002 on HIPD 8000 can be provided and / or displayed on another device. For example, the swiping gesture made by user 5002 on HIPD 8000 is displayed on the virtual keyboard of messaging user interface 5012 displayed by AR system 7000.

[0118] In some embodiments, the wrist-worn device 6000, the AR system 7000, the HIPD 8000, and / or other communicatively coupled devices present one or more notifications to the user 5002. The notification can be an indication of a new message, an incoming call, an app update, or a status update. The user 5002 can select the notification via the wrist-worn device 6000, the AR system 7000, the HIPD 8000, and cause an app or operation associated with the notification to be presented on at least one device. For example, the user 5002 can receive a notification of a message received at the wrist-worn device 6000, the AR system 7000, the HIPD 8000, and / or other communicatively coupled devices, and the user 5002 can provide user input at the wrist-worn device 6000, the AR system 7000, and / or the HIPD 8000 to review the notification, and the device that detects the user input can cause an app associated with the notification to be launched and / or presented at the wrist-worn device 6000, the AR system 7000, and / or the HIPD 8000.

[0119] While the above examples describe coordinated input for interacting with a messaging app, those skilled in the art will recognize upon reading this description that user input can be coordinated to interact with any number of apps, including but not limited to gaming apps, social media apps, camera apps, web-based apps, and financial apps. For example, the AR system 7000 can present gaming app data to the user 5002, and the HIPD 8000 can use a controller to provide input to the game. Similarly, the user 5002 can use the wrist-worn device 6000 to activate the camera of the AR system 7000, and the user can use the wrist-worn device 6000, the AR system 7000, and / or the HIPD 8000 to manipulate image capture (e.g., zoom in or out, apply a filter) and capture image data.

[0120] Having generally discussed example AR systems, devices for interacting with such AR systems, and other computing systems, example devices used in such AR systems and other computing systems will now be discussed in more detail below. For ease of reference, some definitions of the following devices and components are provided here: These devices and components can be included in some or all of the example devices discussed below. Those skilled in the art will recognize that certain types of components described below may be more suitable for a particular set of devices and less suitable for another set of devices. However, subsequent references to the components defined here should be considered to be covered by the provided definitions.

[0121] In some of the embodiments discussed below, example devices and systems including electronic devices and systems will be discussed. Such example devices and systems are not intended to be limiting, and those skilled in the art will understand that alternative devices and systems to the example devices and systems described herein can be used to perform the operations described herein and to construct the systems and devices described herein.

[0122] As described herein, an electronic device is a device that uses electrical energy to perform one or more functions. The electronic device can be any physical object that includes electronic components (e.g., transistors, resistors, capacitors, diodes, and integrated circuits). Examples of electronic devices include smart phones, laptop computers, digital cameras, televisions, game consoles, and music players, as well as the example electronic devices discussed herein. As described herein, an intermediate electronic device is a device that is located between two other electronic devices and / or between subsets of components of one or more electronic devices and that facilitates communication and / or data processing and / or data transfer between the corresponding electronic devices and / or electronic components.

[0123] As described herein, a processor (e.g., a central processing unit (CPU)) is an electronic component that is responsible for executing instructions and controlling the operation of an electronic device (e.g., a computer). There are various types of processors that can be used interchangeably or specifically required by the embodiments described herein. For example, a processor can be: (i) a general-purpose processor that is designed to perform a wide range of tasks, such as running software applications, managing an operating system, and performing arithmetic and logical operations; (ii) a microcontroller that is designed for specific tasks, such as controlling electronic devices, sensors, and motors; (iii) a graphics-processing unit (GPU) that is designed to accelerate the creation and rendering of images, videos, and animations (e.g., virtual reality animations such as 3D modeling); (iv) a field-programmable gate array (FPGA) that can be programmed and reconfigured after manufacturing and / or can be customized to perform specific tasks, such as signal processing, encryption, and machine learning; (v) a digital signal processor (DSP) that is designed to perform mathematical operations on signals (e.g., audio, video, and radio waves). Those skilled in the art will understand that one or more processors of one or more electronic devices can be used in the various embodiments described herein.

[0124] As described herein, a memory refers to an electronic component in a computer or electronic device that stores data and instructions for access and operation by a processor. Examples of memory can include: (i) random access memory, which is configured to temporarily store data and instructions; (ii) read-only memory, which is configured to permanently store data and instructions (e.g., one or more portions of system firmware and / or a boot loader); (iii) flash memory (e.g., a USB drive, a memory card, and / or a solid-state drive), which can be configured to store data in an electronic device; and (iv) cache memory, which is configured to temporarily store frequently accessed data and instructions. As described herein, a memory can include structured data (e.g., an SQL database, a MongoDB database, GraphQL data, and / or JSON data). Other examples of memory can include: (i) profile data, which includes user account data, user settings, and / or other user data stored by a user; (ii) sensor data detected and / or otherwise obtained by one or more sensors; (iii) media content data, which includes stored image data, audio data, documents, and the like; (iv) application data, which can include data collected and / or otherwise obtained and stored during the use of an application; and / or any other type of data described herein.

[0125] As described herein, a controller is an electronic component that manages and coordinates the operation of other components within an electronic device (e.g., controls inputs, processes data, and / or generates outputs). Examples of controllers can include: (i) a microcontroller, which includes a small, low-power controller commonly used in embedded systems and Internet of Things (IoT) devices; (ii) a programmable logic controller, which can be configured for use in industrial automation systems to control and monitor manufacturing processes; (iii) a system-on-a-chip (SoC) controller, which integrates multiple components such as a processor, memory, input / output (I / O) interfaces, and other peripherals into a single chip; and / or a DSP.

[0126] As described herein, the power system of an electronic device is configured to convert input power into a form that can be used to operate the device. The power system may include various components, including: (i) a power source, which may be an AC adapter power source or a DC adapter power source; (ii) a charger input, which may be configured to use a wired connection and / or a wireless connection (which may be part of a peripheral interface, such as a USB, micro-USB interface, near-field magnetic coupling, magnetic induction, and magnetic resonance charging, and / or radio frequency (RF) charging); (iii) a power management integrated circuit, which is configured to distribute power to the various components of the device and ensure that the device operates within safe limits (e.g., regulating voltage, controlling current, and / or managing heat dissipation); and / or (iv) a battery, which is configured to store power to provide available power to the components of one or more electronic devices.

[0127] As described herein, a peripheral interface is an electronic component (e.g., an electronic component of an electronic device) that allows an electronic device to communicate with other devices or peripheral devices and may provide a means for inputting and outputting data and signals. Examples of peripheral interfaces may include: (i) a universal serial bus (USB) interface and / or a micro-USB interface, which are configured to connect a device to an electronic device; (ii) a Bluetooth interface, which is configured to allow multiple devices to communicate with each other, and the Bluetooth interface includes Bluetooth low energy (BLE); (iii) a near-field communication (NFC) interface, which is configured to be a short-range wireless interface for operations such as access control; (iv) a POGO pin, which may be a small spring-loaded pin configured to provide a charging interface; (v) a wireless charging interface; (vi) a global positioning system (GPS) interface; (vii) a Wi-Fi interface, which is used to provide a connection between a device and a wireless network; and (viii) a sensor interface.

[0128] As described herein, a sensor is an electronic component configured to detect physical and environmental changes and generate an electrical signal, which can be physically coupled to and / or otherwise electronically communicate with an electronic device (such as a wearable device). Examples of sensors can include: (i) an imaging sensor for collecting imaging data (e.g., including one or more cameras disposed on a corresponding electronic device); (ii) a bioelectrical potential signal sensor; (iii) an inertial measurement unit (e.g., IMU) for detecting changes in angular velocity, force, magnetic field, and / or acceleration; (iv) a heart rate sensor for measuring a user's heart rate; (v) an SpO2 sensor for measuring a user's blood oxygen saturation (SpO2) and / or other biometric data; (vi) a capacitive sensor for detecting changes in electrical potential at a location on a user's body (e.g., a sensor-skin interface); and a light sensor (e.g., a time-of-flight sensor, an infrared light sensor, a visible light sensor). As described herein, a bioelectrical potential signal sensing component is a device for measuring electrical activity within the body (e.g., a bioelectrical potential signal sensor). Some types of bioelectrical potential signal sensors include: (i) an electroencephalogram sensor configured to measure electrical activity in the brain to diagnose neurological disorders; (ii) an electrocardiography (ECG or EKG) sensor configured to measure electrical activity of the heart to diagnose heart problems; (iii) an EMG sensor configured to measure electrical activity of muscles and diagnose neuromuscular diseases; (iv) an electrooculogram sensor configured to measure electrical activity of eye muscles to detect eye movement and diagnose eye disorders.

[0129] As described herein, an application (e.g., software) stored in the memory of an electronic device includes instructions stored in the memory. Examples of such applications include: (i) games; (ii) word processors; messaging applications; media streaming applications; financial applications; calendars; clocks; a communication interface module for implementing a wired connection and / or a wireless connection between different corresponding electronic devices (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.11a, WirelessHART, or MiWi, a custom or standard wired protocol (e.g., Ethernet or HomePlug), and / or any other suitable communication protocol).

[0130] As described herein, a communication interface is a mechanism that enables different systems or devices to exchange information and data with each other, including hardware, software, or a combination of both. For example, a communication interface may refer to a physical connector and / or port on a device that enables communication with other devices (e.g., USB, Ethernet, High-Definition Multimedia Interface (HDMI), Bluetooth). In some embodiments, a communication interface may be a software layer that enables different software programs to communicate with each other (e.g., application programming interfaces and protocols such as Hypertext Transfer Protocol (HTTP) and Transmission Control Protocol / Internet Protocol (TCP / IP)).

[0131] As described herein, a graphics module is a component or software module designed to process graphics operations and / or graphical processes, and the graphics module may include a hardware module and / or a software module.

[0132] As described herein, a non-transitory computer-readable storage medium is a physical device or storage medium that can be used to store electronic data in a non-transitory form (e.g., such that the data is permanently stored until it is intentionally deleted or modified).

[0133] Example Wrist Wearable Device

[0134] Figure 6A and Figure 6B illustrates a wrist wearable device 6000 according to some embodiments. Figure 6A illustrates various components of the wrist wearable device 6000, which may be used alone or in combination, including combinations that include other electronic devices and / or electronic components.

[0135] As discussed below, Figure 6A illustrates a wearable band 6010 and a watch body 6020 (or capsule) coupled together to form the wrist wearable device 6000. The wrist wearable device 6000 may perform various functions and / or operations associated with navigating in a user interface and selectively opening applications, as well as the functions and / or operations described above with reference to Figures 1A to 4E described.

[0136] As will be described in more detail below, operations performed by the wrist-wearable device 6000 may include: (i) presenting content to a user (e.g., displaying visual content via the display 6005); (ii) detecting (e.g., sensing) user input (e.g., sensing a touch on the peripheral button 6023 and / or a touch at the touchscreen of the display 6005, a gesture detected by a sensor (e.g., a biopotential sensor)); (iii) sensing biometric data (e.g., neuromuscular signals, heart rate, temperature, and / or sleep) via one or more sensors 6013; messaging (e.g., text, voice, and / or video); image capture via one or more imaging devices or cameras 6025; wireless communication (e.g., cellular, near field, Wi-Fi, and / or personal area network); location determination; financial transactions; providing haptic feedback; alerts; notifications; biometric authentication; health monitoring; and sleep monitoring, etc.

[0137] The above example functions may be performed independently in the watch body 6020, independently in the wearable band 6010, and / or by electronic communication between the watch body 6020 and the wearable band 6010. In some embodiments, functions may be performed on the wrist-wearable device 6000 when presenting an AR environment (e.g., via one of the AR systems 5000a to 5000d). As those skilled in the art will recognize upon reading the description provided herein, the novel wearable devices described herein may be used with other types of AR environments.

[0138] The wearable band 6010 may be configured to be worn by a user such that the inner surface of the wearable band 6010 contacts the user's skin. When the user wears it, the plurality of sensors 6013 contact the user's skin. These sensors 6013 may sense biometric data, such as the user's heart rate, saturation oxygen level, temperature, sweat level, neuromuscular signal sensor, or a combination thereof. These sensors 6013 may also sense data about the user's environment, which includes the user's movement, altitude, location, orientation, gait, acceleration, position, or a combination thereof. In some embodiments, these sensors 6013 are configured to track the position and / or movement of the wearable band 6010. One or more sensors 6013 may include any of the sensors among the plurality of sensors defined above and / or discussed below with reference to Figure 6B any of the sensors discussed.

[0139] One or more sensors 6013 may be distributed on the inner surface and / or the outer surface of the wearable band 6010. In some embodiments, one or more sensors 6013 are evenly spaced along the wearable band 6010. Alternatively, in some embodiments, one or more sensors 6013 are located at different points along the wearable band 6010. As Figure 6AAs shown, one or more sensors 6013 can be the same or different. For example, in some embodiments, one or more sensors 6013 can be shaped like a pill (e.g., sensor 6013a), oval, round, square, elliptical (e.g., sensor 6013c), and / or any other shape that maintains contact with the user's skin (e.g., such that neuromuscular signals and / or other biometric data can be accurately measured at the user's skin). In some embodiments, one or more sensors 6013 are aligned to form sensor pairs (e.g., for sensing neuromuscular signals based on differential sensing within each respective sensor). For example, sensor 6013b is aligned with an adjacent sensor to form sensor pair 6014a, and sensor 6013d is aligned with an adjacent sensor to form sensor pair 6014b. In some embodiments, the wearable band 6010 does not have sensor pairs. Alternatively, in some embodiments, the wearable band 6010 has a predetermined number of sensor pairs (e.g., one sensor pair, three sensor pairs, four sensor pairs, six sensor pairs, or sixteen sensor pairs).

[0140] The wearable band 6010 can include any suitable number of sensors 6013. In some embodiments, the number and arrangement of sensors 6013 depend on the particular application for which the wearable band 6010 is used. For example, a wearable band 6010 configured as an armband, wristband, or chest band can include multiple sensors 6013 that have different numbers of sensors 6013 and different arrangements for various use cases (e.g., medical use cases compared to gaming or general daily use cases).

[0141] According to some embodiments, the wearable band 6010 further includes an electrical ground electrode and a shielding electrode. Like the sensors 6013, the electrical ground electrode and the shielding electrode can be distributed on the inner surface of the wearable band 6010 such that they contact a portion of the user's skin. For example, the electrical ground electrode and the shielding electrode can be located on the inner surface of the coupling mechanism 6016 or the inner surface of the wearable structure 6011. The electrical ground electrode and the shielding electrode can be formed as sensors 6013 and / or use the same components as the sensors 6013. In some embodiments, the wearable band 6010 includes more than one electrical ground electrode and more than one shielding electrode.

[0142] The sensor 6013 can be formed as part of a wearable structure 6011 of the wearable band 6010. In some embodiments, the sensor 6013 is flush or substantially flush with the wearable structure 6011 such that the sensors do not extend beyond the surface of the wearable structure 6011. While being flush with the wearable structure 6011, the sensor 6013 is still configured to contact the user's skin (e.g., via a skin contact surface). Alternatively, in some embodiments, the sensor 6013 extends beyond the wearable structure 6011 by a predetermined distance (e.g., 0.1 millimeter (mm) to 2 mm) to contact and press into the user's skin. In some embodiments, a plurality of sensors 6013 are coupled to an actuator (not shown) that is configured to adjust the extension height of these sensors 6013 (e.g., the distance from the surface of the wearable structure 6011) such that the sensors 6013 contact and press into the user's skin. In some embodiments, the actuator adjusts the extension height between 0.01 mm and 1.2 mm. This allows the user to customize the position of the sensor 6013 to improve the overall comfort of the wearable band 6010 when worn while still allowing the sensor 6013 to contact the user's skin. In some embodiments, the sensor 6013 is not distinguishable from the wearable structure 6011 when worn by the user.

[0143] The wearable structure 6011 can be formed of an elastic material, an elastomer, etc., that is configured to be stretched and adapted for user wear. In some embodiments, the wearable structure 6011 is a textile or a woven fabric. As described above, the sensor 6013 can be formed as part of the wearable structure 6011. For example, the sensor 6013 can be molded into the wearable structure 6011 or integrated into a woven fabric (e.g., the sensor 6013 can be sewn into the fabric and mimic the flexibility of the fabric (e.g., the sensor 6013 can be formed of a series of woven fabric threads)).

[0144] The wearable structure 6011 can include flexible electronic connectors that interconnect sensors 6013, electronic circuits, and / or other electronic components (described below with reference to Figure 6B to be included in the wearable band 6010. In some embodiments, the flexible electronic connectors are configured to interconnect the sensors 6013, electronic circuits, and / or other electronic components of the wearable band 6010 with corresponding sensors and / or other electronic components of another electronic device (e.g., the watch body 6020). The flexible electronic connectors are configured to move with the wearable structure 6011 such that adjustments (e.g., resizing, pulling, and / or folding) of the wearable structure 6011 by the user do not stress or strain the electrical coupling of the components of the wearable band 6010.

[0145] As described above, the wearable band 6010 is configured to be worn by a user. In particular, the wearable band 6010 can be shaped or otherwise manipulated to be worn by a user. For example, the wearable band 6010 can be shaped to have a generally circular shape such that it can be configured to be worn on a user's lower arm or wrist. Alternatively, the wearable band 6010 can be shaped to be worn on another body part of the user (such as the user's upper arm (e.g., around the bicep), forearm, chest, or leg). The wearable band 6010 can include a retention mechanism 6012 (such as a buckle or hook-and-loop fastener) for securing the wearable band 6010 to the user's wrist or other body part. When the wearable band 6010 is worn by the user, the sensor 6013 senses data from the user's skin (referred to as sensor data). In particular, the sensor 6013 of the wearable band 6010 acquires (e.g., senses and records) neuromuscular signals.

[0146] The sensed data (e.g., sensed neuromuscular signals) can be used to detect and / or determine the intention of the user to perform certain movement actions. In particular, when the user performs muscle activation (e.g., movement and / or gesture), the sensor 6013 senses and records the neuromuscular signals from the user. The detected and / or determined movement actions (such as phalangeal (or finger) movement, wrist movement, hand movement, and / or other muscle intentions) can be used to determine control commands or control information (instructions to perform certain commands after the data is sensed) for causing a computing device to execute one or more input commands. For example, the sensed neuromuscular signals can be used to control certain user interfaces displayed on the display 6005 of the wrist-worn device 6000, and / or can be sent to a device responsible for rendering an AR environment (such as a head-mounted display) to perform an action in the associated AR environment (such as to control the movement of a virtual device displayed to the user). The muscle activation performed by the user can include: static gestures, such as placing the user's palm down on a table; dynamic gestures, such as grasping a physical object or a virtual object; and covert gestures that are not perceptible to another person, such as slightly tensing a joint by co-contracting opposing muscles or using submuscular activation. The muscle activation performed by the user can include symbolic gestures (such as gestures that are mapped to other gestures, interactions, or commands based on a gesture vocabulary that specifies a mapping of gestures to commands).

[0147] The sensor data sensed by the sensor 6013 can be used to provide the user with enhanced interaction with physical objects (such as devices communicatively coupled to the wearable band 6010) and / or virtual objects generated by an AR system in an AR application (such as user interface objects presented on the display 6005 or another computing device (such as a smartphone)).

[0148] In some embodiments, the wearable band 6010 includes one or more haptic devices 6046 ( Figure 6B , e.g., vibrotactile actuators), which are configured to provide haptic feedback (e.g., cutaneous sensations and / or kinesthetic sensations) to the user's skin. The sensor 6013 and / or the haptic device 6046 may be configured to operate in conjunction with multiple applications, including but not limited to health monitoring, social media, gaming, and artificial reality (e.g., applications associated with artificial reality).

[0149] The wearable band 6010 may further include a coupling mechanism 6016 (e.g., a bracket of the coupling mechanism or the shape may correspond to the shape of the body 6020 of the wrist-wearable device 6000) for detachably coupling a pod (e.g., a computing unit) or the body 6020 (via the coupling surface of the body 6020) to the wearable band 6010. In particular, the coupling mechanism 6016 may be configured to receive the coupling surface of the body 6020 near the bottom side (e.g., the side opposite to the front side where the display 6005 of the body 6020 is located), such that the user can push the body 6020 downward into the coupling mechanism 6016 to attach the body 6020 to the coupling mechanism 6016. In some embodiments, the coupling mechanism 6016 may be configured to receive the top side of the body 6020 (e.g., the side close to the front side where the display 6005 of the body 6020 is located), and the body 6020 is pushed upward into the bracket instead of being pushed downward into the coupling mechanism 6016. In some embodiments, the coupling mechanism 6016 is an integrated component of the wearable band 6010, such that the wearable band 6010 and the coupling mechanism 6016 are a single integral structure. In some embodiments, the coupling mechanism 6016 is a type of frame or housing that allows the coupling surface of the body 6020 to be held within or on the coupling mechanism 6016 (e.g., a bracket, a tracking band, a support base, or a buckle) of the wearable band 6010.

[0150] The coupling mechanism 6016 may allow the watch body 6020 to be detachably coupled to the wearable band 6010 by: friction fit; magnetic coupling; rotation-based connectors; shear pin couplings; retaining springs; one or more magnets; clips; pin shafts; hook-and-loop fasteners; or combinations thereof. The user may perform any type of movement to couple the watch body 6020 to the wearable band 6010 and to separate the watch body 6020 from the wearable band 6010. For example, the user may twist, slide, rotate, push, pull, or spin (or combinations thereof) the watch body 6020 relative to the wearable band 6010 to attach the watch body 6020 to the wearable band 6010 and to detach the watch body 6020 from the wearable band 6010. Alternatively, as discussed below, in some embodiments, the watch body 6020 may be separated from the wearable band 6010 by actuation of a release mechanism 6029.

[0151] The wearable band 6010 may be coupled with the watch body 6020 to increase the functionality of the wearable band 6010 (e.g., convert the wearable band 6010 into a wrist-wearable device 6000, add additional computing units and / or batteries to increase the computing resources and / or battery life of the wearable band 6010, add additional sensors to improve the sensed data). As described above, the wearable band 6010 (and the coupling mechanism 6016) is configured to operate independently of the watch body 6020 (e.g., perform functions independent of the watch body 6020). For example, the coupling mechanism 6016 may include one or more sensors 6013 that contact the user's skin when the user is wearing the wearable band 6010 and provide sensor data for determining control commands.

[0152] The user may remove the watch body 6020 (or pod) from the wearable band 6010 to reduce the burden on the user of the wrist-wearable device 6000. For embodiments in which the watch body 6020 is detachable, the watch body 6020 may be referred to as a detachable structure, such that in these embodiments, the wrist-wearable device 6000 includes a wearable portion (e.g., the wearable band 6010) and a detachable structure (the watch body 6020).

[0153] Turn to the body 6020. The body 6020 can have a generally rectangular or circular shape. The body 6020 is configured to be worn by a user on their wrist or another body part. More specifically, the body 6020 is sized to be easily carried by the user, attached to a part of the user's clothing, and / or coupled to the wearable band 6010 (forming the wrist wearable device 6000). As described above, the body 6020 can have a shape corresponding to the coupling mechanism 6016 of the wearable band 6010. In some embodiments, the body 6020 includes a single release mechanism 6029 or multiple release mechanisms (e.g., two release mechanisms 6029 located on opposite sides of the body 6020, such as spring-loaded buttons) to separate the body 6020 from the wearable band 6010. The release mechanism 6029 can include, but is not limited to, buttons, knobs, plugs, handles, joysticks, fasteners, buckles, dials, latches, or combinations thereof.

[0154] The user can actuate the release mechanism 829 by pushing, turning, lifting, pressing, moving the release mechanism 6029, or performing other actions on the release mechanism 6029. Actuating the release mechanism 6029 can release (e.g., separate) the body 6020 from the coupling structure 6016 of the wearable band 6010, allowing the user to use the body 6020 independently of the wearable band 6010, and vice versa. For example, separating the body 6020 from the wearable band 6010 can allow the user to use the rear camera 6025B to capture images. Although the release mechanism 6029 is shown as being located at the corners of the body 6020, the release mechanism 6029 can be located at any position on the body 6020 that is convenient for the user to operate. Additionally, in some embodiments, the wearable band 6010 can also include a corresponding release mechanism for separating the body 6020 from the coupling mechanism 6016. In some embodiments, the release mechanism 6029 is optional, and the body 6020 can be separated from the coupling mechanism 6016 as described above (e.g., by twisting or rotating).

[0155] The body 6020 can include one or more peripheral buttons 6023 and 6027 for performing various operations on the body 6020. For example, the peripheral buttons 6023 and 6027 can be used to turn on or wake up the display 6005 (e.g., transition from a sleep state to an active state), unlock the body 6020, increase or decrease the volume, increase or decrease the brightness, interact with one or more applications, and / or interact with one or more user interfaces. Additionally or alternatively, in some embodiments, the display 6005 serves as a touchscreen and allows the user to provide one or more inputs for interacting with the body 6020.

[0156] In some embodiments, the watch body 6020 includes one or more sensors 6021. The sensors 6021 of the watch body 6020 may be the same as or different from the sensors 6013 of the wearable band 6010. The one or more sensors 6021 of the watch body 6020 may be distributed on the inner surface and / or outer surface of the watch body 6020. In some embodiments, the sensor 6021 is configured to contact the user's skin when the user wears the watch body 6020. For example, the sensor 6021 may be placed on the bottom side of the watch body 6020, and the coupling mechanism 6016 may be a bracket with an opening that allows the bottom side of the watch body 6020 to directly contact the user's skin. Alternatively, in some embodiments, the watch body 6020 does not include a sensor configured to contact the user's skin (e.g., including sensors inside and / or outside the watch body 6020 that are configured to sense data of the watch body 6020 and data of the surrounding environment of the watch body 6020). In some embodiments, the sensor 6013 is configured to track the position and / or movement of the watch body 6020.

[0157] The watch body 6020 and the wearable band 6010 may use a wired communication method (e.g., Universal Asynchronous Receiver / Transmitter (UART) or USB transceiver) and / or a wireless communication method (e.g., near field communication or Bluetooth) to share data. For example, the watch body 6020 and the wearable band 6010 may share data sensed by the sensors 6013 and 6021, as well as application and device specific information (e.g., activated and / or available applications, output devices (e.g., displays and / or speakers), input devices (e.g., touchscreens, microphones, and / or imaging sensors)).

[0158] In some embodiments, the watch body 6020 may include, but is not limited to, a front camera 6025A and / or a rear camera 6025B, sensors 6021 (e.g., biometric sensors, IMUs, heart rate sensors, oxygen saturation sensors, neuromuscular signal sensors, altimeter sensors, temperature sensors, bioimpedance sensors, pedometer sensors, optical sensors (e.g., imaging sensor 6063; Figure 6B ), touch sensors, and / or sweat sensors). In some embodiments, the watch body 6020 may include one or more haptic devices 6076 ( Figure 6B; a vibrotactile actuator), the one or more haptic devices 6076 are configured to provide haptic feedback to a user (e.g., cutaneous sensations and / or kinesthetic sensations). The sensor 6021 and / or the haptic device 6076 may also be configured to operate in conjunction with a plurality of applications, including but not limited to health monitoring applications, social media applications, gaming applications, and artificial reality applications (e.g., applications associated with artificial reality).

[0159] As described above, the watch body 6020 and the wearable band 6010 can form a wrist wearable device 6000 when coupled. When the watch body 6020 and the wearable band 6010 are coupled, they function as a single device to perform the functions (operations, detections, and / or communications) described herein. In some embodiments, each device is provided with specific instructions for performing one or more operations of the wrist wearable device 6000. For example, based on determining that the watch body 6020 does not include a neuromuscular signal sensor, the wearable band 6010 may include alternative instructions for performing the associated instructions (e.g., providing sensed neuromuscular signal data to the watch body 6020 through a different electronic device). The operations of the wrist wearable device 6000 can be performed by the watch body 6020 alone or by the watch body 6020 in cooperation with the wearable band 6010 (e.g., through respective processors and / or hardware components), and vice versa. In some embodiments, the operations of the wrist wearable device 6000, the watch body 6020, and / or the wearable band 6010 can be performed in conjunction with one or more processors and / or hardware components of another communicatively coupled device (e.g., HIPD 8000; Figure 8A and Figure 8B ).

[0160] As described below with reference to Figure 6B the block diagrams of, the wearable band 6010 and / or the watch body 6020 may each include independent resources required to perform functions independently. For example, the wearable band 6010 and / or the watch body 6020 may each include a power source (e.g., a battery), a memory, a data storage device, a processor (e.g., a central processing unit (CPU)), a communication, a light source, and / or an input / output device.

[0161] Figure 6B A block diagram of a computing system is shown according to some embodiments: a computing system 6030 corresponding to the wearable band 6010, and a computing system 6060 corresponding to the watch body 6020. According to some embodiments, the computing system of the wrist wearable device 6000 includes a combination of components of the wearable band computing system 6030 and the watch body computing system 6060.

[0162] The watch body 6020 and / or the wearable band 6010 may include one or more components shown in the watch body computing system 6060. In some embodiments, a single integrated circuit includes all or most of the components of the watch body computing system 6060, and these components are included in a single integrated circuit. Alternatively, in some embodiments, the components of the watch body computing system 6060 are included in a plurality of integrated circuits that are communicatively coupled. In some embodiments, the watch body computing system 6060 is configured to be coupled (e.g., via a wired connection or a wireless connection) to the wearable band computing system 6030, which allows the two computing systems to share components, distribute tasks, and / or perform other operations described herein (either individually or as a single device).

[0163] The watch body computing system 6060 may include one or more processors 6079, a controller 6077, a peripheral interface 6061, a power system 6095, and a memory (e.g., memory 6080), each of which is defined above and described in more detail below.

[0164] The power system 6095 may include a charger input 6057, a power-management integrated circuit (PMIC) 6097, and a battery 6096, each of which is defined above. In some embodiments, the watch body 6020 and the wearable band 6010 may each have a battery (e.g., batteries 6098 and 6059), and may share power with each other. The watch body 6020 and the wearable band 6010 may use various techniques to receive a charge. In some embodiments, the watch body 6020 and the wearable band 6010 may use a wired charging component (e.g., a power line) to receive a charge. Alternatively or additionally, the watch body 6020 and / or the wearable band 6010 may be configured for wireless charging. For example, a portable charging device may be designed to match a portion of the watch body 6020 and / or the wearable band 6010 and wirelessly transmit available power to the battery of the watch body 6020 and / or the battery of the wearable band 6010. The watch body 6020 and the wearable band 6010 may have independent power systems (e.g., power systems 6095 and 6056) to enable them to operate independently. The watch body 6020 and the wearable band 6010 may also share power (e.g., one may charge the other) through their respective PMICs (e.g., PMICs 6097 and 6058), which may share power through power conductors and ground conductors and / or through wireless charging antennas.

[0165] In some embodiments, the peripheral interface 6061 may include one or more sensors 6021, and many of the one or more sensors 6021 listed below are defined thereon. Each sensor 6021 may include one or more coupling sensors 6062 for detecting when the watch body 6020 is coupled to another electronic device (e.g., the wearable band 6010). The sensor 6021 may include an imaging sensor 6063 (one or more of the camera 6025 and / or a separate imaging sensor 6063 (e.g., a thermal imaging sensor)). In some embodiments, the sensor 6021 includes one or more SpO2 sensors 6064. In some embodiments, the sensor 6021 includes one or more biopotential signal sensors (e.g., EMG sensors 6065 and 6035, and the EMG sensors 6065 and 6035 may be disposed on the user-facing portions of the watch body 6020 and / or the wearable band 6010). In some embodiments, the sensor 6021 includes one or more capacitive sensors 6066. In some embodiments, the sensor 6021 includes one or more heart rate sensors 6067. In some embodiments, the sensor 6021 includes one or more IMU sensors 6068. In some embodiments, one or more IMU sensors 6068 may be configured to detect the movement of the user's hand or other positions where the watch body 6020 is placed or held).

[0166] In some embodiments, the peripheral interface 6061 includes a near field communication (NFC) component 6069, a global positioning system (GPS) component 6070, a long-term evolution (LTE) component 6071, and / or Wi-Fi and / or Bluetooth (BT) communication components 6072. In some embodiments, the peripheral interface 6061 includes one or more buttons 6073 (e.g., Figure 6A the peripheral buttons 6023 and 6027 in , and the one or more buttons 6073 cause an operation to be performed at the watch body 6020 when selected by the user. In some embodiments, the peripheral interface 6061 includes one or more indicators (e.g., light-emitting diodes (LEDs)) to provide visual indicators to the user (e.g., received message, low battery level, active microphone and / or camera).

[0167] The table body 6020 may include at least one display 6005 to display a visual representation of information or data to a user, the visual representation including user interface elements and / or three-dimensional (3D) virtual objects. The display may also include a touch screen for inputting user inputs (such as touch gestures and swipe gestures, etc.). The table body 6020 may include at least one speaker 6074 and at least one microphone 6075 to provide an audio signal to the user and receive an audio input from the user. The user may provide a user input through the microphone 6075 and may also receive an audio output from the speaker 6074 as part of a haptic event provided by the haptic controller 6078. The table body 6020 may include at least one camera 6025, and the at least one camera 6025 includes a front camera 6025A and a rear camera 6025B. The camera 6025 may include an ultra-wide-angle camera, a wide-angle camera, a fish-eye camera, a spherical camera, a telephoto camera, a depth-sensing camera, or other types of cameras.

[0168] The table body computing system 6060 may include one or more haptic controllers 6077 and associated components (such as, the haptic device 6076), and the one or more haptic controllers 878 and associated components are used to provide haptic events at the table body 6020 (such as, a vibration perception or an audio output provided in response to an event at the table body 6020). The haptic controller 6078 may communicate with one or more haptic devices 6076 (such as electroacoustic devices), and the one or more haptic devices 6076 include a speaker in one or more speakers 6074 and / or other audio components and / or electromechanical devices that convert energy into linear motion (such as motors, electromagnetic coils, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (such as, components that convert an electrical signal into a haptic output on a device)). The haptic controller 6078 may provide a haptic event that can be perceived by a user of the table body 6020 to one or more haptic actuators. In some embodiments, one or more haptic controllers 6078 may receive an input signal from an application in a plurality of applications 6082.

[0169] In some embodiments, computing system 6030 and / or computing system 6060 may include a memory 6080, which may be controlled by a memory controller in one or more controllers 6077. In some embodiments, the software components stored in the memory 6080 include one or more applications 6082 configured to perform operations at the watch body 6020. In some embodiments, the one or more applications 6082 include games, word processors, messaging applications, calling applications, web browsers, social media applications, media streaming applications, financial applications, calendars, and / or clocks. In some embodiments, the software components stored in the memory 6080 include one or more communication interface modules 6083 as defined above. In some embodiments, the software components stored in the memory 6080 include: one or more graphics modules 6084 for rendering, encoding, and / or decoding audio data and / or visual data; and one or more data management modules 6085 for collecting, organizing, and / or providing access to the data 6087 stored in the memory 6080. In some embodiments, one or more of the multiple applications 6082 and / or one or more modules may work together to perform various tasks at the watch body 6020.

[0170] In some embodiments, the software components stored in the memory 6080 may include one or more operating systems 6081 (e.g., a Linux-based operating system or an Android operating system). The memory 6080 may also include data 6087. The data 6087 may include profile data 6088A, sensor data 6089A, media content data 6090, and application data 6091.

[0171] It should be recognized that the watch body computing system 6060 is an example of a computing system within the watch body 6020, and compared to the components shown in the watch body computing system 6060, the watch body 6020 may have more or fewer components, two or more components may be combined, and / or may have different component configurations and / or arrangements. The various components shown in the watch body computing system 6060 are implemented in hardware, software, firmware, or a combination thereof, which includes one or more signal processing and / or application specific integrated circuits.

[0172] Turning to wearable band computing system 6030, one or more components that may be included in wearable band 6010 are shown. Compared to the components shown in body computing system 6060, wearable band computing system 6030 may include more or fewer components, may combine two or more components, and / or may have different configurations and / or arrangements of some or all of the components. In some embodiments, all or most of the components of wearable band computing system 6030 are included in a single integrated circuit. Alternatively, in some embodiments, multiple components of wearable band computing system 6030 are included in multiple integrated circuits that are communicatively coupled. As described above, in some embodiments, wearable band computing system 6030 is configured to be coupled (e.g., via a wired connection or a wireless connection) to body computing system 6060, which allows the computing systems to share components, distribute tasks, and / or perform other operations described herein (either individually or as a single device).

[0173] Similar to body computing system 6060, wearable band computing system 6030 may include one or more processors 6049, one or more controllers 6047 (including one or more haptic controllers 6048), a peripheral interface 6031 (which may include one or more sensors 6013 and other peripheral devices), a power source (e.g., power system 6056), and a memory (e.g., memory 6050) that includes an operating system (e.g., operating system 6051), data (e.g., data 6054, which includes profile data 6088B and / or sensor data 6089B), and one or more modules (e.g., communication interface module 6052 and / or data management module 6053).

[0174] Given the above definitions, one or more sensors 6013 may be similar to sensors 6021 of computing system 6060. For example, sensors 6013 may include one or more coupled sensors 6032, one or more SpO2 sensors 6034, one or more EMG sensors 6035, one or more capacitive sensors 6036, one or more heart rate sensors 6037, and one or more IMU sensors 6038.

[0175] The peripheral interface 6031 may also include other components similar to those included in the peripheral interface 6061 of the computing system 6060. These components include the NFC component 6039, GPS component 6040, LTE component 6041, Wi-Fi and / or Bluetooth communication component 6042, and / or one or more haptic devices 6076 as described above with reference to the peripheral interface 6061. In some embodiments, the peripheral interface 6061 includes one or more buttons 6043, a display 6033, a speaker 6044, a microphone 6045, and a camera 6055. In some embodiments, the peripheral interface 6061 includes one or more indicators, such as LEDs.

[0176] It should be recognized that the wearable band computing system 6030 is an example of a computing system within the wearable band 6010, and compared to the components shown in the wearable band computing system 6030, the wearable band 6010 may have more or fewer components, two or more components may be combined, and / or may have different component configurations and / or arrangements. Each of the components shown in the wearable band computing system 6030 may be implemented in one of hardware, software, and firmware or a combination of hardware, software, and firmware (including one or more signal processing circuits and / or application specific integrated circuits).

[0177] Reference Figure 6A The wrist wearable device 6000 of [] is an example of the coupling of the wearable band 6010 and the watch body 6020, and thus the wrist wearable device 6000 will be understood to include the components shown and described for the wearable band computing system 6030 and the watch body computing system 6060. In some embodiments, the wrist wearable device 6000 has a split architecture (e.g., a split mechanical architecture or a split electrical architecture) between the watch body 6020 and the wearable band 6010. In other words, all of the components shown in the wearable band computing system 6030 and the watch body computing system 6060 may be housed or otherwise provided in the combined watch device 6000, or may be housed or otherwise provided in individual components of the watch body 6020, the wearable band 6010, and / or portions thereof (e.g., the coupling mechanism 6016 of the wearable band 6010).

[0178] The above techniques may be used with any device for sensing neuromuscular signals (including Figure 6A and Figure 6B the arm wearable device in []), but may also be used with other types of wearable devices for sensing neuromuscular signals (e.g., body wearable devices or head wearable devices that may have neuromuscular sensors closer to the brain or spine).

[0179] In some embodiments, the wrist-worn device 6000 can be used in combination with the head-worn devices described below (e.g., the AR system 7000 and the VR headset 7010) and / or the HIPD 8000; and the wrist-worn device 6000 can also be configured to allow a user to control aspects of the artificial reality (e.g., by using EMG-based gestures to control user interface objects in the artificial reality, and / or by allowing the user to interact with a touchscreen on the wrist-worn device to also control aspects of the artificial reality). In some embodiments, the wrist-worn device 6000 can also be used in combination with wearable apparel (e.g., the wearable glove described below with reference to Figures 9A to 9C ). After describing the example wrist-worn device in this way, attention will now be turned to example head-worn devices, such as the AR system 7000 and the VR headset 7010.

[0180] Example head-worn devices

[0181] Figure 7A , Figure 7B-1 , Figure 7B-2 and Figure 7C illustrate example AR systems, which include the AR system 7000. In some embodiments, the AR system 7000 is a glasses device as shown in Figure 7A . In some embodiments, the VR system 7010 includes a head-mounted display (HMD) 7012, as shown in Figure 7B-1 and Figure 7B-2 . In some embodiments, the AR system 7000 and the VR system 7010 include one or more similar components (e.g., components for presenting an interactive AR environment, such as processors, memories, and / or presentation devices, which presentation devices include one or more displays and / or one or more waveguides), and some of these components are described in more detail with reference to Figure 7C . As described herein, the head-worn device can include components of the glasses device 7002 and / or components of the head-mounted display 7012. Some embodiments of the head-worn device do not include any displays (which displays include any displays described with respect to the AR system 7000 and / or the VR system 7010). Although the example AR systems are described herein as the AR system 7000 and the VR system 7010 respectively, any one or both of the example AR systems described herein can be configured to present a fully immersive VR scene presented in substantially the entire user's field of view, as a supplement or alternative to a more subtle augmented reality scene presented in a portion of the user's field of view that is less than the entire field of view.

[0182] Figure 7AShows an example visual depiction of an AR system 7000 (which may also be described herein as augmented reality glasses and / or smart glasses). The AR system 7000 may include Figure 7A Additional electronic components not shown in Figure 7A (such as wearable accessory devices and / or intermediate processing devices), which are configured to be used in combination with the glasses device in electronic communication or otherwise. In some embodiments, the wearable accessory device and / or the intermediate processing device may be configured to be coupled to the glasses device via a coupling mechanism that is in electronic communication with the coupling sensor 7024, where the coupling sensor 7024 may detect when an electronic device becomes physically or electronically coupled to the glasses device. In some embodiments, the glasses device is configured to be coupled to a housing 7090, and the housing 7090 may include one or more additional coupling mechanisms that are configured to be coupled to additional accessory devices. Figure 7A The components shown in Figure 7A may be implemented in hardware, software, firmware, or a combination thereof (including one or more signal processing components and / or application-specific integrated circuits (ASICs)).

[0183] The glasses device includes mechanical glasses components, and these mechanical glasses components include a frame 7004, which is configured to hold one or more lenses (e.g., one lens or two lenses 7006-1 and 7006-2). Those of ordinary skill in the art will recognize that the glasses device may include additional mechanical components, such as hinges configured to allow portions of the frame 7004 of the glasses device 7002 to fold and unfold, a bridge configured to span the gap between the lenses 7006-1 and 7006-2 and rest on the user's nose, nose pads configured to rest on the nose and provide support for the glasses device, earpieces configured to rest on the user's ears and provide additional support for the glasses device, temple arms configured to extend from the hinge to the earpiece of the glasses device, etc. Those of ordinary skill in the art will further recognize that some examples of the AR system 7000 may not include the mechanical components described herein. For example, smart contact lenses configured to present an artificial reality to the user may not include any components of the glasses device.

[0184] The glasses device includes electronic components, and many of these electronic components will be described in more detail below with reference to Figure 7C Some example electronic components are as Figure 7AAs shown, these electronic components include acoustic sensors 7025-1, 7025-2, 7025-3, 7025-4, 7025-5, and 7025-6 that can be distributed along most of the frame 7004 of the glasses device. The glasses device also includes a left camera 7039A and a right camera 7039B located on different sides of the frame 7004. And the glasses device includes a processor 7048 (e.g., an integrated microprocessor such as an ASIC) embedded in a part of the frame 7004.

[0185] Figure 7B-1 and Figure 7B-2 Shown is a head-mounted display (HMD) 7012 (e.g., also referred to herein as an AR head-mounted device, a head-wearable device, or a VR head-mounted device) according to some embodiments. As described above, some AR systems (e.g., AR system 7000) can generally replace one or more of a user's multiple sensory perceptions of the real world with a virtual experience (e.g., AR systems 5000c and 5000d), rather than mixing artificial reality with actual reality.

[0186] The HMD 7012 includes a front body 7014 and a frame 7016 (e.g., a bar or a strap) shaped to fit a user's head. In some embodiments, the front body 7014 and / or the frame 7016 includes one or more such electronic components (e.g., a display, an IMU, a tracking transmitter, or a detector): the one or more electronic components are used to facilitate the presentation of an AR system and / or a VR system and / or the interaction with an AR system and / or a VR system. In some embodiments, as Figure 7B-2 shown, the HMD 7012 includes an output audio transducer (e.g., audio transducer 7018-1). In some embodiments, as Figure 7B-2 shown, one or more components (e.g., one or more output audio transducers 7018-1 and the frame 7016) (e.g., a part or all of the frame 7016 and / or the audio transducer 7018-1) can be configured to be attached (e.g., detachably attached) to and detached from the HMD 7012. In some embodiments, coupling a detachable component to the HMD 7012 enables the detachable component to enter into electronic communication with the HMD 7012.

[0187] Figure 7B-1 and Figure 7B-2The VR system 7010 is also shown to have one or more cameras, such as a left camera 7039A and a right camera 7039B (which may be similar to the left and right cameras on the frame 7004 of the glasses device 7002). In some embodiments, the VR system 7010 includes one or more additional cameras (e.g., cameras 7039C and 7039D), which may be configured to enhance the image data acquired by cameras 7039A and 7039B by providing more information. For example, camera 839C may be used to provide color information not recognized by cameras 7039A and 7039B. In some embodiments, one or more of the cameras 7039A to 7039D may include an optional infrared (IR) cut-off filter, which is configured to remove IR light from the light received by the corresponding camera sensor.

[0188] Figure 7C The computing system 7020 and an optional housing 7090 are shown, and each of the computing system and the housing shows components that may be included in the AR system 7000 and / or the VR system 7010. In some embodiments, more or fewer components may be included in the optional housing 7090 depending on the actual constraints of the corresponding AR system being described.

[0189] In some embodiments, the computing system 7020 and / or the optional housing 7090 may include one or more peripheral interfaces 7022, one or more power systems 7042, one or more controllers 7046 (including one or more haptic controllers 7047), one or more processors 7048 (as defined above, including any of the examples provided in the examples), and a memory 7050, all of which may communicate electronically with each other. For example, one or more processors 7048 may be configured to execute instructions stored in the memory 7050, and the instructions may cause a controller in one or more of the controllers 7046 to cause a plurality of operations to be performed at one or more peripheral devices of the peripheral interface 7022. In some embodiments, each of the operations described may be performed based on the power provided by the power system 7042.

[0190] In some embodiments, the peripheral interface 7022 may include one or more devices configured as part of the computing system 7020, and many of the one or more devices have been defined and / or referenced above Figure 6A and Figure 6BA wrist-wearable device as shown is described. For example, the peripheral interface may include one or more sensors 7023. Some example sensors include: one or more coupled sensors 7024, one or more acoustic sensors 7025, one or more imaging sensors 7026, one or more EMG sensors 7027, one or more capacitive sensors 7028, and / or one or more IMU sensors 7029; and / or any other type of sensor defined above or described with respect to any other embodiment discussed herein.

[0191] In some embodiments, the peripheral interface may include one or more additional peripheral devices, the one or more additional peripheral devices including: one or more NFC devices 7030, one or more GPS devices 7031, one or more LTE devices 7032, one or more Wi-Fi and / or Bluetooth devices 7033, one or more buttons 7034 (e.g., including slidable or otherwise adjustable buttons), one or more displays 7035, one or more speakers 7036, one or more microphones 7037, one or more cameras 7038 (e.g., including a left camera 7039A and / or a right camera 7039B), and / or one or more haptic devices 7040; and / or any other type of peripheral device defined above or described with respect to any other embodiment discussed herein.

[0192] The AR system may include various types of visual feedback mechanisms (e.g., presentation devices). For example, the display device in the AR system 7000 and / or the VR system 7010 may include one or more liquid-crystal displays (LCDs), light-emitting diode (LED) displays, organic light-emitting diode (OLED) displays, and / or any other suitable type of display screen. The AR system may include a single display screen (e.g., configured to be seen by both eyes), and / or may provide a separate display screen for each eye, which may allow for additional flexibility for zoom adjustment and / or for correcting refractive errors associated with the user's vision. Some embodiments of the AR system also include an optical subsystem having one or more lenses (e.g., conventional concave or convex lenses, Fresnel lenses, or adjustable liquid lenses) through which the user may view the display screen.

[0193] For example, a corresponding display may be coupled to each of lenses 7006-1 and 7006-2 of AR system 7000. A plurality of displays coupled to each of lenses 7006-1 and 7006-2 may be used together or independently to present an image or series of images to a user. In some embodiments, AR system 7000 includes a single display (e.g., a near-eye display) or more than two displays. In some embodiments, one or more displays of a first group may be used to present an augmented reality environment, and one or more display devices of a second group may be used to present a virtual reality environment. In some embodiments, one or more waveguides are used in conjunction with presenting AR content to a user of AR system 7000 (e.g., as a way to transmit light from one or more displays to the user's eyes). In some embodiments, one or more waveguides are fully or partially integrated into eyewear device 7002. As a supplement or alternative to a display screen, some AR systems include one or more projection systems. For example, the display device in AR system 7000 and / or virtual reality system 7010 may include a micro-LED projector that projects light (e.g., using a waveguide) into a display device (such as clear combiner lenses that allow ambient light to pass through). The display device may refract the projected light into the user's pupil and enable the user to view both AR content and the real world simultaneously. The AR system may also be configured with any other suitable type or form of image projection system. In some embodiments, one or more waveguides are provided to supplement or replace one or more displays.

[0194] The computing system 7020 and / or optional housing 7090 of AR system 7000 or VR system 7010 may include some or all of the components of power system 7042. Power system 7042 may include one or more charger inputs 7043, one or more PMICs 7044, and / or one or more batteries 7045.

[0195] Memory 7050 includes instructions and data, some or all of which may be stored in memory 7050 as a non-transitory computer-readable storage medium. For example, memory 7050 may include: one or more operating systems 7051; one or more applications 7052; one or more communication interface applications 7053; one or more graphics applications 7054; one or more AR processing applications 7055; and / or any other type of data defined above or described with respect to any other embodiments discussed herein.

[0196] The memory 7050 also includes data 7060 that can be used in conjunction with one or more of the above applications. The data 7060 can include: profile data 7061; sensor data 7062; media content data 7063; AR application data 7064; and / or any other type of data defined above or described with respect to any other embodiments discussed herein.

[0197] In some embodiments, the controller 7046 of the eyewear device 7002 processes information generated by sensors 7023 on the eyewear device 7002 within the AR system 7000 and / or another electronic device. For example, the controller 7046 can process information from the acoustic sensors 7025-1 and 7025-2. For each detected sound, the controller 7046 can perform a direction of arrival (DOA) estimation to estimate from which direction the detected sound arrives at the eyewear device 7002 of the AR system 7000. When one or more of the acoustic sensors 7025 detect a sound, the controller 7046 can populate an audio data set with the information (e.g., represented as sensor data 7062 in Figure 7C ).

[0198] In some embodiments, a physical electronic connector can transfer information between the eyewear device and another electronic device and / or between one or more processors and the controller 7046 of the AR system 7000 or VR system 7010. This information can be in the form of optical data, electrical data, wireless data, or any other transmissible data form. Shifting the processing of information generated by the eyewear device to an intermediate processing device can reduce the weight and heat of the eyewear device, making the eyewear device more comfortable and safer for the user. In some embodiments, an optional wearable accessory device (e.g., an electronic tie) is coupled to the eyewear device via one or more connectors. Each connector can be a wired connector or a wireless connector and can include electrical and / or non-electrical (e.g., structural) components. In some embodiments, the eyewear device and the wearable accessory device can operate independently without any wired or wireless connection between them.

[0199] In some cases, an external device such as an intermediate processing device (e.g., HIPD 8000) is paired with a glasses device 7002 (e.g., as part of an AR system 7000), enabling the glasses device 7002 to achieve the shape elements similar to those of a pair of glasses while still providing sufficient battery and computing power for the extended capabilities. Some or all of the battery power, computing resources, and / or additional features of the AR system 7000 can be provided by or shared between the paired device and the glasses device 7002, thus generally reducing the weight, heat profile, and shape elements of the glasses device 7002 while allowing the glasses device 7002 to maintain its desired functions. For example, a wearable accessory device can allow components that would otherwise be included on the glasses device 7002 to be included in the wearable accessory device and / or the intermediate processing device, thereby transferring the weight load from the user's head and neck to one or more other parts of the user's body. In some embodiments, the intermediate processing device has a larger surface area to dissipate and spread heat to the surrounding environment through this larger surface area. Thus, the intermediate processing device can allow for greater battery and computing power compared to the battery and computing power that the glasses device 7002 might otherwise have when used alone. Since the weight carried in the wearable accessory device may have less impact on the user than the weight carried in the glasses device 7002, the user can tolerate wearing a lighter glasses device and carrying or wearing the paired device for a longer time compared to tolerating wearing a heavier glasses device alone, thereby enabling the artificial reality environment to be more fully integrated into the user's daily activities.

[0200] AR systems can include various types of computer vision components and subsystems. For example, the AR system 7000 and / or the VR system 7010 can include one or more optical sensors, such as two-dimensional (2D) or 3D cameras, time-of-flight depth sensors, single-beam or scanning lidar sensors, 3D lidar (LiDAR) sensors, and / or any other suitable type or form of optical sensor. The AR system can process data from one or more of these sensors to identify the user's location and / or multiple aspects of the user's real-world physical environment (including the location of real-world objects in the real-world physical environment). In some embodiments, the methods described herein are used to map the real world, provide context to the user about the real-world environment, and / or generate digital twins (e.g., interactive virtual objects), as well as various other functions. For example, Figure 7B-1 and Figure 7B-2Shows a VR system 7010 with cameras 7039A to 7039D, which can be used to provide depth information for creating a voxel field and a 2D grid to provide object information to the user to avoid collisions.

[0201] In some embodiments, the AR system 7000 and / or the VR system 7010 may include a haptic (tactile) feedback system that can be incorporated into a headset, gloves, a bodysuit, a handheld controller, environmental devices (e.g., a chair or footpad), and / or any other type of device or system (e.g., the wearable devices discussed herein). The haptic feedback system can provide various types of cutaneous feedback, including vibration, force, pull, shear, texture, and / or temperature. The haptic feedback system can also provide various types of kinesthetic feedback, such as motion and compliance. The haptic feedback can be implemented using motors, piezoelectric actuators, fluid systems, and / or various other types of feedback mechanisms. The haptic feedback system can be implemented independently of, within, and / or in combination with other AR devices (e.g., the haptic feedback system described in reference Figures 9A to 9C ).

[0202] In some embodiments of AR systems such as the AR system 7000 and / or the VR system 7010, ambient light (e.g., a live feed of the surrounding environment that the user would normally see) can pass through the display element of the corresponding head-mounted device presenting aspects of the AR system. In some embodiments, the ambient light can pass through less than the entire AR environment presented within the user's field of view, e.g., a portion of the AR environment that is in the same location as a physical object in the user's real-world environment, where the physical object is within a specified boundary (e.g., a guardian boundary), and the portion of the AR environment is configured for the user to use when interacting with the AR environment. For example, visual user interface elements (e.g., notification user interface elements) can be presented at the head-mounted device, and a certain amount of ambient light (e.g., 15% to 50% of the ambient light) can pass through the user interface element such that the user can distinguish at least a portion of the physical environment on which the user interface element is being displayed.

[0203] Example handheld intermediate processing device

[0204] Figure 8A and Figure 8BIllustrates an example handheld intermediate processing device (HIPD) 8000 according to some embodiments. The HIPD 8000 is an example of the intermediate device described herein, such that the HIPD 8000 should be understood to have the features described with respect to any intermediate device defined above or otherwise described herein, and vice versa. Figure 8A Shows a top view 8005 and a side view 8025 of the HIPD 8000. The HIPD 8000 is configured to be communicatively coupled to one or more wearable devices (or other electronic devices) associated with a user. For example, the HIPD 8000 is configured to be communicatively coupled to the user's wrist wearable device 6000 (or its components, such as the watch body 6020 and the wearable band 6010), the AR system 7000, and / or the VR headset 7010. The HIPD 8000 can be configured to be held by a user (e.g., as a handheld controller), carried by the user (e.g., in their pocket, in their bag), placed near the user (e.g., on their table when the user is sitting at their table, on a charging dock), and / or placed at or within a predetermined distance from the wearable device or other electronic device (e.g., where, in some embodiments, the predetermined distance is the maximum distance at which the HIPD 8000 can successfully communicatively couple to an electronic device (e.g., a wearable device) (e.g., 10 meters)).

[0205] The HIPD 8000 can perform various functions independently and / or in combination with one or more wearable devices (e.g., wrist wearable device 6000, AR system 7000, and / or VR headset 7010). The HIPD 8000 is configured to augment and / or improve the functionality of devices that are communicatively coupled (such as wearable devices). The HIPD 8000 is configured to perform one or more functions or operations associated with: interacting with the user interfaces and applications of communicatively coupled devices, interacting with an AR environment, interacting with a VR environment, and / or operating as a human-machine interface controller. Additionally, as will be described in more detail below, the functions and / or operations of the HIPD 8000 can include, but are not limited to: task offloading and / or transfer; heat offloading and / or transfer; six degrees of freedom (6DoF) ray casting and / or gaming (e.g., using imaging device or camera 8014, which can be used for simultaneous localization and mapping (SLAM) and / or used in conjunction with other image processing techniques); portable charging; messaging; image capture via one or more imaging devices or cameras 8022; sensing user input (e.g., sensing a touch on touch input surface 8002); wireless communication and / or interconnectivity (e.g., cellular, near field, Wi-Fi, personal area network); location determination; financial transactions; providing haptic feedback; alerts; notifications; biometric authentication; health monitoring; and sleep monitoring. The example functions above can be performed independently in the HIPD 8000 and / or through communication between the HIPD 8000 and another wearable device described herein. In some embodiments, the functions can be performed in the HIPD 8000 in conjunction with an AR environment. As those skilled in the art will recognize upon reading the various descriptions provided herein, the novel HIPD 8000 described herein can be used with any suitable type of AR environment.

[0206] When the HIPD 8000 is communicatively coupled to a wearable device and / or other electronic devices, the HIPD 8000 is configured to perform one or more operations initiated at the wearable device and / or the other electronic devices. In particular, one or more operations of the wearable device and / or other electronic devices can be transferred to the HIPD 8000 for execution. The HIPD 8000 performs one or more operations of the wearable device and / or other electronic devices and provides data corresponding to the completed operations to the wearable device and / or other electronic devices. For example, a user can initiate a video stream using the AR system 7000, and the backend tasks associated with performing the video stream (e.g., video rendering) can be transferred to the HIPD 8000, which performs the backend tasks and provides the corresponding data to the AR system 7000 to perform the remaining front-end tasks associated with the video stream (e.g., presenting the rendered video data through the display of the AR system 7000). In this way, the HIPD 8000, which has more computing resources and greater thermal headroom compared to the wearable device, can perform computationally intensive tasks for the wearable device to improve the performance of the operations performed by the wearable device.

[0207] The HIPD 8000 includes a multi-touch input surface 8002 on a first side (e.g., front surface), which is configured to detect one or more user inputs. In particular, the multi-touch input surface 8002 can detect single-tap inputs, multi-tap inputs, swipe gestures and / or inputs, force- and / or pressure-based touch inputs, and hold taps, etc. The multi-touch input surface 8002 is configured to detect capacitive touch inputs and / or force (and / or pressure) touch inputs. The multi-touch input surface 8002 includes a touch input surface 8004 defined by a surface depression and a touch input surface 8006 defined by a generally flat portion. The touch input surface 8004 can be set adjacent to the touch input surface 8006. In some embodiments, the touch input surface 8004 and the touch input surface 8006 can be of different sizes, shapes, and / or cover different portions of the multi-touch input surface 8002. For example, the touch input surface 8004 can be generally circular, and the touch input surface 8006 is generally rectangular. In some embodiments, the surface depression of the multi-touch input surface 8002 is configured to guide the user's operation of the HIPD 8000. In particular, the surface depression is configured such that the user holds the HIPD 8000 vertically when holding it with one hand (e.g., such that the imaging devices or cameras 8014A and 8014B used point towards the ceiling or sky). Additionally, the surface depression is configured such that the user's thumb is located within the touch input surface 8004.

[0208] In some embodiments, different touch input surfaces include a plurality of touch input areas. For example, touch input surface 8006 includes at least touch input area 8008 within touch input area 8006 and touch input area 8010 within touch input area 8008. In some embodiments, one or more of the touch input areas in each touch input area are optional and / or user-defined (e.g., a user can specify a touch input area based on their preferences). In some embodiments, each touch input surface and / or touch input area is associated with a predetermined set of commands. For example, user input detected within touch input area 8008 causes HIPD 8000 to execute a first command, and user input detected within touch input area 8006 causes HIPD 8000 to execute a second command, the second command being different from the first command. In some embodiments, different touch input surfaces and / or touch input areas are configured to detect one or more types of user input. Different touch input surfaces and / or touch input areas can be configured to detect the same type or different types of user input. For example, touch input area 8008 can be configured to detect force touch input (e.g., the magnitude of a user press) and capacitive touch input, and touch input area 8006 can be configured to detect capacitive touch input.

[0209] HIPD 8000 includes one or more sensors 8051 that are used to sense data used in performing one or more operations and / or functions. For example, HIPD 8000 can include an IMU sensor that is used in conjunction with camera 8014 for 3D object manipulation (e.g., scaling, moving, or destroying an object) in an AR or VR environment. Non-limiting examples of sensors 8051 included in HIPD 8000 include light sensors, magnetometers, depth sensors, pressure sensors, and force sensors. Additional examples of sensors 8051 are provided below with reference to Figure 8B Provide additional examples of sensors 8051.

[0210] The HIPD 8000 may include one or more light indicators 8012 to provide one or more notifications to a user. In some embodiments, the light indicator is an LED or other type of lighting device. The light indicator 8012 may serve as a privacy light to notify the user and / or others near the user that the imaging device and / or microphone is activated. In some embodiments, the light indicator is located near one or more touch input surfaces. For example, the light indicator may be located around the touch input surface 8004. The light indicator may illuminate in different colors and / or patterns to provide one or more notifications and / or information to the user about the device. For example, the light indicator located around the touch input surface 8004 may blink when the user receives a notification (e.g., a message), may turn red when the HIPD 8000 is powered off, may be used as a progress bar (e.g., a light ring that closes when a task is completed (e.g., 0% to 100%)), used as a volume indicator, etc.

[0211] In some embodiments, the HIPD 8000 includes one or more additional sensors on another surface. For example, as Figure 8A shown, the HIPD 8000 includes a group of one or more sensors (e.g., sensor group 8020) located on the edge of the HIPD 8000. When located on the edge of the HIPD 8000, the sensor group 8020 may be positioned at a predetermined tilt angle (e.g., 26 degrees), which allows the sensor group 8020 to be tilted towards the user when placed on a table or other flat surface. Alternatively, in some embodiments, the sensor group 8020 is located on the surface (e.g., the back) opposite the multi-touch input surface 8002. One or more sensors in the sensor group 8020 are discussed in detail below.

[0212] The side view 8025 of the HIPD 8000 shows the sensor group 8020 and the camera 8014B. The sensor group 8020 includes one or more cameras 8022A and 8022B, a depth projector 8024, an ambient light sensor 8028, and a depth receiver 8030. In some embodiments, the sensor group 8020 includes a light indicator 8026. The light indicator 8026 can be used as a privacy indicator to let the user and / or people around the user know that the camera and / or microphone are active. The sensor group 8020 is configured to capture the facial expressions of the user such that the user can manipulate a customized avatar (e.g., display the user's emotions, such as a smile and / or laugh on the user's avatar or digital representation). The sensor group 8020 can be configured as a side stereo RGB system, a rear indirect Time-of-Flight (iToF) system, or a rear stereo Red Green Blue (RGB) system. As those skilled in the art will recognize upon reading the description provided herein, the HIPD 8000 described herein can use different sensor group 8020 configurations and / or sensor group 8020 arrangements.

[0213] In some embodiments, the HIPD 8000 includes one or more haptic devices 8071 (e.g., vibrotactile actuators), and the one or more haptic devices 8071 are configured to provide haptic feedback (e.g., kinesthetic sensations). The sensor 8051 and / or the haptic device 8071 can be configured to operate in conjunction with a plurality of applications and / or communication-coupled devices, including but not limited to wearable devices, health monitoring applications, social media applications, gaming applications, and AR applications (e.g., applications associated with artificial reality).

[0214] The HIPD 8000 is configured to operate without a display. However, in an alternative embodiment, the HIPD 8000 can include a display 8068( Figure 8B ). The HIPD 8000 can also include one or more optional peripheral buttons 8067( Figure 8B)。For example, the peripheral button 8067 can be used to turn the HIPD 8000 on or off. Additionally, the housing of the HIPD 8000 can be formed of a polymer and / or an elastomer. The HIPD 8000 can be configured to have a non-slip surface to allow the HIPD 8000 to be placed on a surface without the user having to monitor the HIPD 8000. In other words, the HIPD 8000 is designed so that it will not easily slide off the surface. In some embodiments, the HIPD 8000 includes one or more magnets for coupling the HIPD 8000 to another surface. This allows the user to place the HIPD 8000 on different surfaces and provides the user with greater flexibility in using the HIPD 8000.

[0215] As described above, the HIPD 8000 can distribute and / or provide instructions for performing one or more tasks at the HIPD 8000 and / or a communication-coupled device. For example, the HIPD 8000 can identify one or more backend tasks to be performed by the HIPD 8000 and one or more frontend tasks to be performed by the communication-coupled device. Although the HIPD 8000 is configured to transfer and / or convey tasks of the communication-coupled device, the HIPD 8000 can (e.g., via one or more processors such as the CPU 8077; Figure 8B ) perform both backend tasks and frontend tasks. The HIPD 8000 can be used to perform (but is not limited to): enhanced calling (e.g., receiving and / or sending 3D or 2.5D live volumetric calls, live digital human representation calls, and / or avatar calls), discreet messaging, 6DoF portrait / landscape gaming, AR / VR object manipulation, AR / VR content display (e.g., presenting content via a virtual display), and / or other AR / VR interactions. The HIPD 8000 can perform the above operations alone or in combination with a wearable device (or other communication-coupled electronic device).

[0216] Figure 8B A block diagram of a computing system 8040 of the HIPD 8000 according to some embodiments is shown. The HIPD 8000 described in detail above can include one or more components shown in the computing system 8040 of the HIPD. The HIPD 8000 will be understood to include the components shown and described below for the computing system 8040 of the HIPD. In some embodiments, all or most of the components of the computing system 8040 of the HIPD are included in a single integrated circuit. Alternatively, in some embodiments, multiple components of the computing system 8040 of the HIPD are included in multiple communication-coupled integrated circuits.

[0217] The computing system 8040 of the HIPD may include: a processor (e.g., CPU 8077, GPU, and / or a CPU with integrated graphics); a controller 8075; a peripheral interface 8050 that includes one or more sensors 8051 and other peripheral devices; a power supply (e.g., a power system 8095); and a memory (e.g., memory 8078) that includes an operating system (e.g., operating system 8079), data (e.g., data 8088), one or more applications (e.g., application 8080), one or more modules (e.g., a communication interface module 8081, a graphics module 8082, a task and process management module 8083, an interoperability module 8084, an AR processing module 8085, and / or a data management module 8086). The computing system 8040 of the HIPD further includes a power system 8095 that includes a charger input and output 8096, a PMIC 8097, and a battery 8098, all of which are defined above.

[0218] In some embodiments, the peripheral interface 8050 may include one or more sensors 8051. The sensors 8051 may include sensors similar to those described above with reference to Figure 6B For example, the sensors 8051 may include an imaging sensor 8054, an (optional) EMG sensor 8056, an IMU sensor 8058, and a capacitive sensor 8060. In some embodiments, the sensors 8051 may include one or more pressure sensors 8052 for sensing pressure data, an altimeter 8053 for sensing the height of the HIPD 8000, a magnetometer 8055 for sensing a magnetic field, a depth sensor 8057 (or a time-of-flight sensor) for determining the difference between a camera and an object in an image, a position sensor 8059 (e.g., a flexible position sensor) for sensing a relative displacement or a change in position of a part of the HIPD 8000, a force sensor 8061 for sensing a force applied to a part of the HIPD 8000, and a light sensor 8062 (e.g., an ambient light sensor) for detecting the amount of light. The sensors 8051 may include Figure 8B one or more sensors not shown in

[0219] Similar to the peripheral devices described above with reference to Figure 8B the peripheral interface 8050 may further include an NFC component 8063, a GPS component 8064, an LTE component 8065, Wi-Fi and / or Bluetooth communication components 8066, a speaker 8069, a haptic device 8071, and a microphone 8073. As described above with reference to Figure 8AAs described above, the HIPD 8000 may optionally include a display 8068 and / or one or more buttons 8067. The peripheral interface 8050 may also include one or more cameras 8070, a touch surface 8072, and / or one or more light emitters 8074. The multi-touch input surface 8002 described above with reference to Figure 8A is an example of the touch surface 8072. The light emitter 8074 may be one or more LEDs, lasers, etc., and may be used to project or present information to the user. For example, the light emitter 8074 may include the light indicators 8012 and 8026 described above with reference to Figure 8A . The camera 8070 (e.g., the cameras 8014 and 8022 described above in Figure 8A ) may include one or more wide-angle cameras, fisheye cameras, spherical cameras, compound eye cameras (e.g., stereo and multi-cameras), depth cameras, RGB cameras, ToF cameras, RGB-D cameras (depth and ToF cameras), and / or other available cameras. The camera 8070 can be used for: SLAM; 6DoF ray casting, gaming, object manipulation, and / or other rendering; face recognition and facial expression recognition, etc.

[0220] Similar to the computing system 6060 of the watch body and the computing system 6030 of the watch band described above with reference to Figure 6B , the computing system 8040 of the HIPD may include one or more haptic controllers 8076 and associated components (e.g., haptic devices 8071), and the one or more haptic controllers and the associated components are used to provide haptic events at the HIPD 8000.

[0221] The memory 8078 may include high-speed random access memory and / or non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Access to the memory 8078 by other components of the HIPD 8000 (e.g., one or more processors and the peripheral interface 8050) may be controlled by the memory controller in the controller 8075.

[0222] In some embodiments, the software components stored in the memory 8078 include one or more operating systems 8079, one or more applications 8080, one or more communication interface modules 8081, one or more graphics modules 8082, and one or more data management modules 8086, and these software components are similar to the software components described above with reference to Figure 6B .

[0223] In some embodiments, the software components stored in the memory 8078 include a task and process management module 8083 that is configured to identify one or more front-end tasks and back-end tasks associated with an operation performed by a user, execute the one or more front-end tasks and / or back-end tasks, and / or provide instructions to one or more communicatively coupled devices that cause the one or more front-end tasks and / or back-end tasks to be executed. In some embodiments, the task and process management module 8083 uses data 8088 (e.g., device data 8090) to distribute one or more front-end and / or back-end tasks based on the computing resources, available power, thermal headroom, ongoing operations, and / or other factors of the communicatively coupled devices. For example, the task and process management module 8083 may cause one or more back-end tasks of an operation to be executed at the HIPD 8000 based on determining that the operation being performed at the communicatively coupled AR system 7000 is utilizing a predetermined amount (e.g., at least 70%) of the computing resources available at the AR system 7000.

[0224] In some embodiments, the software components stored in the memory 8078 include an interoperability module 8084 that is configured to exchange and utilize information received and / or provided to different communicatively coupled devices. The interoperability module 8084 allows different systems, devices, and / or applications to connect and communicate in a coordinated manner without user input. In some embodiments, the software components stored in the memory 8078 include an AR module 8085 that is configured to process signals based at least in part on sensor data for use in an AR environment and / or a VR environment. For example, the AR module 8085 can be used for 3D object manipulation, gesture recognition, and / or face recognition and facial expression recognition.

[0225] The memory 8078 may also include data 8088 that includes structured data. In some embodiments, the data 8088 includes profile data 8089, device data 8090 (including device data of one or more devices communicatively coupled to the HIPD 8000, such as device type, hardware, software, and / or configuration), sensor data 8091, media content data 8092, and application data 8093.

[0226] It should be recognized that the computing system 8040 of the HIPD is an example of a computing system within the HIPD 8000, and compared to the components shown in the computing system 8040 of the HIPD, the HIPD 8000 may have more or fewer components, two or more components may be combined, and / or may have different component configurations and / or arrangements. The various components shown in the computing system 8040 of the HIPD are implemented in hardware, software, firmware, or a combination thereof (including one or more signal processing and / or application specific integrated circuits).

[0227] The various techniques described above in Figure 8A and Figure 8B can be used with any device used as a human-machine interface controller. In some embodiments, the HIPD 8000 can be used in combination with one or more wearable devices (such as head wearable devices (e.g., AR system 7000 and VR system 7010) and / or wrist wearable devices 6000 (or components thereof)). In some embodiments, the HIPD 8000 is used in combination with wearable apparel (e.g., Figure 11 A to Figure 11 C wearable gloves). After thus describing the example HIPD 8000, attention will now be turned to an example feedback device, such as device 9000.

[0228] Example feedback device

[0229] Figure 9A and Figure 9B illustrate an example haptic feedback system (e.g., a hand wearable device) for providing feedback to a user regarding the user's interaction with a computing system (e.g., an artificial reality environment presented by an AR system 7000 or a VR system 7010). In some embodiments, the computing system (e.g., AR system 5000d) can also provide feedback to one or more users based on actions performed within the computing system and / or interactions provided by the AR system (e.g., the actions and / or interactions can be based on instructions that are executed in conjunction with the operation of an application of the computing system). Such feedback can include visual feedback and / or audio feedback, and can also include haptic feedback provided through haptic components (e.g., one or more haptic components 9062 of device 9000 (e.g., haptic components 9062-1, 9062-2, and 9062-3)). For example, the haptic feedback can prevent one or more fingers of the user from bending past a certain point (or at least impede / resist the action of one or more fingers of the user from bending past a certain point) to simulate the feeling of touching a solid coffee cup. When driving such haptic effects, device 9000 can (directly or indirectly) change the pressurization state of one or more of the plurality of haptic components 9062.

[0230] Each of the plurality of haptic components 9062 includes means that provides at least resistance when the respective haptic component 9062 transitions from a first pressurized state (e.g., atmospheric pressure or deflated) to a second pressurized state (e.g., inflated to a threshold pressure). The structure of the haptic component 9062 can be integrated into a variety of devices configured to contact or be proximate to a user's skin, including but not limited to devices such as glove-worn devices, body-worn apparel devices, and head-mounted devices.

[0231] As described above, the haptic components 9062 described herein can be configured to transition between a first pressurized state and a second pressurized state to provide haptic feedback to a user. Due to the ever-changing nature of artificial reality, during a single use, the haptic components 9062 may need to transition between the two states hundreds or possibly thousands of times. Thus, the haptic components 9062 described herein are durable and designed to transition quickly from one state to another. To provide some context, in the first pressurized state, the haptic component 9062 does not impede the free movement of a portion of the wearer's body. For example, one or more haptic components 9062 incorporated into a glove are made of a flexible material (e.g., an electrostatic-zipping actuator) that does not impede the free movement of the wearer's hand and fingers. The haptic component 9062 is configured to conform to the shape of a portion of the wearer's body when in the first pressurized state. However, once in the second pressurized state, the haptic component 9062 can be configured to restrict and / or impede the free movement of that portion of the wearer's body (e.g., appendages of the user's hand). For example, when the haptic component 9062 is in the second pressurized state, the respective haptic component 9062 (or respective haptic components) can restrict the movement of the wearer's fingers (e.g., prevent finger curling or extension). Additionally, once in the second pressurized state, the haptic component 9062 can adopt different shapes, with some haptic components 9062 configured to adopt a planar, rigid shape (e.g., planar and rigid), while some other haptic components 9062 are configured to be at least partially curved or bent.

[0232] As a non-limiting example, device 9000 includes a plurality of haptic devices (e.g., a pair of haptic gloves, and a wrist-worn device (e.g., refer to Figure 6A and Figure 6BThe haptic components of any of the described wrist-wearable devices), each of the plurality of haptic devices may include a garment component (e.g., garment 9004) and one or more haptic components coupled (e.g., physically coupled) to the garment component. For example, each of the haptic components 9062-1, 9062-2, 9062-3 is physically coupled to the garment 9004, and the garment 9004 is configured to contact the corresponding phalanges of the user's thumb and fingers. As described above, the haptic component 9062 is configured to provide a haptic simulation to the wearer of the device 9000. The garment 9004 of each device 9000 may be one of various types of garments (e.g., gloves, socks, shirts, or pants). Thus, a user may wear multiple devices 9000, each of which is configured to provide haptic stimulation to the corresponding part of the body where the device 9000 is worn.

[0233] Figure 9C FIG. shows a block diagram of a computing system 9040 of a device 9000 according to some embodiments. The computing system 9040 may include one or more peripheral interfaces 9050, one or more power systems 9095, one or more controllers 9075 (including one or more haptic controllers 9076), one or more processors 9077 (as defined above, including any of the examples provided), and a memory 9078, all of which may communicate electronically with each other. For example, one or more processors 9077 may be configured to execute instructions stored in the memory 9078, and the instructions may cause a controller in one or more controllers 9075 to cause a plurality of operations to be performed at one or more peripheral devices of the peripheral interface 9050. In some embodiments, each of the described operations may be performed based on power provided by the power system 9095. The power system 9095 includes a charger input 9096, a PMIC 9097, and a battery 9098.

[0234] In some embodiments, the peripheral interface 9050 may include one or more devices configured as part of the computing system 9040, and many of the one or more devices have been defined and / or referenced above Figure 6A and Figure 6B described in the wrist-wearable device shown in. For example, the peripheral interface 9050 may include one or more sensors 9051. Some example sensors include: one or more pressure sensors 9052, one or more EMG sensors 9056, one or more IMU sensors 9058, one or more position sensors 9059, one or more capacitive sensors 9060, one or more force sensors 9061; and / or any other type of sensor defined above or described in connection with any other embodiment discussed herein.

[0235] In some embodiments, the peripheral interface may include one or more additional peripheral devices, the one or more additional peripheral devices including one or more Wi-Fi and / or Bluetooth devices 9068; one or more haptic components 9062; one or more support structures 9063 (which may include one or more bladders 9064; one or more manifolds 9065; one or more pressure-changing devices 9067; and / or any other type of peripheral device defined above or described with respect to any other embodiment discussed herein).

[0236] In some embodiments, each haptic component 9062 includes a support structure 9063 and at least one bladder 9064. The bladder 9064 (e.g., a membrane) is a sealed inflatable pocket made of a durable and puncture-resistant material (e.g., thermoplastic polyurethane (TPU) or a flexible polymer, etc.). The bladder 9064 contains a medium (e.g., a fluid such as air, an inert gas, or even a liquid) that can be added to or removed from the bladder 9064 to change the pressure (e.g., fluid pressure) inside the bladder 9064. The support structure 9063 is made of a material that is stronger and stiffer than the material of the bladder 9064. The corresponding support structure 9063 coupled to the corresponding bladder 9064 is configured to reinforce the corresponding bladder 9064 as the corresponding bladder changes shape and size due to a change in the pressure (e.g., fluid pressure) inside the bladder.

[0237] Device 9000 also includes a haptic controller 9076 and a pressure-changing device 9067. In some embodiments, the haptic controller 9076 is part of the computer system 9040 (e.g., a part of the computer system 9040 that is in electronic communication with one or more processors 9077 of the computer system 9040). The haptic controller 9076 is configured to control the operation of the pressure-changing device 9067 and, in turn, control the operation of the device 9000. For example, the controller 9076 sends one or more signals to the pressure-changing device 9067 to activate the pressure-changing device 9067 (e.g., turn the pressure-changing device 9067 on and off). One or more signals may specify the desired pressure (e.g., pounds per square inch) to be output by the pressure-changing device 9067. The generation of the one or more signals and, in turn, the pressure output by the pressure-changing device 9067 may be based on information collected by Figure 5A and Figure 5B the sensors in Figure 5A and Figure 5BThe information collected by the sensors therein (e.g., the user touches the artificial coffee cup) causes the pressure changing device 9067 to increase the pressure (e.g., fluid pressure) within the haptic component 9062 at a first time. Then, the controller can send one or more additional signals to the pressure changing device 9067 based on the additional information collected by the sensor 9051, and the one or more additional signals cause the pressure changing device 9067 to further increase the pressure within the haptic component 9062 at a second time after the first time. Additionally, one or more signals can cause the pressure changing device 9067 to inflate one or more bladders 9064 in the device 9000 - A, while the one or more bladders 9064 in the device 9000 - B remain unchanged. Further, one or more signals can cause the pressure changing device 9067 to inflate one or more bladders 9064 in the device 9000 - A to a first pressure and inflate one or more other bladders 9064 in the device 9000 - A to a second pressure different from the first pressure. Depending on the number of devices 9000 served by the pressure changing device 9067 and the number of bladders in the device 9000, many different inflation configurations can be achieved by one or more signals, and the examples above are not meant to be limiting.

[0238] The device 9000 may include an optional manifold 9065 located between the pressure changing device 9067 and the device 9000. The manifold 9065 may include one or more valves (not shown), and the one or more valves pneumatically couple each haptic component in the haptic component 9062 to the pressure changing device 9067 through pipes. In some embodiments, the manifold 9065 communicates with the controller 9075, and the controller 9075 controls one or more valves of the manifold 9065 (e.g., the controller generates one or more control signals). The manifold 9065 is configured to switchably couple the pressure changing device 9067 to one or more haptic components 9062 of the same or different devices 9000 based on one or more control signals from the controller 9075. In some embodiments, the device 9000 may include multiple pressure changing devices 9067, where each pressure changing device 9067 is directly pneumatically coupled to a single (or multiple) haptic component 9062 instead of using the manifold 9065 to pneumatically couple the pressure changing device 9067 to the haptic component 9062. In some embodiments, the pressure changing device 9067 and the optional manifold 9065 are configured as part of one or more devices 9000 (not shown), while in other embodiments, the pressure changing device 9067 and the optional manifold 9065 are configured to be external to the device 9000. A single pressure changing device 9067 can be shared by multiple devices 9000.

[0239] In some embodiments, the pressure changing device 9067 is a pneumatic device, a hydraulic device, a pneumatic-hydraulic device, or some other device capable of adding a medium (e.g., a fluid, a liquid, a gas) and removing the medium from one or more haptic components 9062.

[0240] Figures 9A to 9C The devices shown in may be coupled via a wired connection (e.g., via a bus). Alternatively, Figures 9A to 9C one or more of the multiple devices shown in may be wirelessly connected (e.g., via a short-range communication signal).

[0241] The memory 9078 includes instructions and data, some or all of which may be stored in the memory 9078 as a non-transitory computer-readable storage medium. For example, the memory 9078 may include: one or more operating systems 9079; one or more communication interface applications 9081; one or more interoperability modules 9084; one or more AR processing modules 9085; one or more data management modules 9086; and / or any other type of data defined above or described with respect to any other embodiment discussed herein.

[0242] The memory 9078 also includes data 9088 that may be used in conjunction with one or more of the above applications. The data 9088 may include: device data 9090; sensor data 9091; and / or any other type of data defined above or described with respect to any other embodiment discussed herein.

[0243] Having thus described the system block diagram and then described the example devices, attention is now turned to certain example embodiments.

[0244] Example Embodiments

[0245] Figure 10 Shows a flowchart of a method 900 for determining an adaptive adjustment to a physical activity being performed by a user wearing a wearable electronic device, according to some embodiments. The method 900 is executed at a computing system (e.g., a wearable device or an intermediate device) having one or more processors and a memory. In some embodiments, the memory stores one or more programs that are configured to be executed by the one or more processors. At least some of the multiple operations shown in FIG. 8 correspond to instructions stored in a computer memory or a computer-readable storage medium (e.g., the memory 6080 of the computer system 6060, or the wearable device 304). In some embodiments, the computing system is a wearable device such as the wrist-worn wearable device 102. In some embodiments, the computing system is or includes an intermediate device such as a smartphone, a personal computer, or a video game console.

[0246] In some embodiments, in combination with detecting, by one or more sensors located at a wearable electronic device (e.g., a wrist wearable device, an arm wearable device, a leg wearable device, a head wearable device, etc.), that a user is performing a physical activity at a particular activity rate, some or all of the operations discussed in reference method 900 are performed.

[0247] (A1) Method 900 includes: detecting (902), by one or more sensors located at a wearable electronic device (e.g., wrist wearable device 102), that a user is performing a physical activity at a particular activity rate. For example, in Figure 1A it, the wrist wearable device indicates via user interface 107 that user 101 is performing a bicep curl. The display 104 of the wrist wearable device 102 also indicates (via force indicator 118) that user 101 has a reference level of force. In some embodiments, the detection that the user is performing a physical activity is based on one or more neuromuscular signal sensors (e.g., neuromuscular signal sensor 192), which may be the same neuromuscular signal sensors as described in connection with other operations of reference method 900 below. In some embodiments, the detection is performed continuously (or periodically) while the user is performing the physical activity. In some embodiments, a first subset of sensors is used continuously, and other sensors are powered based on data detected by the first subset. For example, in Figure 1B it, chart 193 indicates that sensor channels 199-1 and 199-4 are used to detect the force of user 101 while performing a bicep curl.

[0248] Method 900 includes: detecting (904) the level of force of the user using a neuromuscular signal sensor (e.g., a biopotential signal sensor, an EMG sensor, which may include individual sensor channels (e.g., sensor channels 199-1 and 199-4)) located at and / or otherwise communicatively coupled to the wearable electronic device.

[0249] Method 900 includes: determining (906), based on determining that the level of force differs from the reference level of force by at least a threshold amount (e.g., a peak execution threshold value based on historical user data), an adjustment to the particular activity rate while the user is performing the physical activity (e.g., by adjusting the number of repetitions to be performed and / or the duration of performing the physical activity). For example, in Figure 1C it, the display 104 of the wrist wearable device 102 includes user interface elements that indicate that user 101 has a lower-than-normal level of force and recommends that the user perform more repetitions during the next set of physical activities.

[0250] (A2)In some embodiments of A1, method 900 includes: based on (908) determining a second exertion level that differs from a baseline exertion level by more than a threshold amount, causing one or more operations of the electronic fitness equipment being used by the user to stop (e.g., stopping a timed workout). For example, in Figure 3H wherein, the user interface element 364 indicates that the electronic fitness equipment 306 is stopped based on the user having an exertion level higher than the second exertion level.

[0251] (A3)In some embodiments of A1 or A2, method 900 includes: before determining that the exertion level differs from the baseline exertion level by at least a threshold amount: (i) separately from detecting the exertion level, using a neuromuscular signal sensor to detect another exertion level, and (ii) based on determining that the other exertion level differs from the baseline exertion level by at least another threshold amount, providing a notification to the user. For example, in Figure 3G wherein, based on determining that the user's exertion level is higher than the threshold exertion level, the user interface presents a notification 350 that the user is performing a physical activity at a high exertion level, and the user interface includes a selectable button 351 that allows the user to adjust the activity rate at the electronic fitness equipment 306.

[0252] In some embodiments, the difference between the other threshold amount and the baseline exertion level is less than the difference between the exertion level and the baseline exertion level. In some embodiments, the other threshold amount indicates that the period during which the user's exertion level exceeds the baseline exertion level is shorter than the period indicated by the exertion level.

[0253] In some embodiments, the notification includes information about the exertion level and / or adjustment information (e.g., "abnormally high muscle exertion detected"). In some embodiments, the notification is an audio notification provided via a speaker of a wearable electronic device. In some embodiments, the notification is presented via a display at the wearable electronic device. In some embodiments, the notification is presented at a head-mounted device different from the wearable electronic device. In some embodiments, the notification is presented at an exercise device associated with the physical activity.

[0254] (A4)In some embodiments of A2 or A3, causing the notification to be provided at an electronic exercise device (e.g., an electronic fitness device), and the notification includes selectable user interface elements (e.g., including one or more selectable user interface elements for reducing resistance, speed, repetitions, execution goals). In response to the user selecting the selectable user interface element, (e.g., by the wearable electronic device via an electronic communication signal) causing the electronic exercise device (e.g., via an actuating component, in real time, immediately, iteratively at discrete intervals) to adjust the activity rate. In some embodiments, the electronic exercise device communicates electronically with one or more of the user's wearable electronic devices.

[0255] (A5) In some embodiments of any one of A2 through A4, method 900 includes: after determining that the other exertion level differs from the baseline exertion level by at least another threshold amount, and before determining that the exertion level differs from the baseline exertion level by at least the threshold: (i) separately from detecting the exertion level and detecting the other exertion level, detecting yet another exertion level using a neuromuscular signal sensor among one or more sensors located at the wearable electronic device, and (ii) based on determining that the additional exertion level differs from the baseline exertion level by yet another threshold amount, determining another adjustment to a particular activity rate, the other adjustment being different from the adjustment. For example, in Figure 1F and Figure 1G the wrist-worn device 102 presents additional user interface elements different from the user interface elements shown in Figure 1C and Figure 1D These additional user interface elements indicate that user 101 is still performing the physical activity at a below-normal exertion level of the physical activity and suggest another adjustment to the physical activity.

[0256] In some embodiments, the adjustment decreases the activity rate, and the other adjustment stops the execution of the physical activity.

[0257] (A6) In some embodiments of A5, (i) the other adjustment to a particular activity rate is to decrease the activity rate, and (ii) the adjustment to a particular activity rate is to stop the physical activity (e.g., stop Figure 3H the electronic fitness equipment 306 in

[0258] (A7) In some embodiments of any one of A1 through A6: (i) the user performs a physical activity on an electronic exercise device, and (ii) determining that the exertion level is different from the baseline exertion level is further based on data from the electronic exercise device (e.g., Figures 3A to 3I the electronic fitness equipment 306 in

[0259] (A8) In some embodiments of A7, the data from the electronic exercise device is generated by one or more of the following: (i) a PPG sensing device; (ii) an electrocardiogram (ECG) sensing device; and (iii) a gyroscope sensor. In some embodiments, the wrist-worn device can determine that the sensors at the electronic exercise device are more accurate for detecting a particular aspect of the physical activity the user is performing, and can determine that the corresponding sensors of the electronic exercise device should be used instead of the corresponding sensors of the wrist-worn device.

[0260] (A9) In some embodiments of any one of A1 to A8: (i) one or more sensors at the wearable electronic device include an inertial measurement unit (IMU) sensor, and (ii) determining that the degree of exertion is different from a reference degree of exertion is further based on data from the IMU sensor. In some embodiments, the IMU sensor detects movement differences of the user's body, where the movement differences correspond to the physical activity being performed. For example, the IMU sensor can determine that the user sways back and forth when performing a set of bicep curls, or arches their back when performing repetitions of a bench press, and indications of these movements can be used to determine the corresponding degree of exertion.

[0261] (A10) In some embodiments of A9, determining that the degree of exertion is different from a reference degree of exertion further includes: (i) applying a first weight to data from the neuromuscular signal sensor, and (ii) applying a second weight to data from the IMU sensor. In some embodiments, the first weight is based on whether the neuromuscular signal sensor is detecting the primary muscle groups of the physical activity being performed. For example, Figure 1B neuromuscular signal sensor channels 199-1 and 199-4 in can correspond to one or more primary muscle groups of the bicep curls being performed by user 101, while Figure 3C neuromuscular signal sensor channels 199-2 and 199-4 in may not correspond to any primary muscle groups of the rowing exercise being performed by the user. Thus, compared to IMU sensor 194, Figure 1B neuromuscular signal sensor channels 199-1 and 199-4 in have a higher weight than Figure 3C neuromuscular signal sensor channels 199-2 and 199-4 in.

[0262] (A11) In some embodiments of A10, the first weight and the second weight are based on a calibration performed by the user prior to performing the physical activity at the activity rate. For example, the user can receive an indication of one or more actions for performing the corresponding activity prior to performing the physical activity (e.g., by bending the arm to simulate the movement of a bicep curl). In some embodiments, the calibration is performed without the user performing any gestures related to the actual physical activity (e.g., one or more of a gentle squeeze, a medium squeeze, and a hard squeeze of the user's palm). For example, to calibrate Figures 1A to 1MFor the neuromuscular signal sensor for bicep curls shown in [figure reference], the wrist-worn device 102 can instruct the user to perform bicep flexion at various intensities. In some embodiments, based on a set of predefined criteria, a notification for recommended calibration is provided to the user when it is recognized that the user is performing a specific activity and there are no further instructions from the user. In some embodiments, an indication is provided to the user to calibrate one or more sensors of the wrist-worn device based on the amount of time since the last calibration of the sensors. For example, based on the fact that five weeks have passed since the previous calibration of the wrist-worn device 102, Figure 2A The selectable button 214 in [figure reference] indicates that recalibration is recommended at the wrist-worn device 102.

[0263] (A12) In some embodiments of either A10 or A11, the first weight and the second weight are based on the type of physical activity being performed. For example, a first weight can be applied to the neuromuscular signal sensor for bench press exercises, while a different first weight can be applied to the neuromuscular signal sensor for rowing exercises, where each IMU sensor can be capable of having a different relative accuracy. In some embodiments, the wrist-worn device provides the user with one or more indications related to the accuracy of the wrist-worn device for detecting specific aspects of the physical activity the user is performing. For example, the repetition indicator 116 and the exertion indicator 118 of the wrist-worn device 102 indicate the accuracy of the wrist-worn device 102 for detecting various aspects of the physical activity performed in each sequence.

[0264] (A13) In some embodiments of A12, according to determining that the neuromuscular signal sensor is configured to sense the primary muscle group associated with the physical activity (e.g., the muscle group that is actively performing the function of the physical activity) (e.g., located at the primary muscle group), the first weight is higher than the second weight. In some embodiments, the determination of which muscle group among multiple muscle groups is involved in a given physical activity corresponds to: the neuromuscular signal sensor channel having the highest activity level during at least a portion of the physical activity (e.g., based on Figure 1B sensor channel 199-1 in [figure reference] for detecting the bicep activity of the user 101, and this sensor channel 199-1 can have the highest activity level). In some cases, there can be multiple sub-activities associated with the physical activity (e.g., pushing the barbell up and letting the barbell return down).

[0265] (A14)In some embodiments of any one of A1 to A13, (i) the activity rate is a first activity rate, (ii) the exertion level is lower than a reference exertion level by a threshold amount, and (iii) automatically increase the first activity rate (e.g., the number of repetitions, the resistance associated with the physical activity, the activity time of the physical activity) to a second activity rate, which is higher than the first activity rate (e.g., without further instructions from the user). For example, if the electronic fitness equipment 306 detects that the user 301 is performing a rowing activity with a lower-than-normal exertion level, the electronic fitness equipment 306 can be configured to automatically increase the resistance level during the rowing exercise.

[0266] (A15)In some embodiments of any one of A1 to A14: (i) the exertion level is higher than a reference exertion level by a threshold amount; (ii) the adjustment causes the electronic exercise device to stop the operation of causing the physical activity (e.g., adjust the physical activity rate to zero). In some embodiments, the stop is automatic and does not require further instructions from the user.

[0267] (A16)In some embodiments of any one of A1 to A15, the user wears a head-worn electronic device (e.g., Figures 3A to 3H the head-worn device 304 shown). The head-worn electronic device is presenting a user interface to the user, and the user interface corresponds to an interactive exercise (e.g., presenting timing instructions or repetition instructions corresponding to the physical activity being performed). For example, the user interface 310 includes various user interface elements related to the user configuration of the physical activity to be performed. Based on determining that the exertion level differs from the reference exertion level by a threshold amount, the adjustment of the physical activity includes: causing an adjustment to the user interface presented by the head-worn electronic device. In some embodiments, the notification provided to the user via the user interface includes adjustment information, exertion information, suggestions, and / or adjustment mechanisms.

[0268] (A17)In some embodiments of any one of A1 to A16, determining that the exertion level differs from the reference exertion level by a threshold amount is further based on the number of identified repetitions of the physical activity that the user has performed at the activity rate. For example, the threshold amount can be changed based on the number of repetitions of the physical activity that the user has performed to account for the level of fatigue that the user should reach. In some embodiments, the reference level is based on the number of repetitions and / or based on historical information about the user's repetitions of the physical activity and the corresponding exertion levels. For example, based on the fact that the user has performed another physical activity (e.g., bicep curl) as Figures 1A to 1M shown, the exertion level detected in Figures 2A to 2E can be compared with the increased exertion level.

[0269] (A18)In some embodiments of any one of A1 to A17, determining that the degree of exertion differs from a baseline degree of exertion by a threshold amount is further based on the number of recognized repetitions of another physical activity that the user has performed, the other physical activity being different from the physical activity that the user is currently performing. For example, if the user performed a weightlifting activity before performing a jogging activity, the threshold amount for the jogging activity may be different. As another example, based on Figures 2A to 2E User 101 in

[0270] (A19)In some embodiments of any one of A1 to A18, the threshold amount (and / or baseline level) is based on a plurality of context criteria associated with the user. In some embodiments, the threshold amount (and / or baseline level) is determined based on inputting one or more of the plurality of context criteria into a machine learning model. In some embodiments, data from two or more neuromuscular signal channels is used as a feature vector for training the machine learning model. In some embodiments, one feature vector is EMG data and another feature vector is IMU data. In some embodiments, one or more additional feature vectors are generated based on a combination of two or more different types of sensors. For example, based on ​ the user's performance of bicep curls, the machine learning model may include one or more feature vectors based on a combination of neuromuscular signal sensor channels 199-1 and 199-4.

[0271] (A20)In some embodiments of any one of A1 to A19, the neuromuscular signal sensor corresponds to one of a plurality of neuromuscular signal sensor channels (e.g., two to ten neuromuscular signal sensor channels, each of which may include one or more electrodes for detecting neuromuscular signals). Before detecting the user's degree of exertion, the neuromuscular signal sensor is selected from the plurality of neuromuscular signal sensor channels based on a determination of the type of activity that the user is performing (e.g., as a power saving technique).

[0272] (A21)In some embodiments of any one of A1 to A20, adjusting a particular activity rate includes providing the user with the following indication: stop performing the physical activity until the user's exertion level has decreased towards the resting baseline exertion level. In some embodiments of A20, method 900 further includes: after providing the user with the indication to stop performing the physical activity, detecting another exertion level of the user using data from the neuromuscular signal sensor. In some embodiments of A20, method 900 includes: based on determining that the other exertion level is closer to the resting baseline exertion level, providing the user with an indication to continue performing the physical activity. For example, after the user completes Figure 1B the first set of bicep curls shown, the user 101 may be provided with an indication that their exertion level has dropped to the baseline level, and a corresponding indication may be provided to indicate that the user may perform another set of bicep curls based on having the baseline (e.g., resting) exertion level.

[0273] (A22)In some embodiments of A21, determining that the other exertion level is closer to the resting baseline exertion level includes: determining the signal-to-noise ratio of the data detected by the neuromuscular signal sensor. For example, the determination may include: comparing the data corresponding to the new exertion level from the neuromuscular signal sensor with the noise floor, which represents the sum of all noise sources and unwanted signals within the measurement system.

[0274] (A23)In some embodiments of any one of A1 to A22, method 900 further includes: after detecting the user's exertion level: (i) detecting involuntary movements of the user using data from the neuromuscular signal sensor. In some embodiments, the system first uses the neuromuscular signal sensor located at the wearable electronic device to determine a baseline level of the user's muscle behavior, which includes postures that can be recognized by a trained artificial intelligence model (e.g., a machine learning model, a neural network, a generative model).

[0275] Figure 11 FIG. shows a flowchart of a method 1000 for determining the repetitions of a physical activity being performed by a user according to some embodiments. Method 1000 is executed at a computing system (e.g., a wearable device or an intermediate device) having one or more processors and a memory (e.g., one or more processors 8077 and memory 8078 of HIPD8000 in system 8040). In some embodiments, the memory stores one or more programs that are configured to be executed by the one or more processors. Figure 11At least some of the multiple operations shown correspond to instructions stored in a computer memory or a computer-readable storage medium (e.g., the memory 8078 of the HIPD 8000 or the wearable devices 6000 and 7000). In some embodiments, the computing system is a wearable device such as the wrist wearable device 102. In some embodiments, the computing system is or includes an intermediate device such as a smartphone, a personal computer, or a video game console. In some embodiments, the method 1000 is executed at a wearable electronic device having one or more sensors, the one or more sensors including neuromuscular signal sensors (e.g., biopotential signal sensors, EMG sensors, PPG sensors).

[0276] (B1) The method 1000 includes: detecting (1002) a physical activity being performed by a user using one or more sensors. For example, in Figure 2A , the user interface element 208 includes an activity indicator indicating that the user is performing a bench press.

[0277] The method 1000 includes: identifying (1004) one or more repetitions of the physical activity using data from a neuromuscular signal sensor based on the detected physical activity being performed (e.g., one repetition, five repetitions, a set of repetitions). For example, in Figure 2C , the repetition indicator 116 indicates that the user 101 has performed seven repetitions of the bench press.

[0278] Identifying one or more repetitions includes identifying: (i) a first level (1006) associated with a local maximum force of the physical activity, or (ii) a second level (1008) associated with a local minimum force of the physical activity. For example, in Figure 1B , the wrist wearable device 102 detects a local minimum force corresponding to the user's physical activity at time t0 and detects a local maximum of the user's physical activity at time t1.

[0279] The method 1000 includes: obtaining (1010) the number of the one or more repetitions to produce a count of the identified repetitions of the physical activity being performed. In some embodiments, the neuromuscular signal sensor is separate from but communicates with the wearable electronic device. In some embodiments, the local maximum is different from the absolute maximum (e.g., peak force) associated with the physical activity. For example, the degree of force detected by the neuromuscular signal sensor 199-4 at time points t0 to t3 in Figure 1B can be lower than the degree of force detected at time points t4 to t7 in Figure 1E , but the wrist wearable device can still detect a local maximum during the time period between t0 and t3.

[0280] (B2) In some embodiments of B1, obtaining the number of one or more repetitions includes identifying the number of times that a neuromuscular signal sensor detects the following items when the user is performing a physical activity: (i) a period of exertion corresponding to a first level, and (ii) another period of exertion corresponding to a second level. For example, Figure 1C the number of repetitions indicated by the repetition indicator 116 in Figure 1B can be based on the number of times local minima and local maxima as shown in

[0281] (B3) In some embodiments of B1 or B2, method 1000 further includes: (i) detecting, via an IMU sensor, a motion corresponding to one repetition, and (ii) abandoning including the repetition in one or more repetitions based on a failure to detect at least one of a first level associated with a local maximum exertion of the activity and a local minimum exertion of the activity. That is, the repetition is not included in the repetition count because the appropriate neuromuscular level is not reached. In this way, the neuromuscular signal sensor is used to distinguish repetitions of a physical activity (e.g., bicep curl) from similar motions (e.g., wiping sweat from the forehead). For example, in Figure 1L and Figure 1M , the user first performs a wiping gesture using a towel, which has motion characteristics similar to repetitions of a bicep curl, but the repetition count is not incremented because the motion does not include local minima and local maxima of the activity.

[0282] (B4) In some embodiments of any one of B1 to B3, (i) the neuromuscular signal sensor corresponds to one neuromuscular signal sensor channel among a plurality of neuromuscular signal sensor channels, and (ii) the neuromuscular signal sensor is selected to detect a physical activity being performed based on identifying the type of physical activity being performed. In some embodiments, the plurality of neuromuscular signal sensor channels includes one or more channels that are not continuously activated. For example, when the user detects the type of physical activity being performed, the wearable electronic device causes one or more (but not all) of the neuromuscular signal sensor channels among the plurality of neuromuscular signal sensor channels to be activated. For example, Figure 1B shows that sensor channels 199-1 and 199-4 are activated for a first activity performed by user 101, Figure 2B shows that sensor channels 199-5 and 199-6 are activated for the performance of a second activity of the user.

[0283] (B5)In some embodiments of any one of B1 through B4, method 1000 further includes: based on determining that the user has stopped performing the physical activity for at least a threshold amount of time: (i) storing a count of the identified repetitions of the physical activity (e.g., at a wrist-wearable device), and (ii) after storing the count, beginning a different count of subsequent identified repetitions of the physical activity. For example, when it is detected that the user has stopped performing the activity, the number of identified repetitions is stored as a set performed by the user, and a new set is begun to be counted. In some embodiments, determining that the user has stopped performing the physical activity includes: detecting that the user's exertion level is below a threshold level for at least a threshold amount of time (e.g., Figure 2C in which the user interface element 230 in Figure 2C indicates that user 101 has completed a set of bench presses).

[0284] (B6)In some embodiments of B5, determining that the user has stopped performing the physical activity for at least a threshold amount of time includes: determining that the user's relaxation level is within a preset threshold of a baseline relaxation level (e.g., a resting state). In some embodiments, the baseline level corresponds to the user performing a low-intensity physical activity (e.g., a cooldown workout, a walk). In some embodiments, the baseline exertion level corresponds to an exertion level detected by fewer than all of the neuromuscular signal sensor channels of the wearable electronic device.

[0285] (B7)In some embodiments of B5 or B6, method 1000 further includes: after storing, based on detecting that the user is performing the physical activity: (i) identifying a third level associated with another local maximum exertion of the physical activity, and (ii) identifying a fourth level associated with another local minimum exertion of the physical activity. For example, a second set of repetitions is detected, and the exertion difference between the two sets is determined. For example, a set of bicep curls corresponding to time points t4 through t7 may have a higher exertion level than a set of bicep curls corresponding to time points t0 through t3, such that time t0 through t3 may include a first exertion level corresponding to a local maximum exertion that is different from a third exertion level corresponding to the local maximum exertion at time t4 through t7.

[0286] (B8)In some embodiments of any one of B1 through B7: (i) presenting the number of repetitions to the user while the user is performing the physical activity, and (ii) providing an indication based on determining that the user has performed a threshold number of the identified repetitions. For example, the user may configure the wrist-wearable device to indicate that the user has performed the tenth repetition of a particular activity in a set, where the user intended to perform that many repetitions.

[0287] (B9) In some embodiments of any one of B1 to B8, identifying a repetition of a body activity being performed includes: aggregating data from a neuromuscular signal sensor and data from an IMU sensor. In some embodiments, in combination with aggregating data from a neuromuscular signal sensor and data from an IMU sensor, a sensor fusion algorithm is used.

[0288] (B10) In some embodiments of B9, the data from the neuromuscular signal sensor is preprocessed (e.g., reduced, weighted by channel) before being aggregated with the data from the IMU sensor. In some embodiments, preprocessing the neuromuscular signal sensor data includes selecting one or more neuromuscular signal sensor channels from a plurality of neuromuscular signal sensor channels, where each of these neuromuscular signal sensor channels may correspond to at least one or more neuromuscular signal sensors corresponding to the neuromuscular signal sensor and / or a plurality of neuromuscular signal sensors. In some embodiments, different weights are applied to each of the plurality of neuromuscular signal sensor channels. In some embodiments, the preprocessing includes amplifying the respective signals from one or more neuromuscular signal sensors associated with the respective neuromuscular signal sensor channels among the plurality of neuromuscular signal sensors.

[0289] (B11) In some embodiments of any one of B9 and B10: (i) aggregating includes: applying a first weight to the data from the neuromuscular signal sensor and a second weight to the data from the IMU sensor, and the first weight is based on whether the wearable electronic device is configured to sense a major muscle group of the user corresponding to the body activity (e.g., located at the major muscle group).

[0290] (B12) In some embodiments of B11, aggregating includes: applying a first set of weights at a first time of a duration associated with a repetition of the user performing a body activity, the first set of weights including a respective first weight for the data from the neuromuscular signal sensor and a respective second weight for the data from the IMU sensor. In some embodiments, aggregating includes: applying a second set of weights at a second time of the duration associated with the repetition of the user performing a body activity, the second time being different from the first time, the second set of weights being different from the first set of weights, and the second set of weights including another respective first weight for the data from the neuromuscular signal sensor and another respective second weight for the data from the IMU sensor. In some embodiments, the concept of applying different weights to respective sensors of the wearable electronic device is used for different neuromuscular signal sensor channels of the wearable electronic device (e.g., applying a first weight to a first channel, applying a second weight to a second channel, and / or reducing the power of a third channel).

[0291] In some embodiments of any one of B9 to B12: (i) aggregation is performed using a machine learning model, and (ii) the machine learning model is trained based on data collected from a user of the wearable electronic device. For example, detecting that user 101 is performing Figure 1A a bicep curl in can be based on data from previous instances of user 101 performing a bicep curl and / or other different physical activities.

[0292] In some embodiments of any one of B9 to B13, aggregation includes: (i) determining an estimate of future measurement results of the series of measurements based on previous measurement results of the series of measurements, (ii) determining a probability distribution of predicted sensor data for each of the plurality of sensors used for aggregation based on the estimate of the future measurement results, and (iii) adjusting the respective weights applied to each corresponding sensor of the plurality of sensors based on the correspondence between the data from the respective sensor and the probability distribution of the predicted sensor data. For example, the method can reduce the weight of data from sensors that are less likely to output accurate data based on the estimate of the future measurement results. For example, the corresponding sensor among the plurality of sensors may not be located on a part of the user's skin that experiences high-fidelity neuromuscular activity.

[0293] In some embodiments of any one of B1 to B14, method 1000 further includes: dynamically adjusting the first level or the second level based on detecting a change in the intensity of the physical activity performed by the user. In some embodiments, the dynamic adjustment includes adjusting the levels during the execution of the physical activity (e.g., before the user stops performing the activity).

[0294] In some embodiments of any one of B1 to B15, identifying a repetition of a physical activity includes: comparing the first level and / or the second level with historical data, where the historical data includes measurements from corresponding sensors of other wearable electronic devices worn by other users and / or the same user. In some embodiments, some or all of the training data is collected from sources other than the wearable electronic device and / or includes predictive data for improving the detection efficiency of aspects of the physical activity. In some embodiments, the training data includes sensor data that does not correspond to recognizable gestures performed by the user.

[0295] In some embodiments of B16, the historical data is selected for comparison based on identifying one or more major muscle groups corresponding to the type of physical activity being performed.

[0296] (B18)In some embodiments of any one of B1 to B17, method 1000 further includes: (i) determining an accuracy level of the detected one or more repetitions based on data from one or more sensors, and (ii) based on the accuracy level, providing an indication to the user to adjust the position of the wearable electronic device. In some embodiments, determining the accuracy level includes: determining the signal-to-noise ratio (SNR) of the repetitive data. For example, the repetitive indicator 116 displayed by the wrist wearable device 102 indicates the corresponding accuracy level of the wrist wearable device 102 for detecting the physical activities performed by the user 101 at different corresponding time points.

[0297] (B19)In some embodiments of any one of B1 to B18, method 1000 further includes: determining an accuracy level of the identified one or more repetitions based on data from one or more sensors. In some embodiments, based on the accuracy level, providing an indication to the user that a higher accuracy level is achieved by activating a device with one or more sensors at another location of the user's body; wherein the other location of the user's body corresponds to the major muscle groups of the physical activity. For example, based on detecting that the user 401 is performing Figure 4A the jogging activity in, the head wearable device 304 can provide an indication to the user that if the user activates one or more sensors of the ankle wearable device 406, the jogging activity will be detected at a higher accuracy level.

[0298] (B20)In some embodiments of any one of B1 to B19, detecting the physical activity being performed includes detecting one or more user movements. And method 1000 further includes: (i) determining whether the one or more user movements include involuntary user movements, and (ii) based on determining that the one or more user movements include involuntary user movements, providing a notification including information about the involuntary user movement. In some embodiments, determining whether the one or more user movements include involuntary user movements includes using a machine learning model (e.g., providing the one or more user movements to the machine learning model for analysis). For example, based on the wrist wearable device 102 detecting that the user 301 performed an involuntary gesture during the execution of the rowing activity, an alarm corresponding to Figure 3I the alarm gesture 372 performed in can be automatically initiated without the user 301 performing the alarm gesture 372.

[0299] (B21)In some embodiments of B20, method 1000 further includes: (i) identifying a health condition corresponding to the involuntary user movement, and (ii) in response to identifying the health condition, providing a notification with information about the health condition to a remote electronic device.

[0300] Figure 12 FIG. 1100 is a flow chart of a method for providing an alert to a remote device according to some embodiments. Method 1100 is executed at a computing system (e.g., a wearable device or an intermediate device) having one or more processors and a memory. In some embodiments, the memory stores one or more programs configured to be executed by the one or more processors. Figure 10 At least some of the operations shown in FIG. 1100 correspond to instructions stored in a computer memory or a computer-readable storage medium (e.g., memory 8078 of HIPD 8000 or wearable devices 6000 and 7000). In some embodiments, the computing system is a wearable device such as a wrist wearable device 102. In some embodiments, the computing system is or includes an intermediate device such as a smartphone, a personal computer, or a video game console. In some embodiments, method 1100 is executed at a wearable electronic device in communication with a neuromuscular signal sensor (e.g., a bioelectrical potential signal sensor, an EMG sensor).

[0301] (C1) Method 1100 includes: determining (1102) that the user is performing an alert gesture (e.g., Figure 2E alert gesture 252 performed in FIG. 252) based on data from a neuromuscular signal sensor.

[0302] Method 1100 includes: in response to (1104) the alert gesture and without further user input: (i) obtaining context information of the alert gesture (e.g., context data sensed by one or more electronic devices associated with and / or near the user) (e.g., the location where user 301 is performing a physical activity when user 301 performs Figure 3I alert gesture 372 in FIG. 372), and (ii) causing a notification and the context information to be sent to a remote device. In some embodiments, the notification is intended for a remote user. In some embodiments, the notification is intended for a system (e.g., an alert system). In some embodiments, the wearable electronic device is configured to create a wireless communication connection (e.g., cellular, Internet, satellite) with another electronic device (e.g., electronic fitness equipment 306, which may be configured to remind the staff of a gym facility of the location where user 301 is performing a physical activity).

[0303] Method 1100 includes: causing (1106) the notification and the context information to be sent to a remote device. For example, it may be notified to a remote server that user 301 has performed Figure 3Ithe alert gesture 372 therein and may optionally provide notification of one or more additional notifications to one or more additional remote servers and / or other electronic devices (e.g., personal electronic devices of one or more other users in the vicinity of the user who has performed the alert gesture).

[0304] (C2) In some embodiments of C1, method 1100 further includes: (1108) based on the alert gesture, activating peripheral devices for collecting context information including the following items: (i) a camera (e.g., a video camera of a mobile electronic device) and (ii) a microphone. In some embodiments, the camera and / or the microphone are components of a wearable electronic device. In some embodiments, the camera and / or the microphone are components of a head-mounted device different from the wearable device (e.g., data and / or other information collected by the head-mounted device 304 may be provided as context information based on the alert gesture 372 detected by the wrist-worn device 102 therein). In some embodiments, the camera and / or the microphone are components of the user's smartphone or smartwatch. Figure 3I the alert gesture 372 therein, one or more facility cameras may be in electronic communication with the electronic fitness equipment 306, and data from one or more of these facility cameras may be provided as context information together with information about the alert gesture 372.

[0305] (C3) In some embodiments of C2, the user does not wear a peripheral device (e.g., a surveillance camera in the room where the user is located when the user performs the alert gesture). For example, when the user performs Figure 3I the alert gesture 372 therein, one or more facility cameras may be in electronic communication with the electronic fitness equipment 306, and data from one or more of these facility cameras may be provided as context information together with information about the alert gesture 372.

[0306] (C4) In some embodiments of any of C1 to C3, the context information includes data from one or more sensors different from the neuromuscular signal sensor (e.g., a GPS sensor, an imaging sensor, an audio sensor, one or more accelerometers). For example, when the user 401 performs Figure 4D the alert gesture 434 therein, the audio sensor may be activated and / or data from the audio sensor may be provided. In some embodiments, additional context information may be provided based on the first context information provided in connection with the corresponding alert. For example, based on determining that a natural disaster is occurring based on the imaging data provided in the first context information provided via the alert gesture, additional data may be provided to, for example, determine the proximity of the user to the natural disaster and / or the intensity of the natural disaster.

[0307] (C5) In some embodiments of any of C1 to C4, the context information includes data from an imaging sensor corresponding to a camera that is in electronic communication (e.g., wireless communication or wired communication) with the wearable electronic device.

[0308] (C6) In some embodiments of C5, the context information includes aggregated data from an imaging sensor and one or more additional sensors that are different from the neuromuscular signal sensor and the imaging sensor.

[0309] (C7) In some embodiments of any one of C1 to C6, the context information includes information about the user's vital status (e.g., heart rate, pulse, oxygen level).

[0310] (C8) In some embodiments of any one of C1 to C7, the notification is automatically broadcast to one or more other electronic devices near the user.

[0311] (C9) In some embodiments of any one of C1 to C8, the one or more other electronic devices include devices corresponding to local authorities (e.g., law enforcement, fishing and gaming clubs, first responders, security personnel determined to be within a threshold distance of the user based on the location provided in the context information).

[0312] (C10) In some embodiments of any one of C1 to C9, the context information of the alert gesture includes the type of disaster that occurs near the user (e.g., in a shared space with the user or close to the user).

[0313] (C11) In some embodiments of any one of C1 to C10, the notification is provided to a remote server, and method 1100 includes: causing instructions to be provided to the user based on information sent in response to the notification from the remote server. In some embodiments, the information provided includes route information. In some embodiments, the information provided includes instructions to attempt to mitigate the hazard. In some embodiments, the information provided includes information about local authorities.

[0314] (C12) In some embodiments of C11, method 1100 includes: providing instructions to the user via a map application at one or more wrist-wearable devices and another communicatively coupled device (e.g., a smart phone, a head-wearable device) without further instructions from the user.

[0315] (C13) In some embodiments of C11 or C12, the instructions provided to the user include one or more navigation user interface elements presented at the head-wearable device (e.g., via an AR presentation system and / or a VR presentation system). For example, Figure 4E the navigation user interface element 440 shown therein indicates to the user a path for avoiding a natural disaster (e.g., a tornado), which is detected via the context information provided in combination with the alert gesture 434.

[0316] (C14)In some embodiments of C13, the navigation user interface element indicates to the user a path to a safe location (e.g., the nearest establishment providing customer access, or a building meeting one or more safety criteria for a particular natural disaster event). In some embodiments, the navigation user interface element is presented in an AR interface via a head-worn device (e.g., such that one or more navigation user interface elements are configured and arranged to appear on the ground in front of the user).

[0317] (C15)In some embodiments of any of C1 to C14, at least a portion of the alert gesture is performed via the tracked eye movement of the user, which is detected by an eye movement tracking module of a head-worn device worn by the user, and the eye movement tracking module is configured to measure the position of the pupil of the eye.

[0318] (C16)In some embodiments of any of C1 to C15, the context information of the alert gesture includes information about the eye movement and / or focus (e.g., an eye movement lookup table, information from a holographic illuminator configured to detect reflections from the eyes). In some embodiments, the information may include multiple reflections, where a subset of the multiple reflections is from the pupil of the eye, and another subset of the multiple reflections is from the sclera of the eye and / or another part of the eye.

[0319] (C17)In some embodiments of C16, method 1100 includes: determining the location of the disaster situation based on information about the eye movement and / or the focus of one or more cameras of the head-worn device and / or the electronic device worn by the user.

[0320] (C18)In some embodiments of any of C1 to C17, an audio notification (e.g., an alert signal, a warning, and / or a confirmation of the detected gesture) is provided by a wrist-worn device in combination with a notification provided to another electronic device.

[0321] (C19)In some embodiments of any of C1 to C18: (i) the alert gesture corresponds to one of a predefined set of operations, (ii) each operation in the predefined set of operations corresponds to a respective level of the emergency situation experienced by the user, and (iii) the context information includes an indication of the respective level of the emergency situation. For example, the user may perform a first gesture indicating a potential danger and a second gesture indicating an impending danger.

[0322] (C20)In some embodiments of any of C1 to C19, the context information of the alert gesture includes a determination of whether the level of the neuromuscular signal corresponds to an increased level of exertion of the user (e.g., compared to a baseline level of exertion corresponding to a resting state).

[0323] (C21)In some embodiments of any one of C1 to C20, the context information includes information about one or more user movements performed by the user. And the method further includes: (i) determining whether the one or more user movements include involuntary user movements, and (ii) based on determining that the one or more user movements include involuntary user movements, including information about the involuntary user movements in a notification (e.g., sending to another electronic device, another user, different devices of the same user; auditory alert / visual alert; broadcasting over a Wi-Fi signal).

[0324] (C22)In some embodiments of any one of C1 to C21, after a health condition is recognized, no further user input is required (e.g., the operation is automatic). For example, if the context information indicates that the user has performed an involuntary gesture (e.g., via an AI model that can optionally be instantiated and stored by a wrist-worn device), the wrist-worn device can automatically receive an alert corresponding to the alert gesture even if the user has not performed the alert gesture.

[0325] Figure 13 A flowchart of a method 1200 for detecting a relaxation state of a user in accordance with some embodiments is shown. Method 1200 is executed at a computing system (e.g., a wearable device or an intermediate device) having one or more processors and a memory. In some embodiments, the memory stores one or more programs that are configured to be executed by the one or more processors. Figure 13 At least some of the plurality of operations shown correspond to instructions stored in a computer memory or a computer-readable storage medium (e.g., the memory 8078 of the HIPD 8000, or the systems 6000 and 7000). In some embodiments, the computing system is a wearable device such as a wrist-worn device 102. In some embodiments, the computing system is or includes an intermediate device such as a smartphone, a personal computer, or a video game console. In some embodiments, method 1200 is executed in combination with the user wearing a wrist-worn device (e.g., during a physical activity).

[0326] (D1)Method 1200 includes: determining (1202) the tension level of a particular muscle group of the user using data from a neuromuscular signal sensor.

[0327] Method 1200 includes: providing (1204) a notification to the user indicating that the user is in a relaxation state based on determining that the tension level meets one or more predefined criteria.

[0328] Method 1200 includes: providing (1206) a notification to the user indicating that the user is not in a relaxation state based on determining that the tension level does not meet one or more predefined criteria.

[0329] (D2) In some embodiments of D1, method 1200 includes: additionally determining (1208) that the level of tension meets one or more predefined criteria such that a notification is provided to the user indicating that the user should wait to perform a repetition of a physical activity (e.g., a golf swing) until a reference level of exertion is detected.

[0330] (D3) In some embodiments of either D1 or D2, determining the level of tension is based on obtaining a request from the user or another user. In some embodiments, the level of tension is determined in response to a user request. In some embodiments, the request is a user input, a request automatically generated as part of an exercise or wellness program, or a request automatically generated based on biofeedback from the user.

[0331] (D4) In some embodiments of any of D1 to D3: (i) the user is performing a meditation activity, (ii) a timer is associated with the meditation activity, and (iii) based on determining that the level of tension does not meet one or more predefined criteria, starting the timer associated with the meditation activity is aborted. For example, when the user is performing a meditation activity and wearing a wrist-worn device, an indication corresponding to the user's corresponding level of exertion (e.g., tension) can be sensed at the wrist-worn device. This is based on the wrist-worn device detecting a threshold level of exertion above a reference level.

[0332] (D5) In some embodiments of D4: (i) the meditation activity is facilitated by a head-worn device configured to present an AR environment to the user, and (ii) the timer is presented by the head-worn device (e.g., intermittently, at a predetermined interval, and / or when a neuromuscular signal sensor detects a change in the user's level of tension) while the user is performing the meditation activity.

[0333] (D6) In some embodiments of any of D1 to D5, method 1200 includes: before determining the level of tension in a particular muscle group: (i) determining that the user is performing a physical activity, and (ii) determining that the user has stopped the physical activity. In some embodiments, the level of tension in a particular muscle group of the user is determined based on determining that the user has stopped the physical activity. For example, Figure 1C and Figure 1D the wrist-worn device 102 in

[0334] (D7)In some embodiments of D6, method 1200 further includes: notifying the user that they can resume physical activity based on determining that the tension level meets one or more predefined criteria. In some embodiments, the user is notified to resume physical activity after they have relaxed for at least a predefined amount of time. For example, a user may be performing a round of golf and an indication may be provided to the user based on determining that they have reached a particular relaxation level (e.g., not tense).

[0335] (E1)According to some embodiments, there is provided a wrist-wearable device comprising: (i) a display; (ii) one or more processors; and (iii) a memory including instructions which, when executed by the wrist-wearable device, cause any of the methods of A1 to A23, B1 to B21, C1 to C22, and C1 to D7 to be performed.

[0336] (F1)According to some embodiments, there is provided a wrist-wearable device comprising means for performing any of the methods of A1 to A23, B1 to B21, C1 to C22, and C1 to D7.

[0337] (G1)According to some embodiments, a pod device removable from a wrist-wearable device, the pod device comprising: (i) a display; (ii) one or more processors; and (iii) a memory including instructions which, when executed by the wrist-wearable device, cause any of the methods of A1 to A23, B1 to B21, C1 to C22, and C1 to D7 to be performed.

[0338] (H1)According to some embodiments, a non-transitory computer-readable storage medium includes instructions which, when executed by a computing device, cause the computing device to perform any of the methods of A1 to A23, B1 to B21, C1 to C22, and C1 to D7.

[0339] (I1)According to some embodiments, a system includes a wrist-wearable device and a connected device that communicates with the wrist-wearable device, the system being configured to perform or cause to be performed any of the methods of A1 to A23, B1 to B21, C1 to C22, and C1 to D7.

[0340] It will be understood that although terms such as "first", "second", etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0341] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the claims. As used in the description of the various embodiments and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms as well. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of the stated features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0342] As used herein, depending on the context, the term "if" can be interpreted to mean: "when the stated condition precedent is true"; or "once" the stated condition precedent is true; or "in response to determining" the stated condition precedent is true; or "in accordance with determining" the stated condition precedent is true; or "in response to detecting" the stated condition precedent is true. Similarly, depending on the context, the phrase "if it is determined [that the stated condition precedent is true]" or "if [the stated condition precedent is true]" or "when [the stated condition precedent is true]" can be interpreted to mean: "once it is determined" the stated condition precedent is true; or "in response to determining" the stated condition precedent is true; or "in accordance with determining" the stated condition precedent is true; or "once detected" the stated condition precedent is true; or "in response to detecting" the stated condition precedent is true.

[0343] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of operation and the practical application, thereby enabling others skilled in the art to implement.

Claims

1. A method for determining an adaptive adjustment to a physical activity being performed by a user wearing a wearable electronic device, comprising: Detecting, using one or more sensors located at the wearable electronic device, that the user is performing the physical activity at a particular activity rate; Detecting, using a neuromuscular signal sensor located at the wearable electronic device, the user's exertion level; And Determining an adjustment to the particular activity rate at which the user is performing the physical activity based on determining that the exertion level differs from a baseline exertion level by at least a threshold amount.

2. The method according to claim 1, further comprising: Before determining that the exertion level differs from the baseline exertion level by at least the threshold amount: Separately from detecting the exertion level, using the neuromuscular signal sensor to detect another exertion level; and Providing a notification to the user based on determining that the another exertion level differs from the baseline exertion level by at least another threshold amount.

3. The method according to claim 2, wherein: The notification is provided at an electronic exercise device; The notification includes selectable user interface elements; and In response to the user selecting the selectable user interface elements, causing the electronic exercise device to adjust the activity rate.

4. The method according to claim 2 or 3, further comprising: After determining that the another exertion level differs from the baseline exertion level by at least the another threshold amount and before determining that the exertion level differs from the baseline exertion level by at least the threshold amount: Separately from detecting the exertion level and detecting the another exertion level, using the neuromuscular signal sensor among the one or more sensors located at the wearable electronic device to detect yet another exertion level; And Determining another adjustment to the particular activity rate based on determining that the yet another exertion level differs from the baseline exertion level by yet another threshold amount, the another adjustment being different from the adjustment; and optionally Wherein, the another adjustment to the particular activity rate is to decrease the activity rate; and Wherein, the adjustment to the particular activity rate is to stop the physical activity.

5. The method according to any one of the preceding claims, wherein: The user is performing the physical activity at an electronic exercise device; and Determining that the exertion level is different from the baseline exertion level is further based on data from the electronic exercise device; and optionally Wherein, the data from the electronic exercise device is generated by one or more of the following: (i) a photoplethysmography (PPG) sensing device; (ii) an electrocardiogram (ECG) sensing device; and (iii) a gyroscope sensor.

6. The method according to any one of the preceding claims, wherein: The one or more sensors at the wearable electronic device include an inertial measurement unit (IMU) sensor; and Determining that the exertion level is different from the baseline exertion level is further based on data from the IMU sensor.

7. The method according to claim 6, wherein, Determining that the exertion level is different from the baseline exertion level further comprises: Apply a first weight to data from the neuromuscular signal sensor; and Apply a second weight to data from the IMU sensor.

8. The method according to claim 7, wherein The first weight and the second weight are based on calibration performed by the user prior to performing the physical activity at the activity rate; and / or wherein the first weight and the second weight are based on the type of physical activity being performed; and optionally wherein: depending on determining that the neuromuscular signal sensor is configured to sense a primary muscle group associated with the physical activity, the first weight is higher than the second weight.

9. The method according to any one of the preceding claims, wherein: The activity rate is a first activity rate; The exertion level is lower than the reference exertion level by the threshold amount; And Automatically increase the first activity rate to a second activity rate, the second activity rate being higher than the first activity rate.

10. The method according to any one of the preceding claims, wherein: The exertion level is higher than the reference exertion level by the threshold amount; and The adjustment causes the electronic exercise device to stop operating to cause the physical activity.

11. The method according to any one of the preceding claims, wherein: The user is wearing a head-wearable electronic device; The head-wearable electronic device is presenting a user interface to the user, the user interface corresponding to an interactive exercise; And Based on determining that the exertion level differs from the reference exertion level by the threshold amount, the adjustment to the physical activity includes: causing an adjustment to the user interface presented by the head-wearable electronic device.

12. The method according to any one of the preceding claims, wherein, Determining that the exertion level differs from the reference exertion level by the threshold amount is further based on at least one of: the number of identified repetitions of the physical activity that the user has performed at the activity rate; and The number of identified repetitions of another physical activity that the user has performed, the another physical activity being different from the physical activity that the user is currently performing.

13. The method according to any one of the preceding claims, wherein: The threshold amount is based on a plurality of context criteria associated with the user; and The threshold amount is determined based on inputting one or more of the plurality of context criteria into a machine learning model.

14. A wrist-wearable device, comprising: A display; One or more processors; And A memory, the memory including instructions that, when executed by the wrist-wearable device, cause the following operations to be performed: In conjunction with one or more sensors located at the wrist-wearable device, detect that a user is performing a physical activity at a particular activity rate: Use a neuromuscular signal sensor located at the wrist-wearable device to detect the user's exertion level; And Based on determining that the exertion level differs from a reference exertion level by at least a threshold amount, determine an adjustment to the particular activity rate at which the user is performing the physical activity.

15. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium comprising instructions that, when executed by a computing device, cause the computing device to perform the following operations: Detect, in combination, that a user is performing a physical activity at a particular activity rate using one or more sensors located at a wearable electronic device; Detect the user's exertion level using a neuromuscular signal sensor located at the wearable electronic device; and Determine an adjustment to the particular activity rate at which the user is performing the physical activity based on determining that the exertion level differs from a baseline exertion level by at least a threshold amount.

Images