Navigate through user interface elements using swipe gestures with variable kinematic profiles.

Abstract

A method for navigating through user interface elements using a swipe gesture with a variable kinematic profile is disclosed. The method includes: displaying a plurality of user interface (UI) elements; acquiring data generated during a first swipe motion of the thumb on the index finger, the first swipe motion having a first kinematic profile; and, based on the first kinematic profile, causing a focus selector to move through a first set of the plurality of UI elements at a first rate. The method further includes: acquiring additional data generated during a second swipe motion of the thumb on the index finger, the second swipe motion having a second kinematic profile different from the first kinematic profile; and, based on the second kinematic profile, causing a focus selector to move through a second set of the plurality of UI elements at a second rate.

Description

[0001] Related applications This application claims priority to U.S. Provisional Patent Application No. 63 / 758,776, filed February 14, 2025, entitled “Navigating Through UserInterface Elements With A Power Swipe Hand Gesture,” and U.S. Non-Provisional Patent Application No. 19 / 537,383, filed February 11, 2026, which are incorporated herein by reference. Technical Field

[0002] This disclosure generally relates to input systems for wearable devices, and more particularly to detecting and interpreting variable kinematic contour gestures used for navigating user interface elements. Background Technology

[0003] Wearable devices have become increasingly prevalent as an input mechanism for interacting with electronic devices and user interfaces. These wearable devices can detect gestures performed by the user to enable navigation through user interface elements, selection of items, and other interactions. Gesture-based input modalities offer advantages in hands-free operation and natural interaction paradigms, especially in contexts where traditional input mechanisms may be impractical or unavailable (e.g., extended reality environments).

[0004] However, wearable devices are worn by different user groups who may perform gestures with varying levels of intensity, speed, force, and displacement.

[0005] Therefore, it is necessary to address one or more of the challenges mentioned above. The following is a brief overview of the solutions to the problems mentioned above. Summary of the Invention

[0006] Individual users can have different physical characteristics, motor control abilities, and personal preferences that influence how they perform gestures. A generic gesture recognition model may not be suitable for every user out of the box, and may not provide the accuracy required by everyday users. Therefore, existing gesture-based navigation methods can benefit from technologies that provide sufficient granularity in response to changes in gesture execution.

[0007] This document describes one or more examples of detecting personalized gestures configured to enhance swipe functionality and distinguish between regular and forceful swipe gestures. Example devices include a non-transitory computer-readable storage medium comprising executable instructions that, when executed by one or more processors, cause the processors to perform one or more operations. These operations include: displaying multiple user interface (UI) elements on a user interface; receiving data generated during the execution of a swipe gesture via one or more neuromuscular signal sensors; and determining the amplitude of the swipe gesture. These operations also include: scrolling through a set of multiple UI elements displayed in the user interface based on determining the amplitude of the swipe gesture as a first amplitude; and scrolling through another set of multiple UI elements based on determining the amplitude of the swipe gesture as a second amplitude greater than the first amplitude. The amplitude of the swipe gesture is proportional to the number of UI elements in the first and / or the second set.

[0008] This document describes an example of using a personalized approach to determine when a user performs a forceful swipe gesture. The example device includes a non-transitory computer-readable storage medium comprising executable instructions that, when executed by one or more processors, cause the processors to perform one or more operations. These operations include: receiving, via one or more neuromuscular signal sensors, a first set of calibration data generated during the execution of a swipe gesture with a first amplitude; receiving, via one or more neuromuscular signal sensors, a second set of calibration data generated during the execution of a swipe gesture with a second amplitude; and applying a bimodal distribution to the first and second sets of calibration data to determine a threshold that distinguishes a swipe gesture from a forceful swipe gesture.

[0009] Instructions for performing the methods and operations described herein can be stored on a non-transitory computer-readable storage medium. This non-transitory computer-readable storage medium may be on a single electronic device or distributed across multiple electronic devices in a system (computing system). A non-exhaustive list of electronic devices that can perform the methods and operations described herein individually or in combination (e.g., a system) includes extended-reality (XR) headsets / glasses (e.g., mixed-reality (MR) headsets or augmented-reality (AR) glasses, as two examples), wrist-worn devices, intermediate processing devices, textile-based smart clothing, etc. For example, instructions may be stored on an AR pair of glasses, or on a combination of an AR pair of glasses and an associated input device (e.g., a wrist-worn device), such that instructions for detecting input operations can be executed on the input device, and instructions for changing the displayed user interface in response to these input operations can be executed on the AR glasses. The devices and systems described herein can be configured for use in conjunction with methods and operations that provide an XR experience. Methods and operations used to provide XR experiences can be stored on non-transitory computer-readable storage media.

[0010] The devices and / or systems described herein may be configured to include instructions that cause the performance of methods and operations associated with: presenting an XR head-mounted device and / or interacting with an XR head-mounted device. These methods and operations may be stored on a non-transitory computer-readable storage medium of the device or system. It should also be noted that the devices and systems described herein may be part of a larger overall system comprising multiple devices. A non-exhaustive list of electronic devices (which may individually or in combination (e.g., systems) include instructions that cause the performance of methods and operations associated with: presenting an XR experience and / or interacting with an XR experience) includes: extended reality head-mounted devices (e.g., mixed reality (MR) head-mounted devices or a pair of augmented reality (AR) glasses, as two examples), wrist-worn devices, intermediate processing devices, textile-based smart clothing, etc. For example, when describing an XR headset, it can be understood that the XR headset can communicate with one or more other devices (e.g., wrist-worn devices, servers, intermediate processing devices), which together can include instructions for performing methods and operations associated with: the presentation of an extended reality system and / or interaction with an extended reality system (i.e., the XR headset will be part of a system that includes one or more additional devices). Various combinations with different associated devices are envisioned, but for brevity, they will not be elaborated upon.

[0011] The features and advantages described in this specification are not necessarily all-encompassing; in particular, certain additional features and advantages will be apparent to those skilled in the art from the accompanying drawings, specification, and claims. Furthermore, it should be noted that the terminology used in this specification has been chosen primarily for readability and guidance purposes.

[0012] Having outlined the examples above, a brief description of the accompanying drawings will now be presented. Attached Figure Description

[0013] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, in which similar reference numerals indicate corresponding parts in all drawings.

[0014] Figures 1A to 1D A wrist-worn wearable device is shown, according to some embodiments, where the detection of a swipe gesture is configured to scroll through one or more user interface (UI) elements.

[0015] Figures 2A to 2C A swipe handgesture calibration module for determining the amplitude of a power swipe handgesture, according to some embodiments, is shown.

[0016] Figures 3A to 3D The illustration shows a hand performing a swipe gesture while wearing a wrist-worn wearable device, according to some embodiments.

[0017] Figure 4 A method for navigating through user interface elements using variable-amplitude sliding motion is illustrated according to some embodiments.

[0018] Figure 5A , Figure 5B , Figure 5C-1 and Figure 5C-2 Example MR and AR systems according to some embodiments are shown.

[0019] By convention, the features shown in the accompanying drawings may not be drawn to scale. Accordingly, the dimensions of the features may be arbitrarily scaled up or down for clarity. Additionally, some of these drawings may not depict all parts of a given system, method, or apparatus. Finally, similar reference numerals may be used throughout the specification and drawings to denote similar features. Detailed Implementation

[0020] This document describes numerous details to provide a thorough understanding of the exemplary embodiments illustrated in the accompanying drawings. However, some embodiments may be practiced without many of these specific details, and the scope of the claims is limited only to those features and aspects specifically recited in the claims. Furthermore, well-known processes, components, and materials are not necessarily described in full detail to avoid obscuring relevant aspects of the embodiments described herein.

[0021] Overview Embodiments of this disclosure may include different types of extended reality (XR), or a combination of different types of XR, such as mixed reality (MR) systems and augmented reality (AR) systems. As described herein, MR and AR are any overlay of functional and / or sensorily detectable presentations provided by MR and AR systems within a user's physical environment. Such MR may include and / or represent virtual reality (VR) and VR, wherein at least some aspects of the surrounding environment are reconstructed in the virtual environment (e.g., displaying a virtual reconstruction of physical objects in the physical environment to avoid collisions between the user and physical objects in the surrounding physical environment). In the case of MR, the surrounding environment presented through a display is captured by one or more sensors configured to capture the surrounding environment (e.g., camera sensors, time-of-flight (ToF) sensors). While the wearer of an MR headset can see full details of the surrounding environment, what they see is an environmental reconstruction reproduced using data from one or more sensors (i.e., the user cannot directly see physical objects). MR headsets may also forgo displaying a reconstruction of objects in the physical environment, thereby providing the user with a fully VR experience. On the other hand, AR systems provide an experience in which information is provided, for example, by using waveguides and in combination with one or more transparent or semi-transparent waveguides and / or lenses through an AR head-mounted device to directly view at least some of the surrounding environment. Throughout this application, the term "extended reality (XR)" is used as a general term encompassing both AR and MR. Furthermore, this application sometimes uses "head-wearable device" or "head-mounted device" as a generic term to cover XR head-mounted devices such as AR glasses and MR head-mounted devices.

[0022] As described above, the MR environment described herein may include, but is not limited to, non-immersive VR environments, semi-immersive VR environments, and fully immersive VR environments. As described above, AR environments may include marker-based AR environments, markerless AR environments, location-based AR environments, and projection-based AR environments. The above description is not exhaustive, and any other environment that allows intentional ambient lighting to pass through to reach the user will fall within the scope of AR, while any other environment that does not allow intentional ambient lighting to pass through to reach the user will fall within the scope of MR.

[0023] AR and MR content can include video, audio, haptic events, sensory events, or some combination thereof, any of which can be presented in a single channel or in multiple channels (e.g., stereoscopic video that creates a three-dimensional effect for the viewer). Furthermore, AR and MR can be associated with applications, products, accessories, services, or some combination thereof, for example, for creating content in an AR or MR environment and / or otherwise using it in an AR or MR environment (e.g., performing activities in an AR or MR environment).

[0024] The interactions with these AR and MR environments described in this article can occur using multiple different modalities, and the resulting outputs can also occur across multiple different modalities. In one example of an AR or MR system, a user can perform an air swipe gesture to skip a song via an application programming interface (API) provided by the song: the song's API provides playback, for example, at a home speaker.

[0025] As described herein, gestures can include air gestures, surface contact gestures, and / or other gestures that can be detected and determined based on: single-hand movement (e.g., a single-hand gesture performed by a user's hand, detected by one or more sensors of a wearable device (e.g., an electromyography (EMG) and / or an inertial measurement unit (IMU) of a wrist wearable device and / or one or more sensors included in a smart textile wearable device), and / or the single-hand gesture is detected by image data captured by an imaging device of the wearable device (e.g., a camera of a head wearable device, an external tracking camera positioned in the surrounding environment). "Air" generally includes gestures in which a portion of a user's hand does not contact a surface, object, or electronic device (e.g., a head wearable device or other communication-coupled device, such as a wrist wearable device); in other words, the gesture is performed in open space in 3D space and does not contact a surface, object, or electronic device. More generally, surface contact gestures (contacts on surfaces, objects, user body parts, or electronic devices) are also considered, in which contact (or intention to contact) is detected at the surface (e.g., on a table, on the user's hand or another finger, on the user's leg, a sofa, or a steering wheel with a single or two-finger tap). The various gestures disclosed herein can be detected using image data and / or sensor data (e.g., neuromuscular signals sensed by one or more bioelectric sensors (e.g., EMG sensors) or other types of data from other sensors (e.g., proximity sensors, time-of-flight (ToF) sensors, IMU sensors, capacitive sensors, strain sensors) detected by wearable devices worn by the user and / or other electronic devices owned by the user (e.g., smartphones, laptops, imaging devices, intermediate devices, and / or other devices described herein).

[0026] The input modalities described above can be different and depend on the user's experience. For example, in interactions using a wrist-worn wearable device, a user can provide input using air gestures or surface contact gestures detected by the neuromuscular signal sensors of the wrist-worn wearable device. In the absence of a wrist-worn wearable device, alternative and fully interchangeable input modalities can be used, such as one or more cameras located on a head-mounted device / glasses or elsewhere to detect air gestures or surface contact gestures, or input at an intermediate processing device (e.g., via physical input components such as buttons and touchpads). These different input modalities can be interchanged based on the desired user experience, portability, and / or product feature set (e.g., low-cost products may not include handheld tracking cameras).

[0027] Different inputs result in different outputs. For example, an air gesture input detected by a camera on a head-mounted wearable device can cause an output to occur at the head-mounted wearable device or to control another electronic device different from the head-mounted wearable device. In another example, input detected using data from a neuromuscular signal sensor can also cause an output to occur at the head-mounted wearable device or to control another electronic device different from the head-mounted wearable device. While only a few examples have been described above, those skilled in the art will understand that different input patterns and different output patterns in response to inputs are interchangeable.

[0028] The specific operations described above may occur depending on the specific hardware. The devices described are not limiting, and features may be removed from these devices or additional features may be added to them. Different devices may include one or more similar hardware components. For the sake of brevity, similar devices and components are described herein. Any differences between the devices and components will be described in their respective sections below.

[0029] As described herein, a processor (e.g., a central processing unit (CPU) or a micro controller unit (MCU)) is an electronic component responsible for executing instructions and controlling the operation of electronic devices (e.g., wrist-worn devices, head-worn devices, hand-held intermediary processing devices (HIPDs), textile-based smart clothing, or other computer systems). Various types of processors exist, which may be used interchangeably or are specifically required by the embodiments described herein. For example, the processor can be: (i) a general-purpose processor designed to perform a wide range of tasks, such as running software applications, managing operating systems, and performing arithmetic and logical operations; (ii) a microcontroller designed for specific tasks, such as controlling electronic devices, sensors, and motors; (iii) a graphics processing unit (GPU) designed to accelerate the creation and rendering of images, videos, and animations (e.g., VR animations, such as 3D modeling); (iv) a field-programmable gate array (FPGA) that can be programmed and reconfigured post-manufacturing and / or customized to perform specific tasks, such as signal processing, encryption, and machine learning; or (v) a digital signal processor (DSP) designed to perform mathematical operations on signals (e.g., audio, video, and radio waves). Those skilled in the art will understand that one or more processors of one or more electronic devices can be used in the various embodiments described herein.

[0030] As described herein, a controller is an electronic component that manages and coordinates the operation of other components within an electronic device (e.g., controlling inputs, processing data, and / or generating outputs). Examples of controllers may include: (i) microcontrollers, which include small, low-power controllers commonly used in embedded systems and Internet of Things (IoT) devices; (ii) programmable logic controllers (PLCs), which can be configured for use in industrial automation systems to control and monitor manufacturing processes; (iii) system-on-a-chip (SoC) controllers, which integrate multiple components such as processors, memory, input / output (I / O) interfaces, and other peripherals into a single chip; and / or (iv) digital signal processing units (DSPs). As described herein, a graphics module is a component or software module designed to handle graphics computations and / or graphical processes, and such a graphics module may include hardware modules and / or software modules.

[0031] As described herein, memory refers to electronic components in a computer or electronic device that store data and instructions for access and operation by a processor. The devices described herein may include volatile and non-volatile memory. Examples of memory may include: (i) random access memory (RAM) (e.g., DRAM, SRAM, DDRRAM, or other random access solid-state memory devices) configured to temporarily store data and instructions; (ii) read-only memory (ROM) configured to permanently store data and instructions (e.g., one or more portions of system firmware and / or bootloader); (iii) flash memory, disk storage devices, optical disk storage devices, other non-volatile solid-state storage devices that may be configured to store data in electronic devices (e.g., universal serial bus (USB) drives, memory cards, and / or solid-state drives (SSDs)); and (iv) cache memory configured to temporarily store frequently accessed data and instructions. As described herein, memory may include structured data (e.g., SQL databases, MongoDB databases, Graph QL data, or JSON data). Other examples of storage may include: (i) user profile data, including user account data, user settings and / or other user data stored by the user; (ii) sensor data detected by one or more sensors and / or otherwise acquired; (iii) media content data, including stored image data, audio data and documents, etc.; (iv) application data, which may include data collected and / or otherwise acquired and stored during use of the application; and / or (v) any other type of data described herein.

[0032] As described herein, the power system of an electronic device is configured to convert input power into a form usable for operating the device. The power system may include various components, including: (i) a power source, which may be an alternating current (AC) adapter power source or a direct current (DC) adapter power source; (ii) a charger input, which may be configured to use wired and / or wireless connections (which may be part of a peripheral interface, such as USB, microUSB, near-field magnetic coupling, magnetic induction and magnetic resonance charging, and / or radio frequency (RF) charging); (iii) a power management integrated circuit configured to distribute power to the various components of the device and ensure that the device operates within safety limits (e.g., regulating voltage, controlling current, and / or managing heat dissipation); and / or (iv) a battery configured to store power to provide usable power to the various components of one or more electronic devices.

[0033] As described herein, a peripheral interface is an electronic component (e.g., an electronic component of an electronic device) that allows the electronic device to communicate with other devices or peripheral devices and can provide means for inputting and outputting data and signals. Examples of peripheral interfaces may include: (i) USB and / or micro USB interfaces configured to connect devices to an electronic device; (ii) Bluetooth interfaces configured to allow devices to communicate with each other, including Bluetooth Low Energy (BLE); (iii) Near-Field Communication (NFC) interfaces configured as short-range wireless interfaces for operations such as access control; (iv) Pogo pins, which may be small, spring-loaded pins configured to provide a charging interface; (v) Wireless charging interfaces; (vi) Global Positioning System (GPS) interfaces; (vii) Wi-Fi interfaces for providing connectivity between a device and a wireless network; and (viii) Sensor interfaces.

[0034] As described herein, a sensor is an electronic component (e.g., an electronic component in an electronic device (e.g., a wearable device) and / or an electronic component that otherwise communicates electronically with the electronic device). Examples of sensors may include: (i) imaging sensors for collecting imaging data (e.g., including one or more cameras arranged on a corresponding electronic device, such as a Simultaneous Localization and Mapping (SLAM) camera); (ii) bioelectric potential signal sensors; (iii) IMUs for detecting, for example, changes in angular velocity, force, magnetic field, and / or acceleration; (iv) heart rate sensors for measuring a user's heart rate; (v) peripheral oxygen saturation (SpO2) sensors for measuring a user's blood oxygen saturation and / or other biometric data; (vi) capacitive sensors for detecting potential changes at a part of the user's body (e.g., a sensor-skin interface) and / or near other devices or objects; (vii) sensors for detecting certain inputs (e.g., capacitive sensors and force sensors); and (viii) light sensors (e.g., TFO sensors, infrared light sensors, or visible light sensors, etc.), and / or sensors for sensing data from the user or the user's environment. As described herein, bioelectric potential signal sensing components are devices for measuring electrical activity within the body (e.g., bioelectric potential signal sensors). Some types of bioelectric potential signal sensors include: (i) electroencephalography (EEG) sensors, which are configured to measure electrical activity in the brain to diagnose neurological diseases; (ii) electrocardiography (ECG or EKG) sensors, which are configured to measure electrical activity in the heart to diagnose heart problems; (iii) EMG sensors, which are configured to measure electrical activity in muscles and diagnose neuromuscular diseases; and (iv) electrooculography (EOG) sensors, which are configured to measure electrical activity in eye muscles to detect eye movements and diagnose eye diseases.

[0035] As described herein, applications (e.g., software) stored in the memory of an electronic device include instructions stored in the memory. Examples of such applications include: (i) games; (ii) word processors; (iii) messaging applications; (iv) media streaming applications; (v) financial applications; (vi) calendars; (vii) clocks; (viii) web browsers; (ix) social media applications; (x) camera applications; (xi) web-based applications; (xii) health applications; (xiii) AR and MR applications; and / or (xiv) any other applications that may be stored in memory. These applications may operate in conjunction with one or more components of a data, and / or device or communication-coupled device to perform one or more operations and / or functions.

[0036] As described herein, a communication interface module may include hardware and / or software capable of data communication using any of the following protocols: custom wireless protocols or standard wireless protocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.11a, WirelessHART, or MiWi); custom wired protocols or standard wired protocols (e.g., Ethernet or HomePlug); and / or any other suitable communication protocol (including communication protocols not yet developed as of the date of this submission). A communication interface is a mechanism that enables different systems or devices to exchange information and data with each other, including hardware, software, or a combination of both. For example, a communication interface may refer to a physical connector and / or port on a device that enables communication with other devices (e.g., USB, Ethernet, HDMI, or Bluetooth). A communication interface may refer to a software layer that enables different software programs to communicate with each other (e.g., APIs and protocols such as HTTP and TCP / IP).

[0037] As described herein, a graphics module is a component or software module designed to handle graphics operations and / or graphical processes, and the graphics module may include hardware modules and / or software modules.

[0038] As described herein, a nontransitory computer-readable storage medium is a physical device or storage medium that can be used to store electronic data in a nontransitory form (e.g., such that the data is permanently stored until it is intentionally deleted or modified).

[0039] Detecting strong swipe gestures performed by the user Figures 1A to 1DA wrist-worn wearable device is shown, according to some embodiments, where the detection of a swipe gesture is configured to scroll through one or more user interface (UI) elements. Figure 1A A plurality of user UI elements (e.g., UI elements 112 to 120) displayed on user interface 110 according to some embodiments are shown. In some embodiments, UI elements 112 to 120 include, for example: Figure 1A The UI elements shown are a series of numbers, but may also include a list of items and / or one or more images. In some embodiments, the user interface 110 may include a two-dimensional or three-dimensional navigation space, which may include various UI elements at discrete points within the respective navigation space, rather than an explicit list of UI elements as shown herein. Figure 1A It also includes a UI selector 122, which highlights the currently selected UI element (e.g., in...). Figure 1A UI element 116 is highlighted. In some embodiments, the user interface 110 is on the head-mounted wearable device (AR device 528). Figure 5A ) or wrist-worn wearable devices (wrist-worn wearable devices 526; Figure 5A It is displayed on the monitor. Figure 1A Also shown is a wrist-worn wearable device 130, which includes one or more neuromuscular signal sensors coupled to a user's wrist, configured to detect one or more gestures performed by the user's hand 132. While the embodiments described herein illustrate gestures detected by neuromuscular signal sensors, those skilled in the art will recognize that other sensors may be used in addition to or as alternatives to the neuromuscular signal sensors described herein. For example, data obtained from an IMU sensor of the wrist-worn wearable device and / or an imaging sensor located at either the wrist-worn wearable device 130 or another device communicatively coupled to the wrist-worn wearable device 130 (e.g., AR device 528), or both, may be used to identify the corresponding gesture performed by the user.

[0040] Figure 1B The illustration shows a user performing a forceful swipe gesture according to some embodiments. Figure 1BMultiple user UI elements (e.g., UI elements 142 to 150) displayed on user interface 110 according to some embodiments are shown. In some embodiments, a swipe gesture is performed by sliding one of a user's multiple phalanges (e.g., thumb 134) in a first direction (e.g., to the left). The thumb may slide on another phalanx (e.g., index finger 136) or on a surface such as a table. In some embodiments, the swipe gesture includes thumb movement on the corresponding other phalanx or surface, whereby the swipe gesture does not involve contact with another body part or another object. The user can perform the swipe gesture with different kinematic profiles representing force, velocity, acceleration, and displacement. For example, the user can slide their thumb on index finger 136 quickly or at a more moderate speed, with considerable force or gently, etc. Data received at a neuromuscular signal sensor can be used to determine the kinematic profile of the swipe gesture. For example, Figure 1B It also shows a user sliding their thumb 134 over their index finger and ending the slide a few centimeters (e.g., 1 to 5 centimeters) away from the index finger 136. The displacement of the thumb indicates that this is a strong swipe gesture. In some embodiments, one or more intensity thresholds can be used to determine whether a particular gesture constitutes a strong swipe gesture. In response to a strong swipe gesture, the UI selector 122 scrolls through several (e.g., more than 10 UI elements) UI elements at a rate proportional to the kinematic profile of the strong swipe gesture. For example, in Figure 1A In the middle, UI selector 122 highlights UI element 116 that is displayed as 52, while... Figure 1B In response to a strong swipe gesture, UI selector 122 highlights the UI element 144 displayed on 23.

[0041] Figure 1C The illustration shows a user performing a gentle swipe gesture according to some embodiments. Figure 1C A plurality of user UI elements (e.g., UI elements 112 to 120) displayed on user interface 110 according to some embodiments are shown. Figure 1C It also shows the user's thumb 134 to... Figure 1B The kinematic contours of the strong swipe gesture shown are different from those of the kinematic contours (e.g., smaller displacement, smaller velocity, smaller force, etc.) swiping on the user's index finger 136. For example, compared to... Figure 1B In contrast to a strong swipe gesture performed in the middle, the user's thumb 134 does not extend beyond the user's index finger 136. According to some embodiments, the UI selector 122 is made to respond to... Figure 1B The strong swipe gesture shown moves UI elements presented in user interface 110 at different rates. For example, as Figure 1CAs shown, UI selector 122 begins at UI element 116, and UI selector 122 shifts only two positions to the left in response to a gentler swipe gesture. In some embodiments, when a swipe gesture fails to meet one or more intensity criteria for performing a strong swipe gesture (e.g., associated with the corresponding kinematic profile of the gesture), the focus selector or other navigation elements within the user interface are caused to move at a specific rate (e.g., a predefined increment) disproportionate to the intensity associated with the kinematic profile of the swipe gesture.

[0042] Figure 1D The illustration shows a user performing a forceful swipe gesture in another direction, according to some embodiments. Figure 1D A plurality of user UI elements (e.g., UI elements 162 to 170) displayed on user interface 110 according to some embodiments are shown. Figure 1D It also shows the user's interaction with Figures 1B to 1C A strong swipe gesture is performed in the opposite direction (e.g., to the right instead of the left). In some embodiments, the UI selector 122 moves in a direction corresponding to the direction of the swipe gesture. For example, in response to a strong swipe gesture direction performed to the right, the UI selector 122 moves several numbers to the right to UI element 168. Figure 1B As described, the user slides their thumb 134 on their index finger 136 and ends the slide a few centimeters (e.g., 1 to 5 centimeters) away from the index finger 136, indicating that a strong swipe gesture is being performed. Figure 1D This shows the user's thumb 134 several centimeters (e.g., more than 3 centimeters) away from the user's index finger 136. The displacement of the thumb 134 indicates a strong gliding gesture. In some embodiments, this is used to determine... Figure 1B The gesture shown is determined by a set of one or more intensity thresholds that are different from the set of one or more intensity thresholds for a strong swipe gesture. Figure 1D The gesture shown is a strong swipe gesture, which can be based on the direction of thumb movement. That is, in some embodiments, the specific threshold used to identify a strong swipe gesture is direction-specific. In some embodiments, the UI element stops scrolling when the user performs a swipe gesture in the opposite direction of a previously performed swipe gesture.

[0043] In some embodiments, the kinematic profile of a swipe gesture is determined based on data from neuromuscular signal sensors, such as electromyography (EMG) sensors. The system can read EMG activity in real time and identify EMG activity patterns corresponding to specific gestures. In some embodiments, a regression model trained on multi-user data is used to classify swipes, and EMG recordings are used to predict swipe intensity. When a thumb swipe is detected, the model estimates the intensity of the swipe and triggers a corresponding rolling motion. The determination of the kinematic profile can be based on signal amplitude, frequency characteristics, pattern recognition, or a combination thereof.

[0044] In some embodiments, a user may receive feedback during or after performing a swipe gesture. This feedback may include visual indicators displayed on the user interface, showing the scrolling speed, scrolling position, or the number of UI elements traversed. In some embodiments, haptic feedback is provided via a wrist-worn wearable device to confirm gesture recognition and / or indicate the kinematic profile of a detected gesture. Audio cues may also be provided to indicate successful gesture detection or to convey information about the scrolling behavior. In some embodiments, the techniques described herein are configured to be performed on a device that does not include a display; for example, a user may be able to use the gesture detection techniques described herein to navigate through a playlist of music content without the system providing any visual indication of movement through the playlist, and the system may be configured to provide audio output to indicate to the user the effect of a particular gesture (e.g., "Move from song 1 to song 6 in your workout playlist based on a detected strong swipe gesture").

[0045] In some embodiments, scrolling through UI elements can be continuous, providing smooth animation as the UI selector moves through multiple UI elements. Alternatively, scrolling can be discrete, with the UI selector jumping incrementally between UI elements. Scrolling speed can be proportional to the intensity of the swipe gesture, such that a swipe with a higher kinematic profile results in faster scrolling, while a swipe with a lower kinematic profile results in slower scrolling. In some embodiments, the system supports multiple discrete scrolling speeds (e.g., three or more different speeds) corresponding to different ranges of gesture kinematic profiles.

[0046] In some embodiments, alternative sensor modalities can be used to detect the kinematic profile of a gesture, in addition to or in place of a neuromuscular signal sensor. For example, a computer vision model using a camera (e.g., a camera in a head-worn wearable device such as a VR headset) can track finger movements to determine the intensity of a slide. Wearable devices with touch or force sensors (e.g., gloves) can also be used to detect the kinematic profile of a gesture. Other sensor types that can be used include capacitive sensors, strain sensors, and inertial measurement units (IMUs). The advantage of an EMG-based approach is that it allows for interaction in a variety of hand positions, including when the user's hand is in a pocket or otherwise invisible to the camera.

[0047] In some embodiments, multiple UI elements that can be navigated using variable kinematic contour swipe gestures may include various types of content. For example, UI elements may include media playlists (e.g., a list of movies in a streaming service, a list of songs in a music app), calendar dates, map regions, photo galleries, volume or brightness controls, text documents, web pages, and vertically scrolling content on a phone screen or other display. Variable kinematic contour swipe gestures can be used to navigate through long lists of items, providing a more efficient user experience than scrolling through individual items.

[0048] In some embodiments, a swipe gesture can be performed using a combination of fingers instead of the thumb 134 sliding on the index finger 136. For example, a swipe can be performed by sliding the thumb 134 across multiple fingers, the palm, or other body surfaces. Vertical swipes (e.g., up or down) and diagonal swipes are also considered, which can be used to navigate a two-dimensional UI layout or to scroll through content in different directions. Figure 3C and Figure 3D As shown, a swipe gesture can be used to navigate the calendar interface, where the kinematic profile of the swipe determines how many days or weeks have been traversed.

[0049] In some embodiments, the system determines when a swipe gesture ends based on neuromuscular signal data and / or other sensor data. Gesture termination can be detected when EMG activity associated with the swipe motion falls below a threshold, or when the motion state indicates that the swipe motion has stopped. In some embodiments, momentum-based scrolling is implemented such that UI elements continue scrolling after the gesture ends, with the scrolling speed gradually decreasing over time. Users can perform a swipe gesture in the opposite direction to stop momentum-based scrolling, as referenced above. Figure 1D As described.

[0050] Figures 2A to 2CA swipe gesture calibration module for determining the magnitude of a strong swipe gesture according to some embodiments is illustrated. The swipe gesture calibration module 208 includes a coordinate graph with a vertical axis representing force measurements ranging from 0.0 to 1.0 and a horizontal axis representing time. This coordinate graph displays visual indicators categorizing gentle and strong swipes, allowing the user and system to visualize the force profile of each performed gesture. In some embodiments, the user navigates multiple cue screens (e.g., performs a swipe gesture to slide to the number 1) and is prompted to perform a swipe that they perceive as either right or left. Furthermore, the cue screen prompts the user to anticipate swiping over one or more UI elements at a time. When collecting data on swipe and strong swipe gestures, a bimodal distribution can be developed, where each corresponding mode of the bimodal distribution is associated with either a gentle or strong swipe gesture performed by the user. In some embodiments, thresholds for optimally separating the individual modes of the bimodal distribution are identified. In some embodiments, different bimodal distributions are obtained for gestures performed in different directions, such that the system identifies different thresholds for swipe gestures performed in different directions (e.g., one for left and one for right). This method takes into account the natural difference in thumb movement when sliding inward and outward, which affects the corresponding force profile of these slides.

[0051] In the personalized approach, users are instructed to perform gentle and strong swipe gestures in both directions at their own speed and in any order of their choice. In some embodiments, the system uses k-means clustering to determine the centroids of the two categories (e.g., the mean peak covariance norm of the gentle and strong swipes) in real time and sets a threshold between these two points (e.g., the gentle and strong swipe gestures) to classify the swipes. Thus, this approach produces a single threshold to distinguish between gentle and strong swipes in any direction.

[0052] Figure 2A A swipe gesture calibration module 208 is shown prior to receiving data from a wrist-worn wearable device 230 in response to a swipe gesture performed by the user's thumb 234 and index finger 236. The swipe gesture calibration module 208 includes a coordinate graph configured to display kinematic profile measurements (e.g., force measurements) over time as the user performs the calibration gesture. A user interface 210 displays a series of UI elements, including several numbers, arranged horizontally and featuring arrows indicating the direction of movement. A UI selector 222 includes... Figures 1A to 1DThe UI selector 222 described herein has similar features and is configured to move in response to a swipe gesture detected by the wrist wearable device 230. The UI selector 222 is shown as a dashed circle highlighting a UI element (e.g., the number 58) to indicate the currently selected element. The user interface 210 also includes prompts configured to encourage the user to perform a swipe gesture (e.g., "Go to 1, then index tap"), allowing the swipe gesture calibration module 208 to determine what type of swipe gesture the user is performing (e.g., a gentle swipe or a forceful swipe).

[0053] Figure 2B A swipe gesture calibration module 208, according to some embodiments, is shown capturing multiple strong swipe gestures. Figure 2B It is also shown that, when a user performs a swipe gesture, the wrist-worn wearable device 230 captures calibration data via one or more neuromuscular signal sensors. The swipe gesture calibration module 208, in response to the swipe gesture, determines the kinematic profile (e.g., force, velocity, acceleration, and / or displacement) used to perform the swipe gesture and plots it as follows: Figure 2B The coordinate graph shown illustrates a rising curve that stabilizes near a value of 1.0, indicating the kinematic profile of a strong swipe gesture. Multiple rectangular markers are displayed under the "strong swipe" label, representing a detected strong swipe gesture. Based on the kinematic profile used to perform the swipe gesture, the swipe gesture calibration module 208 classifies the swipe gesture as either a strong swipe or a gentle swipe. Figure 2B The swipe gesture performed has a measured kinematic profile indicating a strong swipe gesture. User interface 210 displays the prompt "Start, then tap with your index finger" and a series of numbered UI elements (e.g., displaying values ​​37, 38, 39, 40, and 41). UI selector 222 is used in conjunction with... Figures 1A to 1D The kinematic profiles used in the described forceful swipe gestures shift proportionally. A bimodal distribution is applied to each swipe gesture to determine whether it should be classified as a gentle or forceful gesture.

[0054] Figure 2C A swipe gesture calibration module 208 according to some embodiments is shown capturing a gentle swipe gesture. Figure 2CThe swipe gesture calibration module 208 is also shown to determine that the kinematic profile (e.g., force) used to perform the swipe gesture is below a threshold and classifies the swipe gesture as a gentle swipe gesture. The coordinate graph shows a curve that peaks at 1.0 near the 500 mark on the horizontal axis and then drops to approximately 0.0 at the 2000 mark, representing the kinematic profile of a gentle swipe in which force is briefly applied and then released. As shown in the coordinate graph, a lower kinematic profile is detected when performing a gentle gesture than when the user performs a strong swipe gesture. In this coordinate graph, the value for a strong swipe gesture is approximately 1, while the value for a gentle swipe gesture is between 0 and 0.5. The user interface 210 displays a series of UI elements, including a first UI element 133, a third UI element 135, and a fifth UI element 137. The UI selector 222 is shown as a dashed circle highlighting the third UI element 135. The gesture position of the thumb 234 positioned near the index finger 236 demonstrates a gentle swipe where the thumb does not extend beyond the index finger. The determination of the swipe gesture classification involves applying the bimodal distribution described above. In some embodiments, although Figures 2A to 2C The force is shown as an example kinematic profile parameter, but the kinematic profile may include other parameters, such as velocity, acceleration, displacement, or combinations thereof.

[0055] In some embodiments, the swipe gesture calibration module 208 can display real-time feedback to the user during calibration, showing the classification of each performed gesture and allowing the user to see how their gestures are interpreted by the system. The real-time feedback may include visual indicators on a coordinate graph showing the position of each gesture relative to a threshold between a gentle swipe classification and a strong swipe classification. In some embodiments, if the performed gesture is not obvious enough, the calibration module may provide guidance to the user, such as prompting the user to perform a strong swipe gesture with a higher kinematic profile (e.g., greater force or greater displacement) or a gentle swipe gesture with a lower kinematic profile. The feedback mechanism can help the user understand the expected gesture characteristics and improve the accuracy of the calibration process.

[0056] In some embodiments, the kinematic profiles can be orientation-specific, such that a swipe in a first direction (e.g., leftward or inward toward the palm) is associated with a first kinematic profile threshold, while a swipe in a second direction (e.g., rightward or outward away from the palm) is associated with a second kinematic profile threshold. The first and second kinematic profile thresholds can differ from each other due to the natural differences in thumb movement during inward and outward swipes (which affect the corresponding kinematic profiles). For example, the kinematic profile for an inward swipe can differ from that for an outward swipe because range of motion, muscle activation patterns, and force profiles can vary depending on the direction of the swipe. In some embodiments, the calibration module can prompt the user to perform a swipe gesture in each of multiple directions (e.g., left, right, up, down, diagonal) to capture orientation-specific calibration data and determine a separate kinematic profile threshold for each direction. This orientation-specific calibration can improve classification accuracy by taking into account the natural biomechanical differences when a user performs a swipe in different directions.

[0057] In some embodiments, calibration data may include the kinematic profile of a corresponding gesture performed in response to a visual cue presented within the calibration user interface. This kinematic profile may capture characteristics associated with each gesture performed during the calibration process, such as velocity, acceleration, displacement, and force. The visual cue may instruct the user to perform a specific type of swipe gesture, such as a gentle swipe or a forceful swipe, thereby allowing the system to associate the captured kinematic profile with the corresponding gesture category.

[0058] Figure 3A The illustration shows a user interface transition resulting from a forceful swipe gesture performed while wearing a wrist wearable device 130, according to some embodiments. The figure shows a user's hand, with the thumb 134 positioned to perform a swipe motion on the index finger 136. The wrist wearable device 130 is shown worn on the user's wrist and includes one or more neuromuscular signal sensors configured to detect the gesture performed by the user. The thumb 134 is depicted in a position indicating movement along an upward arcing direction relative to the index finger 136 (as indicated by the curved arrow).

[0059] In the first UI display 302a, the UI selector 304 is located on the display of the wrist wearable device (e.g., or another device including a head wearable device) showing December 1st. The wrist wearable device 130 receives data generated during the execution of a swipe gesture and determines the kinematic profile of the swipe gesture based on the sliding motion of the thumb 134 on the index finger 136. A strong swipe gesture is detected (where the user's input exceeds the threshold of...). Figures 2A to 2CFollowing the defined kinematic profile threshold, the second UI display 302b shows that the UI selector 304 has moved forward a full month to January 1st. The transition from the first UI display 302a to the second UI display 302b indicates that a strong swipe gesture causes the UI selector 304 to move a larger increment (e.g., a whole month) compared to a standard swipe gesture that causes the UI selector 304 to move in a smaller increment (e.g., a day or a week). The kinematic profile of the swipe gesture can be used to control scrolling through user interface elements, where the scrolling rate and increment are proportional to the kinematic profile of the detected gesture.

[0060] In some embodiments, UI selector 304 can provide visual feedback indicating the type of swipe gesture detected. For example, when a standard swipe gesture is detected, UI selector 304 can be displayed with a first visual appearance (e.g., a first color, size, or animation), while when a strong swipe gesture is detected, UI selector 304 can be displayed with a second visual appearance (e.g., a second color, size, or animation). Visual feedback helps users understand how their gestures are interpreted by the system. In some embodiments, the movement increment caused by a swipe gesture can be configured by the user. For example, a user can configure a strong swipe to advance by a month, a week, or another user-defined increment. Similarly, a user can configure a standard swipe to advance by a day, an hour, or another user-defined increment. Configurable increments can be stored in user preferences associated with the user's profile.

[0061] In some embodiments, the system can provide predictive scrolling based on detected kinematic profiles. For example, if the kinematic profiles indicate a strong swipe operation, the system can predict that the user intends to navigate to a more distant date and can preload or pre-render UI elements corresponding to dates further in the future or past. This predictive behavior can reduce latency and improve the responsiveness of the user interface. In some embodiments, the dials displayed in the first UI display 302a and the second UI display 302b may include additional contextual information that varies based on the selected date. For example, when the UI selector 304 is on a date that includes a calendar event, the dial may display an indicator (e.g., a dot, highlight, or text preview) associated with that event. This contextual information can be dynamically updated as the UI selector 304 moves through the dates in response to a swipe gesture.

[0062] In some embodiments, the system may support gesture chains, where multiple rapidly executed swipe gestures are combined to produce cumulative navigation effects. For example, if a user performs two consecutive strong swipe gestures, the UI selector 304 may advance two months (e.g., from December to February). The system may detect consecutive gestures based on timing between gesture completions and combine their effects accordingly. In some embodiments, the system may implement boundary behavior when the UI selector 304 reaches the beginning or end of a navigable range. For example, when navigating the calendar and the UI selector 304 reaches December 31, a subsequent forward swipe gesture may cause the UI selector 304 to loop back to January 1 of the following year; alternatively, the system may provide haptic or visual feedback indicating that a boundary has been reached and looping back is not possible.

[0063] Arrow 306 indicates the direction of the swipe gesture performed by the user's thumb 134. The length of arrow 306 is proportional to the kinematic profile of the swipe gesture; therefore, a longer arrow indicates a swipe with greater displacement, velocity, or force. Figure 3A In the image, arrow 306 is depicted as having a longer length, indicating that the swipe gesture is a strong swipe that exceeds the kinematic profile threshold. As shown in the transition from the first UI display 302a to the second UI display 302b, the longer arrow 306 visually corresponds to a larger increment of movement (e.g., a whole month) caused by the strong swipe gesture.

[0064] Figure 3B The diagram illustrates the user interface transitions resulting from a non-forceful swipe gesture (e.g., a small or gentle swipe) performed while wearing a wrist-worn wearable device 130, according to some embodiments. The figure shows a user's hand, with the thumb 134 and index finger 136 positioned to perform the swipe gesture. The thumb 134 is shown near the index finger 136, indicating that the thumb 134 is being used to engage with... Figure 3A The described strong gliding gesture is a gliding gesture that slides on or near the index finger 136 compared to a gliding gesture with a lower kinematic profile.

[0065] In the first UI display 302c, the UI selector 304 is located on the "5" of December 5th on the watch face. This is achieved when a non-forced swipe gesture is detected (where the user's input does not exceed the specified range). Figures 2A to 2C After the kinematic profile threshold (as described and determined) is reached, the second UI display 302d displays the UI selector 304, which has been incremented by one day to "4" on December 4th. The transition from the first UI display 302c to the second UI display 302d indicates that, with Figure 3A Compared to the strong swipe gesture described in the document that moves UI selector 304 by a large increment (e.g., a whole month), the non-strong swipe gesture moves UI selector 304 by a small increment (e.g., a day).

[0066] The swipe gesture is directional, causing a swipe in a first direction (e.g., left) to move UI selector 304 through UI elements in the corresponding direction. For example, a swipe to the left moves UI selector 304 from "5" to "4", while a swipe to the right moves UI selector 304 from "5" to "6". The kinematic profile of a non-forceful swipe gesture can be used to control scrolling through user interface elements, where the scrolling rate and increment are proportional to the kinematic profile of the detected gesture.

[0067] Arrow 308 indicates the direction of the swipe gesture performed by the user's thumb 134. The length of arrow 308 is proportional to the kinematic profile of the swipe gesture; therefore, a shorter arrow indicates a swipe with lower displacement, velocity, or force. Figure 3B In the diagram, arrow 308 is depicted as having a shorter length, indicating that the swipe gesture is not a strong swipe exceeding the kinematic profile threshold. As shown in the transition from UI display 302c to UI display 302d, the shorter arrow 308 visually corresponds to a smaller increment of movement (e.g., one day) caused by the swipe gesture.

[0068] Figure 3C A user interface for a calendar displayed next to a hand 132 performing a swipe gesture while wearing a wrist-worn wearable device 130 is shown, according to some embodiments. The calendar displays a monthly view of December, with dates arranged in a grid format. In a first UI display 302e, a UI selector 304 is positioned over December 25th, indicating the currently selected date. The hand 132 is shown performing a forceful gesture by extending the thumb 134 in an upward swiping motion. An arrow 310 indicates the direction and distance of the user's swipe gesture, wherein the length of the arrow 310 is proportional to the kinematic profile of the gesture. In some embodiments, the forceful gesture is used in conjunction with… Figures 2A to 2C The described strong swipe gesture has different kinematic profile thresholds, but the kinematic profile of this strong gesture is still smaller than that of the gesture described above. Figures 2A to 2C The kinematic profile threshold detected by the calibration module described in the text.

[0069] In response to a strong gesture detected by the wrist-worn wearable device 130, the UI selector 304 moves through calendar dates proportionally to the kinematic profile of the gesture. In the second UI display 302f, the UI selector 304 has moved to December 5th, indicating that the swipe-up gesture causes a selection to move up several weeks (e.g., three weeks from the 25th to the 5th). The kinematic profile of the strong gesture determines how many calendar dates are traversed, where a stronger gesture causes the UI selector 304 to move more dates compared to a weaker gesture. In some embodiments, the UI selector 304 can move up several additional weeks based on the kinematic profile of the gesture, so a gesture with a higher kinematic profile can cause the UI selector 304 to move four weeks or more in a single gesture.

[0070] Figure 3D A user interface for a calendar displayed next to a hand 132 performing a small swipe gesture while wearing a wrist-worn wearable device 130 is shown, according to some embodiments. For example, the calendar displays a monthly view of December, with dates arranged in a grid format. In a first UI display 302g, a UI selector 304 is positioned over December 25th, indicating the currently selected date. The hand 132 is shown performing a short upward gesture with the thumb 134 extended. An arrow 312 indicates the direction and distance of the user's swipe gesture, wherein the length of the arrow 312 is proportional to the kinematic profile of the gesture. Because the gesture is small, the arrow 312 is larger than... Figure 3C The arrow 310 shown is short.

[0071] In response to a small swipe gesture detected by the wrist wearable device 130, the UI selector 304 moves proportionally to the kinematic contour of the gesture along the calendar date. Because the kinematic contour of the gesture is small, the UI selector 304 only moves upwards by one revolution. In the second UI display 302h, the UI selector 304 has moved to December 18th, indicating that... Figure 3C Compared to the powerful gesture that moves UI selector 304 up three weeks, the short upward swipe gesture moves the selection up only one week (from the 25th to the 18th). Figure 3C and Figure 3D The difference in the number of dates traversed between the two gestures shows how the kinematic profile of the swipe gesture is proportional to the number of UI elements (such as calendar dates) scrolled through, where a gentler swipe results in fewer dates being traversed.

[0072] This paper discloses a method for scrolling through items on a screen using swipe gestures of varying intensities detected by sensors that read muscle signals. The method also includes displaying items on a display, collecting data from muscle sensors as the user swipes their thumb at different speeds or forces, and moving selections at speeds matching the force or speed of the user's swipes, including scrolling through more items faster with stronger swipes. The method also includes a setup process using statistical analysis to differentiate between regular and forceful swipes, and a wrist device with muscle sensors that works in conjunction with a head-mounted display of the screen.

[0073] Figure 4 A method 400 for navigating through user interface elements using variable-amplitude sliding motions, according to some embodiments, is illustrated. Method 400 can be performed by a system including a wrist-worn wearable device and a head-worn wearable device, the wrist-worn wearable device having one or more neuromuscular signal sensors, and the head-worn wearable device being configured to present a user interface.

[0074] In step 402, the system receives data generated by the thumb during the first sliding motion via one or more neuromuscular signal sensors. The first sliding motion has a first kinematic state, which can be characterized by one or more of the thumb's position, velocity, acceleration, or displacement during the sliding motion. (See also...) Figures 1A to 1D As described, the sliding motion can be performed by sliding the thumb 134 on the index finger 136 while wearing the wrist wearable device 130.

[0075] In step 404, based on a first kinematic state, the system causes the focus selector to move through a first set of user interface elements at a first rate. The focus selector (e.g., UI selector 122) moves through multiple UI elements at a rate corresponding to the kinematic profile of a detected gesture. For example, a gentle swipe gesture with a lower kinematic profile causes the focus selector to move through a smaller number of UI elements.

[0076] In step 406, the system receives additional data generated by the thumb during the second sliding motion via one or more neuromuscular signal sensors. The second sliding motion has a second kinematic state that differs from the first kinematic state, such as higher speed, greater displacement, or increased force.

[0077] In step 408, based on a second kinematic state, the system causes the focus selector to move through a second set of user interface elements at a second rate. When the kinematic profile of the second sliding motion is higher than that of the first sliding motion, the second rate is greater than the first rate, thereby causing the focus selector to traverse a larger number of user interface elements. For example... Figure 3C and Figure 3DAs shown, the kinematic profile of a swipe gesture is proportional to the number of UI elements that the scroll passes through.

[0078] (A1) In some embodiments, a non-transitory computer-readable storage medium includes a plurality of executable instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: display a plurality of user interface (UI) elements on a user interface; acquire data generated during a first sliding motion of a thumb on an index finger via one or more neuromuscular signal sensors, the first sliding motion having a first kinematic profile; and, based on the first kinematic profile, cause a focus selector to move through a first set of the plurality of UI elements at a first rate, wherein the first rate is associated with a corresponding sliding motion below an intensity threshold. These instructions also cause the one or more processors to: acquire additional data generated during a second sliding motion of a thumb on an index finger via one or more neuromuscular signal sensors, the second sliding motion having a second kinematic profile different from the first kinematic profile; and, based on the second kinematic profile, cause a focus selector to move through a second set of the plurality of UI elements at a second rate, wherein the second rate is greater than the first rate, wherein the second rate is proportional to the intensity associated with the second kinematic profile, and wherein the second set includes more UI elements than the first set. Figure 1A The UI elements 112 to 120 shown in the reference user interface 110 describe multiple UI elements, while Figure 1B The focus selector is described in reference UI selector 122. Figures 1B to 1D The image shows the thumb 134 sliding on the index finger 136 while wearing the wrist wearable device 130.

[0079] (A2) In some embodiments of A1, these instructions further cause one or more processors to: receive, via one or more neuromuscular signal sensors, further data generated by the thumb during the execution of a third sliding motion, the third sliding motion having a third kinematic profile different from the second kinematic profile; and, based on the third kinematic profile, cause the focus selector to move at a third rate through a third set of multiple UI elements, wherein the third rate is greater than the second rate, and wherein the third set includes more UI elements than the second set. This enables three or more different scrolling speeds corresponding to different gesture intensities.

[0080] (A3) In some embodiments of any of A1 to A2, the first kinematic profile and the second kinematic profile are each based on one or more of the position, velocity, acceleration, or displacement of the thumb during the respective sliding motion. (See reference...) Figures 1B to 1D The amplitude of the sliding gesture described herein can be determined based on the kinematic state of the sliding motion of the thumb 134 on the index finger 136.

[0081] (A4) In some embodiments of any of A1 to A3, the first sliding motion is in a first direction, and the second sliding motion is in a first direction. For example... Figure 1B and Figure 1C As shown, users can perform multiple swipe gestures in the same direction with different amplitudes.

[0082] (A4a) In some embodiments of any of A1 to A4, an intensity threshold is associated with a corresponding sliding motion in a first direction, and another intensity threshold is associated with a corresponding sliding motion in a direction other than the first direction. (See reference...) Figures 2A to 2C As described, the swipe gesture calibration module 208 can establish different thresholds for different swipe directions based on the natural difference in thumb movement when swiping inward and outward.

[0083] (A5) In some embodiments of any of A1 to A4, these instructions also cause one or more processors to: receive, via one or more neuromuscular signal sensors, further data generated during the thumb’s third sliding movement in a second direction opposite to the first direction; and, based on determining the third sliding movement in the second direction, perform one or more of the following: cause the focus selector to move in a direction opposite to the movement caused by the first sliding movement; or stop the focus selector from moving through multiple UI elements. Figure 1D It shows the user in the context of... Figures 1B to 1C Perform a strong swipe gesture in the opposite direction shown.

[0084] (A6) In some embodiments of any of A1 to A5, these instructions also cause one or more processors to: receive subsequent data generated during the thumb's fourth sliding motion in a third-direction direction, which is different from the first and second directions, via one or more neuromuscular signal sensors; and, based on determining that the fourth sliding motion is in a third-direction direction, cause the focus selector to move in a direction corresponding to the third-direction direction. Figure 3C and Figure 3D The swipe gestures used to navigate the calendar interface in different directions are shown.

[0085] (A7) In some embodiments of any of A1 to A6, the third direction is a diagonal direction, and moving the focus selector in the direction corresponding to the third direction includes: moving the focus selector through a two-dimensional arrangement of multiple UI elements. For example... Figure 3C and Figure 3D As shown, the calendar interface represents a two-dimensional arrangement of UI elements, where the focus selector can be moved in multiple directions through the dates.

[0086] (A8) In some embodiments of any of A1 to A7, these instructions further cause one or more processors to: detect, via one or more neuromuscular signal sensors, the force exerted by the thumb on the surface of the index finger during the first sliding motion; and, based on determining that the force exceeds a force threshold, increase a first rate of movement of the focus selector. (See reference...) Figures 2A to 2C As described, the swipe gesture calibration module 208 captures force measurements during a swipe gesture.

[0087] (A9) In some embodiments of any of A1 to A8, these instructions further cause one or more processors to: detect, via one or more neuromuscular signal sensors, a compressive force exerted by the thumb on the index finger during the first sliding motion or the second sliding motion; and, based on determining that the compressive force exceeds a compression threshold, cause to change the movement of the focus selector, wherein the change includes one or more of the following: increasing the first rate or the second rate; decreasing the first rate or the second rate; or stopping the movement of the focus selector.

[0088] (A10) In some embodiments of any of A1 to A9, these instructions also cause one or more processors to: detect a click gesture performed by the thumb via one or more neuromuscular signal sensors during a first or second sliding motion, while the thumb and index finger remain in contact; and in response to detecting the click gesture, select the UI element currently highlighted by the focus selector among a plurality of UI elements.

[0089] (A11) In some embodiments of any of A1 to A10, the click gesture is detected based on an instantaneous increase in the force exerted by the thumb on the index finger during the first or second sliding motion.

[0090] (A12) In some embodiments of any of A1 to A11, a user interface is presented on an extended reality head-mounted device that is communicatively coupled to a wrist-worn device that includes one or more of the aforementioned neuromuscular signal sensors, wherein the extended reality head-mounted device is at least one of augmented reality glasses or a mixed reality head-mounted device. (See reference...) Figures 5A to 5C-2 As described, AR device 528 or MR device 532 can present a user interface when a gesture is detected by wrist wearable device 526.

[0091] (B1) In some embodiments, a non-transitory computer-readable storage medium includes a plurality of executable instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: receive, via one or more neuromuscular signal sensors, a first set of calibration data generated during the execution of one or more swipe gestures in response to a first type of gesture cue; receive, via one or more neuromuscular signal sensors, a second set of calibration data generated during the execution of one or more other swipe gestures in response to a second type of gesture cue, the second type of gesture cue being different from the first type of gesture cue; and apply a bimodal distribution to the first set of calibration data and the second set of calibration data to determine an intensity threshold for distinguishing one or more other swipe gestures from one or more swipe gestures. Figures 2A to 2C The swipe gesture calibration module 208 is shown capturing calibration data and applying a bimodal distribution to distinguish between gentle swipe gestures and forceful swipe gestures.

[0092] (B2) In some embodiments of B1, these instructions also cause one or more processors to: display a swipe gesture calibration module, configured to determine an intensity threshold for distinguishing one or more other swipe gestures from one or more other swipe gestures, before receiving a first set of calibration data. Figure 2A The swipe gesture calibration module is described with reference to swipe gesture calibration module 208, which displays a coordinate graph and user interface 210 for guiding the user through the calibration process.

[0093] (C1) In some embodiments, a system includes: a wrist-worn wearable device including one or more neuromuscular signal sensors; and a head-worn wearable device configured to present a user interface including a plurality of user interface (UI) elements. The system is configured to: receive data generated during a first sliding motion of the thumb on the index finger via one or more neuromuscular signal sensors, the first sliding motion having a first kinematic profile; and, based on the first kinematic profile, cause a focus selector to move at a first rate through a first set of the plurality of UI elements. The system is also configured to: receive additional data generated during a second sliding motion of the thumb on the index finger via one or more neuromuscular signal sensors, the second sliding motion having a second kinematic profile different from the first kinematic profile; and, based on the second kinematic profile, cause a focus selector to move at a second rate through a second set of the plurality of UI elements, wherein the second rate is greater than the first rate, and wherein the second set includes more UI elements than the first set. See reference... Figures 5A to 5C-2 As described, the wrist-worn wearable device 526 and head-worn wearable devices such as AR device 528 or MR device 532 work together to detect gestures and present a user interface.

[0094] (C2) In some embodiments of C1, the system is further configured to: receive, via one or more neuromuscular signal sensors, further data generated by the thumb during the execution of a third sliding motion having a third kinematic profile different from the second kinematic profile; and based on the third kinematic profile, cause the focus selector to move at a third rate through a third set of multiple UI elements, wherein the third rate is greater than the second rate, and wherein the third set includes more UI elements than the second set.

[0095] (C3) In some embodiments of any of C1 to C2, the first kinematic profile and the second kinematic profile are each based on one or more of the position, velocity, acceleration or displacement of the thumb during the respective sliding motion.

[0096] (C4) In some embodiments of any of C1 to C3, the first sliding motion is in the first direction, and the second sliding motion is in the first direction.

[0097] (C5) In some embodiments of any of C1 to C4, the system is further configured to: receive further data generated during the thumb performing a third sliding movement in a second direction opposite to the first direction via one or more neuromuscular signal sensors; and, based on determining the third sliding movement in the second direction, perform one or more of the following: cause the focus selector to move in a direction opposite to the movement caused by the first sliding movement; or stop the focus selector from moving through multiple UI elements.

[0098] (C6) In some embodiments of any of C1 to C5, the system is further configured to: detect the force exerted by the thumb on the surface of the index finger during the first sliding motion by one or more neuromuscular signal sensors; and increase the first rate of movement of the focus selector based on determining that the force exceeds a force threshold.

[0099] (D1) In some embodiments, a non-transitory computer-readable storage medium includes a plurality of executable instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: display a plurality of user interface (UI) elements on a user interface; acquire data generated during the execution of an air gesture via one or more neuromuscular signal sensors; and acquire a kinematic profile of a sliding motion of the thumb on the index finger performed as part of an air gesture. The instructions also cause the one or more processors to: move a focus selector through a set of the plurality of UI elements displayed in the user interface, based on determining that the kinematic profile of the sliding motion does not satisfy one or more intensity-based navigation criteria, wherein the number of elements in the set is predefined relative to a corresponding gesture that does not satisfy one or more intensity-based navigation criteria. The instructions further cause the one or more processors to: acquire data generated during the execution of another air gesture via one or more neuromuscular signal sensors; acquire another kinematic profile of a sliding motion of the thumb on the index finger performed as part of the other air gesture; and move a focus selector through another set of the plurality of UI elements at a rate proportional to the intensity of another air gesture identified in the other kinematic profile, based on determining that the other kinematic profile of the other sliding gesture satisfies one or more intensity-based navigation criteria. like Figure 3C and Figure 3D As shown, when a user performs a swipe gesture that does not meet an intensity threshold, the focus selector (e.g., UI selector 122) moves through a predetermined number of calendar dates, regardless of the specific intensity of the gesture. In contrast, when the gesture meets intensity-based navigation criteria (e.g., having an amplitude, distance, speed, or acceleration greater than a threshold), the focus selector moves proportionally to the detected intensity, enabling more efficient navigation through longer lists of UI elements.

[0100] (D2) In some embodiments of D1, the movement rate of the focus selector corresponds to the number of another element in another set of multiple UI elements, and this other number of elements is greater than the number of elements in the set of elements through which the air gesture causes the movement. See reference... Figure 1B and Figure 1C As described, when performing a strong swipe gesture, UI selector 122 moves through a greater number of UI elements compared to performing a gentle swipe gesture.

[0101] (D3) In some embodiments of any of D1 to D2, the air gesture and another air gesture include a corresponding sliding movement of the thumb relative to the index finger in a first direction, and one or more intensity-based navigation criteria are associated with the air gesture including the movement of the thumb in the first direction. The intensity threshold is direction-specific; therefore, the criterion for determining whether a leftward swipe satisfies an intensity-based navigation criterion may differ from the criterion used for a rightward swipe. (See reference...) Figures 2A to 2C As described, the swipe gesture calibration module 208 can establish different thresholds for different swipe directions based on the natural difference in thumb movement when swiping inward and outward.

[0102] (D4) In some embodiments of any of D1 to D3, these instructions further cause one or more processors to: acquire data generated during the execution of the additional air gesture via one or more neuromuscular signal sensors; acquire additional kinematic profiles of the additional sliding motion of the thumb on the index finger performed as part of the additional air gesture, wherein the additional sliding motion of the thumb on the index finger is in a second direction different from the first direction; and cause a corresponding movement of the focus selector based on determining that the additional kinematic profile satisfies one or more other intensity-based navigation criteria associated with the air gesture including the movement of the thumb in the second direction, wherein the other one or more intensity-based navigation criteria are different from one or more intensity-based navigation criteria associated with the air gesture including the movement of the thumb in the first direction. Figure 1D It shows the user in the context of... Figures 1B to 1C Performing a swipe gesture in the opposite direction shown demonstrates direction-specific gesture recognition.

[0103] (D5) In some embodiments of any of D1 to D4, the movement rate of the focus selector is based on one or more of the speed of the swipe gesture, the force of the swipe gesture, the acceleration of the swipe gesture, and the displacement of the swipe gesture. See reference... Figures 1B to 1D The amplitude of the sliding gesture described herein can be determined based on the kinematic state of the sliding motion of the thumb 134 on the index finger 136.

[0104] (D6) In some embodiments of any of D1 to D5, these instructions further cause one or more processors to: acquire data generated during the execution of another air gesture via one or more neuromuscular signal sensors while the focus selector moves through another set of UI elements at a rate proportional to the intensity of another air gesture, the other air gesture including movement of the thumb in a direction opposite to the corresponding direction of thumb movement in the other air gesture; and stop the focus selector from moving through the corresponding UI elements of the other set of multiple UI elements. (See reference...) Figure 1DAs described, UI elements stop scrolling when the user performs a swipe gesture in the opposite direction of a previously performed swipe gesture.

[0105] (D7) In some embodiments of any of D1 to D6, the focus selector is in a moving state during the execution of another air gesture, which causes the focus selector to stop moving, regardless of whether the corresponding kinematic profile of the other air gesture satisfies one or more corresponding intensity-based navigation criteria. The gesture that stops moving is not compared to an intensity criterion; gestures in the opposite direction stop moving regardless of intensity or amplitude.

[0106] (E1) In some embodiments, a non-transitory computer-readable storage medium includes a plurality of executable instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: receive, via one or more neuromuscular signal sensors, a first set of calibration data generated during one or more swipe gestures performed in response to a first type of gesture cue; receive, via one or more neuromuscular signal sensors, a second set of calibration data generated during one or more other swipe gestures performed in response to a second type of gesture cue, the second type of gesture cue being different from the first type of gesture cue; and apply a bimodal distribution to the first set of calibration data and the second set of calibration data to determine an intensity threshold for distinguishing the one or more other swipe gestures from the one or more swipe gestures. Calibration includes detecting thumb movement in different directions and establishing a corresponding gesture profile for each direction. Figures 2A to 2C The swipe gesture calibration module 208 is shown capturing calibration data and applying a bimodal distribution to distinguish between gentle swipe gestures and forceful swipe gestures.

[0107] (E2) In some embodiments of E1, these instructions also cause one or more processors to: display a swipe gesture calibration module, which is configured to determine an intensity threshold for distinguishing one or more other swipe gestures from one or more other swipe gestures, before receiving a first set of calibration data. Figure 2A The swipe gesture calibration module is described with reference to swipe gesture calibration module 208, which displays a coordinate graph and user interface (UI) 210 for guiding the user through the calibration process.

[0108] Example Extended Reality System Figure 5A , Figure 5B , Figure 5C-1 and Figure 5C-2 An example XR system, including an AR system and an MR system, is shown according to some embodiments. Figure 5AA first XR system 500a and a first example user interaction are shown, which uses a wrist wearable device 526, a head wearable device (e.g., an AR device 528) and / or a HIPD 542. Figure 5B A second XR system 500b and a second example user interaction are shown, which uses a wrist wearable device 526, an AR device 528 and / or a HIPD 542. Figure 5C-1 and Figure 5C-2 A third MR system 500c and a third example user interaction are illustrated, which uses a wrist wearable device 526, a head wearable device (e.g., an MR device such as a VR device), and / or a HIPD 542. As those skilled in the art will understand upon reading the description provided herein, the above example AR and MR systems (described in detail below) can perform various functions and / or operations.

[0109] The wrist-worn wearable device 526, the head-worn wearable device, and / or the HIPD 542 can be communicatively coupled via a network 525 (e.g., cellular, near-field, Wi-Fi, personal area network, or wireless local area network (LAN)). Additionally, the wrist-worn wearable device 526, the head-worn wearable device, and / or the HIPD 542 can also be communicatively coupled via the network 525 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless LAN) to one or more servers 530, one or more computers 540 (e.g., laptops or computers), one or more mobile devices 550 (e.g., smartphones, tablets), and / or one or more other electronic devices. Similarly, textile-based smart clothing, when used, can also be communicatively coupled via the network 525 to the wrist-worn wearable device 526, one or more head-worn wearable devices, the HIPD 542, one or more servers 530, one or more computers 540, one or more mobile devices 550, and / or one or more other electronic devices.

[0110] Go to Figure 5AThe illustration shows a user 502 wearing a wrist-worn wearable device 526 and an AR device 528, with a HIPD 542 placed on their table. The wrist-worn wearable device 526, AR device 528, and HIPD 542 facilitate user interaction with the AR environment. Specifically, as shown in the first XR system 500a, the wrist-worn wearable device 526, AR device 528, and / or HIPD 542 enable the presentation of one or more avatars 504, digital representations of one or more contacts 506, and virtual objects 508. As discussed below, the user 502 can interact with the one or more avatars 504, the digital representations of one or more contacts 506, and the one or more virtual objects 508 through the wrist-worn wearable device 526, AR device 528, and / or HIPD 542. Additionally, the user 502 can directly view physical objects in the environment, such as a physical table 529, through one or more transparent lenses and one or more waveguides of the AR device 528. Alternatively, an MR device can be used instead of an AR device 528, and a similar user experience can be produced, but the user will not directly see physical objects in the environment (e.g., table 529), but will instead be presented with a virtual reconstruction of table 529 generated from one or more sensors of the MR device (e.g., an outward-facing camera capable of recording the surrounding environment).

[0111] User 502 may provide user input using any of the following: wrist wearable device 526, AR device 528 (e.g., through physical input at the AR device and / or built-in motion tracking of the user's limbs), smart textile apparel, externally mounted limb tracking device, or HIPD 542. For example, user 502 may perform one or more gestures detected by wrist wearable device 526 (e.g., using one or more EMG sensors and / or IMUs built into the wrist wearable device) and / or AR device 528 (e.g., using one or more image sensors or cameras) to provide user input. Alternatively or additionally, user 502 may provide user input via: one or more touch surfaces of wrist wearable device 526, AR device 528, and / or HIPD 542; and / or voice commands captured by the microphones of wrist wearable device 526, AR device 528, and / or HIPD 542. The wrist-worn wearable device 526, AR device 528, and / or HIPD 542 include an artificial intelligence digital assistant to help the user provide user input (e.g., completing a sequence of actions, suggesting different actions or commands, providing reminders, confirming commands). For example, the digital assistant can be invoked by input occurring at AR device 528 (e.g., input via the temple of AR device 528). In some embodiments, the user 502 can provide user input via one or more facial gestures and / or facial expressions. For example, the cameras of the wrist-worn wearable device 526, AR device 528, and / or HIPD 542 can track the eyes of the user 502 for navigating the user interface.

[0112] The wrist-worn wearable device 526, AR device 528, and / or HIPD 542 can operate individually or in combination to allow user 502 to interact with the AR environment. In some embodiments, HIPD 542 is configured to operate as a central hub or control center for the following devices: wrist-worn wearable device 526; AR device 528; and / or another communication-coupled device. For example, user 502 can provide input for interacting with the AR environment at any of the wrist-worn wearable device 526, AR device 528, and / or HIPD 542, and HIPD 542 can identify one or more backend and frontend tasks to perform the requested interaction and distribute instructions to execute the one or more backend and frontend tasks at the wrist-worn wearable device 526, AR device 528, and / or HIPD 542. In some embodiments, backend tasks are user-insensible background processing tasks (e.g., rendering content, decompression, compression, application-specific operations), while frontend tasks are user-insensible user-facing tasks (e.g., presenting information to the user or providing feedback to the user). HIPD 542 can perform backend tasks and provide operational data corresponding to the backend tasks performed to the wrist wearable device 526 and / or AR device 528, enabling the wrist wearable device 526 and / or AR device 528 to perform frontend tasks. In this way, HIPD 542 (which has more computing resources and a larger thermal headroom than the wrist wearable device 526 and / or AR device 528) performs computationally intensive tasks and reduces the computing resource utilization and / or power consumption of the wrist wearable device 526 and / or AR device 528.

[0113] In the example shown in the first XR system 500a, HIPD 542 identifies one or more backend and frontend tasks associated with a user request to initiate an AR video call with one or more other users (represented by avatar 504 and contact digital representation 506); and HIPD 542 issues instructions to execute the one or more backend and frontend tasks. Specifically, HIPD 542 performs backend tasks for processing and / or rendering image data (and other data) associated with the AR video call and provides operational data associated with the performed backend tasks to AR device 528, causing AR device 528 to perform frontend tasks for presenting the AR video call (e.g., presenting avatar 504 and contact digital representation 506).

[0114] In some embodiments, HIPD 542 can function as a focus or anchor point for presenting information. This allows user 502 to generally know where the information is presented. For example, as shown in the first XR system 500a, an avatar 504 and a digital representation 506 of a contact are presented above HIPD 542. Specifically, HIPD 542 and AR device 528 operate in conjunction to determine the location for presenting avatar 504 and the digital representation 506 of the contact. In some embodiments, information can be presented at a predetermined distance from HIPD 542 (e.g., within 5 meters). For example, as shown in the first XR system 500a, a virtual object 508 is presented on a table at a distance from HIPD 542. Similar to the examples above, HIPD 542 and AR device 528 can operate in conjunction to determine the location for presenting virtual object 508. Alternatively, in some embodiments, the presentation of information is not constrained by HIPD 542. More specifically, the avatar 504, the digital representation of the contact 506, and the virtual object 508 do not need to be presented within the predetermined distance of the HIPD 542. Although the AR device 528 is described as working with the HIPD, the MR headset can interact in the same way as the AR device 528.

[0115] The user input provided at the wrist wearable device 526, AR device 528, and / or HIPD 542 is coordinated to enable the user to initiate, continue, and / or complete an operation using any device. For example, user 502 can provide user input to AR device 528 to cause AR device 528 to present virtual object 508, and while AR device 528 presents virtual object 508, user 502 can provide one or more gestures via wrist wearable device 526 to interact with and / or manipulate virtual object 508. Although AR device 528 is described as working in conjunction with wrist wearable device 526, MR headsets can interact in the same manner as AR device 528.

[0116] Integration of Artificial Intelligence and XR Systems Figure 5A An interaction is illustrated where an AI virtual assistant can assist with requests made by user 502. The AI ​​virtual assistant can be used to fulfill open-ended requests made by user 502 through natural language input. For example, in... Figure 5A In this scenario, user 502 issues an auditory request 544 to summarize the conversation and then shares the summarized conversation with others in the meeting. Additionally, the AI ​​virtual assistant is configured to use sensors from the XR system (e.g., the camera and microphone of the XR headset, and various other sensors from any other device in the system) to provide contextual cues to the user to initiate tasks.

[0117] Figure 5A An example neural network 552 used in artificial intelligence applications is also shown. The uses of Artificial Intelligence (AI) are diverse and encompass many different aspects of the devices and systems described herein. AI capabilities cover a wide range of applications and enhance the interaction between user 502 and user devices (e.g., AR device 528, MR device 532, HIPD 542, wrist-worn wearable device 526). The AI ​​discussed in this paper can be obtained using many different training techniques. Although the primary AI model example discussed in this paper is a neural network, other AI models can also be used. Non-limiting examples of AI models include artificial neural networks (ANNs), deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), large language models (LLMs), long short-term memory networks, transformer models, decision trees, random forests, support vector machines, k-nearest neighbors, genetic algorithms, Markov models, Bayesian networks, fuzzy logic systems, and deep reinforcement learning, etc. The AI ​​models can be implemented on one or more user devices and / or any other devices among the user devices described in this paper. For devices and systems employing multiple AI models as described in this paper, different models can be used depending on the task. For example, an LLM can be used for a natural language AI virtual assistant, while a DNN can be used alternatively for object detection in a physical environment.

[0118] In another example, an AI virtual assistant may include many different AI models, and multiple AI models may be employed (concurrently, sequentially, or a combination of concurrent and sequential employment) based on the user's request. For example, an LLM-based AI model may provide instructions to help the user follow a recipe, and these instructions may be based in part on another AI model derived from ANNs, DNNs, RNNs, etc., capable of identifying which part of the recipe the user is performing (e.g., object and scene detection).

[0119] As AI training models evolve, the operations and experiences described herein may be performed using models different from those listed above, and those skilled in the art will understand that the list above is non-limiting.

[0120] User 502 can interact with the AI ​​model through natural language input, text input, or any other input modality that accepts natural language and / or the corresponding sound sensor module, captured by a sound sensor. In another instance, input is provided by tracking the gaze of user 502 via a gaze tracker module. Additionally, the AI ​​model can receive input beyond that provided by user 502. For example, the AI ​​can further generate its responses based on environmental input captured by various types of sensors and / or their corresponding sensor modules in response to user requests (e.g., temperature data, image data, video data, ambient light data, audio data, GPS location data, inertial measurement results (i.e., user motion) data, pattern recognition data, magnetometer data, depth data, pressure data, force data, neuromuscular data, heart rate data, temperature data, sleep data). Sensor data can be retrieved entirely from a single device (e.g., AR device 528) or from multiple devices communicating with each other (e.g., a system including at least two of the following: AR device 528, MR device 532, HIPD 542, wrist-worn wearable device 526, etc.). The AI ​​model can also access additional information (e.g., one or more servers 530, one or more computers 540, one or more mobile devices 550 and / or one or more other electronic devices) via network 525.

[0121] A non-limiting list of AI enhancements includes, but is not limited to, image recognition, speech recognition (e.g., automatic speech recognition), text recognition (e.g., scene text recognition), pattern recognition, natural language processing and understanding, classification, regression, clustering, anomaly detection, sequence generation, content generation, and optimization. In some embodiments, the AI ​​enhancements are executed, wholly or partially, on a cloud computing platform coupled to a user device (e.g., AR device 528, MR device 532, HIPD 542, wrist-worn wearable device 526) via one or more network communications. The cloud computing platform provides scalable computing resources, distributed computing, managed AI services, perturbation acceleration, pre-trained models, APIs, and / or other resources to support the full computational requirements of the AI ​​enhancements.

[0122] Example outputs derived from using AI models may include natural language responses, mathematical calculations, charts displaying information, audio, images, videos, text, meeting minutes, predictions based on environmental factors, classification, pattern recognition, recommendations, evaluations, or other operations. In some embodiments, the generated outputs are stored on the local memory of a user device (e.g., AR device 528, MR device 532, HIPD 542, wrist-worn wearable device 526), ​​storage options of external devices (servers, computers, mobile devices, etc.), and / or storage options of a cloud computing platform.

[0123] AI-based outputs can be presented across different modalities (e.g., audio-based modalities, vision-based modalities, haptic modalities, and any combination thereof) and across different devices within the XR system described herein. Some vision-based outputs may include XR enhancements on XR headsets, or information displayed on user interfaces on wrist-worn wearables, laptop devices, mobile devices, etc. On devices with or without displays (e.g., HIPD 542), haptic feedback may provide information to user 502. The AI ​​model may also use the aforementioned inputs to determine the appropriate modality for presenting content to the user and one or more devices (e.g., audio output may be presented instead of visual output to a user walking on a busy road to avoid distracting user 502).

[0124] Example of augmented reality interaction Figure 5B The illustration shows a user 502 wearing a wrist-worn wearable device 526 and an AR device 528, and holding a HIPD 542. In the second AR system 500b, the wrist-worn wearable device 526, the AR device 528, and / or the HIPD 542 are used to receive one or more messages and / or provide one or more messages to the user 502's contacts. Specifically, the wrist-worn wearable device 526, the AR device 528, and / or the HIPD 542 detect and coordinate one or more user inputs to initiate a messaging application and prepare to reply to messages received through the messaging application.

[0125] In some embodiments, user 502 launches an application on wrist wearable device 526, AR device 528, and / or HIPD 542 via user input, thereby launching the application on at least one device. For example, in the second XR system 500b, user 502 performs a gesture associated with a command to launch a messaging application (represented by messaging user interface 512); wrist wearable device 526 detects the gesture and, based on determining that user 502 is wearing AR device 528, causes AR device 528 to present the messaging user interface 512 of the messaging application. AR device 528 may present the messaging user interface 512 to user 502 via its display (e.g., as shown in user 502's field of view 510). In some embodiments, the application is launched and may run on a device (e.g., wrist wearable device 526, AR device 528, and / or HIPD 542) that detects user input to launch the application, and that device provides operational data to another device to cause the messaging application to be presented. For example, the wrist-worn wearable device 526 can detect user input to launch a messaging application, launch and run the messaging application, and provide operational data to the AR device 528 and / or HIPD 542 to render the messaging application. Alternatively, the application can be launched and run on a different device than the one that detected the user input. For example, the wrist-worn wearable device 526 can detect gestures associated with launching the messaging application and can enable the HIPD 542 to run the messaging application and coordinate its rendering.

[0126] Furthermore, user 502 can provide user input at the wrist wearable device 526, AR device 528, and / or HIPD 542 to continue and / or complete an operation initiated at another device. For example, after launching a messaging application via the wrist wearable device 526 and while the messaging user interface 512 is displayed on the AR device 528, user 502 can provide input at HIPD 542 to prepare a reply (e.g., indicated by a swipe gesture performed on HIPD 542). The gesture performed by user 502 on HIPD 542 can be provided and / or displayed on another device. For example, a swipe gesture performed by user 502 on HIPD 542 can be displayed on the virtual keyboard of the messaging user interface 512 displayed by AR device 528.

[0127] In some embodiments, the wrist wearable device 526, AR device 528, HIPD 542, and / or other communication-coupled devices may present one or more notifications to the user 502. The notification may be an indication of a new message, incoming call, application update, or status update, etc. The user 502 may select a notification via the wrist wearable device 526, AR device 528, or HIPD 542, causing an application or action associated with the notification to be presented on at least one device. For example, the user 502 may receive a notification of a received message at the wrist wearable device 526, AR device 528, HIPD 542, and / or another communication-coupled device, and provide user input at the wrist wearable device 526, AR device 528, and / or HIPD 542 to view the notification. The device that detects the user input may cause the application associated with the notification to be launched and / or the application associated with the notification to be presented on the wrist wearable device 526, AR device 528, and / or HIPD 542.

[0128] While the examples above describe coordinated input for interaction with messaging applications, those skilled in the art will recognize upon reading this description that user input can be coordinated to interact with any number of applications, including but not limited to gaming applications, social media applications, camera applications, web-based applications, and financial applications. For example, AR device 528 can present gaming application data to user 502, while HIPD 542 can use a controller to provide input to the game. Similarly, user 502 can use wrist-worn wearable device 526 to activate the camera of AR device 528, and the user can use wrist-worn wearable device 526, AR device 528, and / or HIPD 542 to manipulate image acquisition (e.g., zoom in or out, or apply filters) and capture image data.

[0129] While AR device 528 is shown to be capable of certain functions, it should be understood that AR devices can be AR devices with different functions based on cost and market demand. For example, an AR device may include a single output modality such as an audio output modality. In another example, an AR device may include a low-fidelity display as one of the output modalities, capable of presenting simple information (e.g., text and / or low-fidelity images / videos) to the user. In yet another example, an AR device may be configured with face-facing light-emitting diodes (LEDs) configured to provide information to the user; for example, when providing direction, an LED around the right lens may illuminate to notify the wearer to turn right, or an LED on the left lens may illuminate to notify the wearer to turn left. In another embodiment, an AR device may include an outward-facing projector, enabling the display of information (e.g., text information, media) on the user's palm or other suitable surface (e.g., a table, a whiteboard). In yet another embodiment, information may also be provided by locally darkening portions of the lens to emphasize portions of the environment to which the user's attention should be directed. Some AR devices can render AR augmentations in a monocular or binocular manner (e.g., AR augmentations can be rendered on a single display associated with a single lens, rather than at two lenses to produce a binocular image). In some instances, AR devices capable of rendering AR augmentations in a binocular manner may also optionally display AR augmentations in a monocular manner (e.g., for energy-saving purposes or other presentation considerations). These examples are not exhaustive, and features of one AR device described above can be combined with features of another AR device described above. Although the features and experiences of AR devices have been described in general terms in the preceding sections, it should be understood that the described features and experiences can be applied in a similar manner to MR headsets, which will be described in the following sections.

[0130] Example of mixed reality interaction Go to Figure 5C-1 and Figure 5C-2The illustration shows a user 502 wearing a wrist-worn wearable device 526 and an MR device 532 (e.g., a device capable of providing a full VR or MR experience, which displays one or more objects from the physical environment on the device's display) and holding a HIPD 542. In the third MR system 500c, the wrist-worn wearable device 526, the MR device 532, and / or the HIPD 542 are used to interact within an MR environment (e.g., a VR game or other MR / AR application). Although the MR device 532 presents a representation of the VR game to the user 502 (e.g., a first MR game environment 520), the wrist-worn wearable device 526, the MR device 532, and / or the HIPD 542 detect and coordinate one or more user inputs to allow the user 502 to interact with the VR game.

[0131] In some embodiments, user 502 may provide user input that elicits movement in the corresponding MR environment via wrist-worn wearable device 526, MR device 532, and / or HIPD 542. For example, a third MR system 500c (such as...) Figure 5C-1 In the illustration, user 502 raises HIPD 542 in preparation for swinging it in the first MR game environment 520. MR device 532 responds to user 502 raising HIPD 542 by causing user's MR representation 522 to perform a similar action (e.g., raising a virtual object, such as virtual sword 524). In some embodiments, each device uses corresponding sensor data and / or image data to detect user input and provide an accurate representation of user 502's movement. For example, the imaging sensor of HIPD 542 (e.g., a Simultaneous Localization and Mapping (SLAM) camera or other camera) can be used to detect the position of HIPD 542 relative to user 502's body, allowing virtual objects to be properly positioned within the first MR game environment 520; sensor data from wrist-worn wearable device 526 can be used to detect the speed at which user 502 raises HIPD 542, synchronizing user's MR representation 522 and virtual sword 524 with user 502's movement; and the image sensor of MR device 532 can be used to represent user 502's body, boundary conditions, or real-world objects within the first MR game environment 520.

[0132] exist Figure 5C-2In this scenario, user 502 performs a downward swing while holding HIPD 542. Wrist wearable device 526, MR device 532, and / or HIPD 542 detect the downward swing by user 502, and the corresponding action is performed within the first MR gaming environment 520. In some embodiments, data captured by each device is used to enhance the user's experience within the MR environment. For example, sensor data from wrist wearable device 526 may be used to determine the speed and / or force of the downward swing, and image sensors from HIPD 542 and / or VR device 532 may be used to determine the location of the swing and how it should be represented within the first MR gaming environment 520. This can then be used as input to the MR environment (e.g., a game mechanic that can classify the user's input using aspects of the detected speed, force, location, and / or user 502's action (e.g., the user performs a tap, a hit, a critical strike, a glancing strike, a miss) or can calculate an output (e.g., damage amount)).

[0133] Figure 5C-2 It is also shown that, when displaying the MR game environment 520, a portion of the physical environment is reconstructed and displayed on the monitor of the MR device 532. In this example, when one or more objects in the physical environment are potentially in the user's path (e.g., the user is likely to collide with objects in the physical environment), a reconstruction 546 of the physical environment is displayed instead of a portion of the MR game environment 520. Thus, this example MR game environment 520 includes: (i) an immersive VR portion 548 (e.g., an environment that has no necessary counterpart in the nearby physical environment); and (ii) a reconstruction 546 of the physical environment (e.g., a table 550 and a cup 552). Although the example shown here is an MR environment that displays a reconstruction of the physical environment to avoid collisions, other uses of the reconstruction of the physical environment can be employed, such as defining the characteristics of the virtual environment based on the surrounding physical environment (e.g., virtual pillars can be placed based on objects in the surrounding physical environment, such as trees).

[0134] Although the wrist-worn wearable device 526, MR device 532, and / or HIPD 542 are described as detecting user input, in some embodiments, user input is detected at a single device (where this single device is responsible for distributing signals to other devices for executing the user input). For example, HIPD 542 may run an application for generating a first MR game environment 520 and provide corresponding data to MR device 532 for rendering the first MR game environment 520, as well as detect movement of user 502 (while holding HIPD 542) to cause corresponding actions to be performed within the first MR game environment 520. Additionally or alternatively, in some embodiments, operational data (e.g., sensor data, image data, application data, device data, and / or other data) of one or more devices is provided to a single device (e.g., HIPD 542) for processing, causing the corresponding device to perform an action associated with the processed operational data.

[0135] In some embodiments, user 502 may wear a wrist-worn wearable device 526, wear an MR device 532, wear a textile-based smart garment 538 (e.g., a wearable haptic glove), and / or hold a HIPD 542 device. In this embodiment, the wrist-worn wearable device 526, the MR device 532, and / or the textile-based smart garment 538 are used in an MR environment (e.g., as referenced above). Figure 5A and Figure 5B Interaction within any AR or MR system described. Although MR device 532 presents a representation of an MR game (e.g., a second MR game environment 520) to user 502, wrist wearable device 526, MR device 532, and / or textile-based smart clothing 538 detect and coordinate one or more user inputs to allow user 502 to interact with the MR environment.

[0136] In some embodiments, user 502 may provide user input that elicits movement in the corresponding MR environment via wrist wearable device 542, MR device 532, and / or textile-based smart clothing 538. In some embodiments, each device uses corresponding sensor data and / or image data to detect user input and provide an accurate representation of user 502's movement. Although four different input devices are shown (e.g., wrist wearable device 526, MR device 532, HIPD 542, and textile-based smart clothing 538), each of these input devices can provide input for full interaction with the MR environment completely independently. For example, the wrist wearable device itself can provide sufficient input for interaction with the MR environment. In some embodiments, if multiple input devices are used (e.g., wrist wearable device and textile-based smart clothing 538), sensor fusion can be used to ensure that the input is correct. Although multiple input devices are described, it should be understood that other input devices, such as, but not limited to, external motion-tracking cameras, other wearable devices mounted on different parts of the user's body, devices that allow the user to experience walking in the MR environment while remaining substantially still in the physical environment, etc., can be used in combination or individually.

[0137] As described above, the data captured by each device is used to enhance the user's experience within the MR environment. Although not shown, the textile-based smart clothing 538 can be used in conjunction with MR devices and / or HIPD 542.

[0138] Although some experiences are described as taking place on AR devices and others on MR devices, those skilled in the art will understand that experiences can be transferred from MR devices to AR devices and vice versa.

[0139] For ease of reference, some definitions of the following devices and components are defined herein: these devices and components may be included in some or all of the example devices discussed. It will be understood by those skilled in the art that certain types of components described may be more suitable for a particular set of devices and less suitable for different sets of devices. However, subsequent references to components defined herein should be considered as being covered by the definitions provided.

[0140] In some embodiments, several example devices and systems, including electronic devices and systems, will be discussed. These example devices and systems are not intended to be limiting, and those skilled in the art will understand that alternative devices and systems to the example devices and systems described herein can be used to perform the operations described herein and to construct the systems and devices described herein.

[0141] As described herein, an electronic device is a device that uses electrical energy to perform a specific function. Such an electronic device can be any physical object containing electronic components, such as transistors, resistors, capacitors, diodes, and integrated circuits. Examples of electronic devices include smartphones, laptops, digital cameras, televisions, game consoles, and music players, as well as the example electronic devices discussed herein. As described herein, an intermediate electronic device is a device that is situated between two other electronic devices and / or between subsets of multiple components of one or more electronic devices, and facilitates communication, and / or data processing, and / or data transmission between the respective electronic devices and / or electronic components.

[0142] The above provided Figures 5A to 5C-2 The foregoing description is intended to expand the reference. Figures 1A to 3D The following description is provided. Although the terminology used in the following description may differ from that used in the preceding description, those skilled in the art will understand that the terms have the same meaning.

[0143] Any data collection performed by the devices described herein and / or by any device configured to perform or cause to perform the different embodiments described above with reference to any of the accompanying drawings (hereinafter referred to as "devices") is conducted with the user's consent and in a manner that complies with all applicable privacy laws. Users are provided with options to allow the devices to collect data and options to restrict or refuse the devices' collection of data. Users can choose to enable or disable any data collection at any time. Furthermore, users are provided with the option to request the deletion of any collected data.

[0144] It will be understood that although the terms “first,” “second,” etc., may be used in this document to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0145] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an,” and “the / said” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that when the terms “comprising” and / or “including” are used in this specification, they specify the presence of the stated features, uniformities, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, uniformities, steps, operations, elements, components, and / or groups thereof.

[0146] As used herein, depending on the context, the term "if" can be interpreted as meaning: "when the stated conditional precedent is true"; or "once" the stated conditional precedent is true; or "in response to determination" the stated conditional precedent is true; or "according to determination" the stated conditional precedent is true; or "in response to detection" the stated conditional precedent is true. Similarly, depending on the context, the phrases "if it is determined [the stated conditional precedent is true]" or "if [the stated conditional precedent is true]" or "when [the stated conditional precedent is true]" can be interpreted as meaning "once" or "in response to determination" or "according to determination", "once" or "in response to detection" the stated conditional precedent is true.

[0147] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of operation and practical application, thereby enabling others skilled in the art to implement them.

Claims

1. A non-transitory computer-readable storage medium comprising executable instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: Display multiple user interface (UI) elements on the user interface; Data generated during a first sliding motion of the thumb on the index finger is acquired via one or more neuromuscular signal sensors, the first sliding motion having a first kinematic profile; Based on the first kinematic profile, the focus selector moves through a first set of the plurality of UI elements at a first rate, wherein the first rate is associated with a corresponding sliding motion below an intensity threshold; Additional data generated during a second sliding motion of the thumb on the index finger, the second sliding motion having a second kinematic profile different from the first kinematic profile, is acquired via the one or more neuromuscular signal sensors; and Based on the second kinematic profile, the focus selector moves through a second set of the plurality of UI elements at a second rate, wherein the second rate is greater than the first rate, and wherein the second rate is proportional to the intensity associated with the second kinematic profile, and wherein the second set includes more UI elements than the first set.

2. The non-transitory computer-readable storage medium of claim 1, further comprising executable instructions that cause the one or more processors to perform the following operations: Further data generated during the thumb's third sliding motion, having a third kinematic profile different from the second kinematic profile, is received via the one or more neuromuscular signal sensors; and Based on the third kinematic profile, the focus selector moves at a third rate through a third set of the plurality of UI elements, wherein, The third rate is greater than the second rate, and the third set includes more UI elements than the second set.

3. The non-transitory computer-readable storage medium according to claim 1, wherein, The first kinematic profile and the second kinematic profile are each based on one or more of the position, velocity, acceleration or displacement of the thumb during the corresponding sliding motion.

4. The non-transitory computer-readable storage medium according to claim 1, wherein, The first sliding motion is in a first direction, and the second sliding motion is in the first direction.

5. The non-transitory computer-readable storage medium according to claim 4, wherein: The intensity threshold is associated with the corresponding sliding motion in the first direction, and Another intensity threshold is associated with the corresponding sliding motion in a direction different from the first direction.

6. The non-transitory computer-readable storage medium of claim 4, further comprising executable instructions that cause the one or more processors to perform the following operations: Further data generated during the thumb's third sliding motion in a second direction opposite to the first direction is received via the one or more neuromuscular signal sensors; and Based on the determination that the third sliding motion is in the second direction, one or more of the following are performed: This causes the focus selector to move in the opposite direction to the movement caused by the first sliding motion; or Stop the focus selector from moving through the multiple UI elements.

7. The non-transitory computer-readable storage medium of claim 6, further comprising executable instructions that cause the one or more processors to perform the following operations: The system receives subsequent data generated during the thumb's fourth sliding motion in a third-direction direction, which differs from both the first and second directions, via the one or more neuromuscular signal sensors; and... Based on the determination of the fourth sliding motion in the third direction, the focus selector moves in the direction corresponding to the third direction.

8. The non-transitory computer-readable storage medium according to claim 7, wherein, The third direction is a diagonal direction, and wherein moving the focus selector in the direction corresponding to the third direction includes moving the focus selector through a two-dimensional arrangement of the plurality of UI elements.

9. The non-transitory computer-readable storage medium of claim 1, further comprising executable instructions that cause the one or more processors to perform the following operations: The force exerted by the thumb on the surface of the index finger during the first sliding motion is detected by the one or more neuromuscular signal sensors; and Based on the determination that the force exceeds the force threshold, the first rate of movement of the focus selector is increased.

10. The non-transitory computer-readable storage medium of claim 1, further comprising executable instructions that cause the one or more processors to perform the following operations: The compressive force exerted by the thumb on the index finger during the first or second sliding motion is detected using the one or more neuromuscular signal sensors; and Based on the determination that the compressive force exceeds a compression threshold, the movement of the focus selector is changed, wherein... The change includes one or more of the following: increasing the first rate or the second rate; decreasing the first rate or the second rate; or stopping the movement of the focus selector.

11. The non-transitory computer-readable storage medium of claim 1, further comprising executable instructions that cause the one or more processors to perform the following operations: During the first or second sliding motion, while the thumb and index finger remain in contact, a clicking gesture performed by the thumb is detected by the one or more neuromuscular signal sensors; and In response to the detection of the click gesture, select the UI element currently highlighted by the focus selector among the plurality of UI elements.

12. The non-transitory computer-readable storage medium according to claim 11, wherein, The click gesture is detected based on a momentary increase in the force applied by the thumb to the index finger during the first or second sliding motion.

13. The non-transitory computer-readable storage medium according to claim 1, wherein, The user interface is presented on an augmented reality head-mounted device, which is communicatively coupled to a wrist-worn wearable device, which includes one or more neuromuscular signal sensors, wherein the augmented reality head-mounted device is at least one of augmented reality glasses or a mixed reality head-mounted device.

14. A non-transitory computer-readable storage medium comprising executable instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: Receive, via one or more neuromuscular signal sensors, a first set of calibration data generated during the execution of one or more swipe gestures in response to a first type of gesture cue; A second set of calibration data generated during the execution of one or more other swipe gestures in response to a second type of gesture cue, which is different from the first type of gesture cue, is received via the one or more neuromuscular signal sensors. as well as A bimodal distribution is applied to the first set of calibration data and the second set of calibration data to determine an intensity threshold for distinguishing the one or more other swipe gestures from the one or more other swipe gestures.

15. The non-transitory computer-readable storage medium of claim 14, further comprising executable instructions that cause the one or more processors to perform the following operations: Before receiving the first set of calibration data, a swipe gesture calibration module is displayed, the swipe gesture calibration module being configured to determine the intensity threshold used to distinguish the one or more other swipe gestures from the one or more swipe gestures.

16. A system comprising: A wrist-worn wearable device, the wrist-worn wearable device including one or more neuromuscular signal sensors; as well as A head-mounted wearable device configured to present a user interface, the user interface including multiple user interface (UI) elements; The system is configured as follows: Data generated during a first sliding motion of the thumb on the index finger is received via one or more neuromuscular signal sensors, the first sliding motion having a first kinematic profile; Based on the first kinematic profile, the focus selector moves at a first rate through a first set of the plurality of UI elements; Additional data generated during a second sliding motion of the thumb on the index finger, having a second kinematic profile different from the first kinematic profile, is received via the one or more neuromuscular signal sensors; and Based on the second kinematic profile, the focus selector moves through a second set of the plurality of UI elements at a second rate, wherein the second rate is greater than the first rate, and wherein the second set includes more UI elements than the first set.

17. The system according to claim 16, wherein, The system is also configured to: Further data generated by the thumb during a third sliding motion, which has a third kinematic profile different from the second kinematic profile, is received via the one or more neuromuscular signal sensors; and Based on the third kinematic profile, the focus selector moves through a third set of the plurality of UI elements at a third rate, wherein the third rate is greater than the second rate, and wherein the third set includes more UI elements than the second set.

18. The system according to claim 16, wherein, The first kinematic profile and the second kinematic profile are each based on one or more of the position, velocity, acceleration or displacement of the thumb during the corresponding sliding motion.

19. The system according to claim 16, wherein, The first sliding motion is in a first direction, and wherein the second sliding motion is in the first direction, and the system is further configured to: Further data generated during the thumb's third sliding motion in a second direction opposite to the first direction is received via the one or more neuromuscular signal sensors; and Based on the determination that the third sliding motion is in the second direction, one or more of the following are performed: This causes the focus selector to move in the opposite direction to the movement caused by the first sliding motion; or Stop the focus selector from moving through the multiple UI elements.

20. The system of claim 16, wherein, The system is also configured to: The force exerted by the thumb on the surface of the index finger during the first sliding motion is detected by the one or more neuromuscular signal sensors. and Based on the determination that the force exceeds the force threshold, the first rate of movement of the focus selector is increased.

Images