Managing indicator movement using gaze-assisted ray cast

Abstract

According to at least one implementation, a method includes identifying a gaze associated with a user of a device and determining a first location for an indicator based on the gaze. The method further includes determining a second location for the indicator in response to the gaze moving outside of an area around the first location.

Description

MANAGING INDICATOR MOVEMENT USING GAZE-ASSISTED RAY CASTCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 568,921, filed March 22, 2024, entitled “MANAGING CURSOR MOVEMENT USING GAZE-ASSISTED RAY CAST,” the disclosure of which is incorporated herein by reference in its entirety.BACKGROUND

[0002] Gesture control using body movements is a technology that enables humans to interact with digital devices or systems through physical gestures without needing to touch hardware interfaces like keyboards, mice, or screens. This technology captures and interprets human motions, which are then converted into commands executed by the computer or device. It encompasses various applications, from consumer electronics to more complex systems like interactive installations and virtual reality environments. For example, a user may use a hand or arm as a cursor to select and manage content on a display without the use of a remote or other control device.SUMMARY

[0003] This disclosure relates to systems and methods for managing indicator movement using gaze and body portions. In at least one implementation, a (head-worn) device can be configured to determine a gaze associated with a user of the device and determine a first location for an indicator based on the gaze. The device can further determine when the gaze moves outside of an area around the first location and choose a second location for the indicator based on the gaze moving outside of the area or beyond a threshold distance from the first location. In some implementations, the device can visually migrate the indicator to the second location. In some implementations, the device can remove the indicator from the display (e.g., in the first area) and display the indicator at the second location. In some implementations, the indicator will move to the second location in response to a movement of a body portion toward the secondlocation. In some implementations, the user can use a body portion to move the indicator from the first and second locations. In at least one implementation, the device can display the indicator at the first location (i.e., an anchor). The user can then move the indicator from the first location using body movements, such as hand movements. The indicator can then be moved to the second location (i.e., second anchor) based on the gaze moving outside of an area around the first location.

[0004] In some aspects, the techniques described herein relate to a method including: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving beyond a threshold distance from the first location.

[0005] In some aspects, the techniques described herein relate to a computing system including: a computer-readable storage medium; at least one processor operatively coupled to the computer-readable storage medium; and program instructions stored on the computer-readable storage medium that, when executed by the at least one processor, direct the computing system to perform a method, the method including: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving outside of an area around the first location.

[0006] In some aspects, the techniques described herein relate to a computer-readable storage medium having program instructions stored thereon that, when executed by at least one processor, direct the at least one processor to perform a method, the method including: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving outside of an area around the first location.

[0007] The details of one or more implementations are outlined in the accompanying drawings and the description below. Other features will be apparent from the description and drawings and the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIG. 1 A illustrates a user perspective of using gaze-assisted input according to an implementation.

[0009] FIG. IB illustrates a user perspective of using gaze-assisted input according to an implementation.

[0010] FIG. 2 illustrates a method of using gaze-assisted input according to an implementation.

[0011] FIG. 3 A illustrates a user perspective of using gaze to move an indicator according to an implementation.

[0012] FIG. 3B illustrates a user perspective of using gaze to move an indicator according to an implementation.

[0013] FIG. 3C illustrates a user perspective of using gaze to move an indicator according to an implementation.

[0014] FIG. 4 illustrates a method of using gaze to move an indicator according to an implementation.

[0015] FIG. 5A illustrates a user perspective of gaze-dependent speed for an indicator according to an implementation.

[0016] FIG. 5B illustrates a user perspective of gaze-dependent speed for an indicator according to an implementation.

[0017] FIG. 6 illustrates a method of providing gaze-dependent speed for an indicator according to an implementation.

[0018] FIG. 7 illustrates a computing system that displays an indicator according to an implementation.DETAILED DESCRIPTION

[0019] In some devices, gesture control is used to interact with digital devices and systems without touching hardware interfaces, such as mice, keyboards, and screens. These products may include wearable devices, gaming consoles, virtual reality (VR) or augmented reality (VR or AR) headsets (examples of extended reality (XR) headsets), smart televisions and streaming devices, computers, and laptops, amongst other types of devices. To support gesture control, a digital device uses cameras, infrared sensors, motion sensors, or some other sensor to capture the motion of the user. Once captured, image or signal processing is applied to the motion to determine a desired action for the user. The actions may include selecting an item in a menu, scrolling a menu, manipulating a menu, manipulating an object presented by the device, orsome other action. For example, a user of an XR headset may use a hand motion to select an option offered on the display of the XR headset. However, at least one technical problem exists in effectively providing input via gestures across a large display space.

[0020] As at least one technical solution, in addition to monitoring the user’s hand or arm gestures, the device further identifies the gaze or eye movement of the user to determine the desired location of an indicator, such as cursor displayed by a device and thus presented on a display of the device. A cursor (i.e., indicator) in computing is a visual indicator that shows the current position for user interaction on a computer display that will respond to input, such as the blinking vertical bar in a text editor or the arrow used to point and click. It allows users to navigate and interact with different user interface elements. In at least one example, the device, such as an XR device, can be configured to identify a first gaze location for the user and select a first central anchor (or centered location) for the indicator based on the first gaze location. The device can identify the gaze location using specialized technology that includes cameras and infrared sensors to monitor and record the movement and position of the eyes. In some examples, the device analyzes the reflections from the cornea and the pupil, determining where a person is looking on a screen or in their environment. In some implementations, the user’s gaze can be derived from vectors associated with the eyes and the head (e.g., the head's orientation relative to the content). The vectors are established using sensors, including cameras, infrared sensors, gyros, or some other sensor that provides information on the orientation of the head and the focus of the eyes. For example, a device can determine that the user’s gaze is viewing a content element (e.g., a button) located a display.

[0021] Once the first gaze location is determined, the device further defines an area available for eye or head movement to maintain the first central anchor or location. The area may represent an angularly defined region that corresponds to a cone with a tip from the gaze location of the user dispersed on the display of the device (e.g., a circular region on the display). The user may manipulate the location of the indicator from the first central anchor to implement desired actions, such as scrolling, object selection, and the like. The indicator anchor is a reference position in the virtual space that determines the starting point or location of the user’s indicator or pointer, enabling precise interaction with virtual objects or interfaces based on the user’s physical gestures. Thus, the anchor may start in the center of the area and move based on the user gestures, such as a hand movement.

[0022] While using the first anchor location, the device further determines when the gaze of the user exceeds the available area associated with the first gaze location. The determination can be based on eye tracking (as described above) or head movement using accelerometers or gyros to determine the gaze of the user relative to the display. Once the user’s gaze exceeds the available area, the device may determine a second anchor location for the indicator based on the current gaze location of the user. In some examples, the device provides a threshold period for the gaze movement before creating a new central anchor location. Thus, when a user quickly glances away, and returns to the first gaze location, a new anchor location will not be created. However, when the user’s glance at another portion of the display exceeds the threshold, the device will determine a second anchor location for the indicator. Once the second anchor location is determined, the user may move the indicator from the second anchor location using various gestures to select, scroll, or provide feedback in association with the information displayed by the device.

[0023] In some implementations, when a new central anchor position is identified for an indicator, the device may manage how the indicator migrates from the first central anchor position to the new central anchor position. In one example, the indicator may “teleport” or jump from the location associated with the first central anchor position to the new central anchor position. In another example, the indicator will be eased or migrated in view from a location associated with the first central anchor to the new central anchor position. The speed and direction of the migrating indicator may be defined based on a direction and speed threshold, determined based on the rate the user’s gaze changes (i.e., glance or slow movement), or restricted based on some other factor.

[0024] In some implementations, the indicator defined by the ray cast or hand ray from the user’s body portion will remain at the first location when the user’s gaze moves to a second location. The hand ray is a virtual concept used in virtual computing environments, representing an invisible line or ray that extends from the user’s hand (i.e., body portion) or controller into the digital environment. It is a pointing device enabling users to interact with virtual objects, select items, or trigger events from a distance by directing the ray toward the desired target and performing a specific action, such as pressing a button or making a gesture. When the user moves the hand ray by moving the extremity toward the second location, the device will migrate the indicator toward the gaze at the second location. In migrating the indicator, the indicatormay move at a speed threshold that does not reflect the movement of the extremity (i.e., the extremity may move faster than the indicator). The slower speed will ease the distraction on the user over an instantaneous jump to the new gaze location. In some examples, the indicator may appear to the user to enter from the edge of a defined area around the second anchor. In some implementations, the area represents an angularly defined region corresponding to a cone with a tip from the gaze location of the user dispersed on the display of the device (e.g., a circular region on the display). The indicator can appear from the edge of this region.

[0025] Although demonstrated in the previous example using a ray cast associated with a hand, similar operations can be performed using a virtual mouse, virtual trackpad, or some other gesture operation. A virtual trackpad or virtual mouse is a software-based input method that simulates the functionality of a physical trackpad or mouse, allowing users to control an indicator or navigate through an interface using gestures like that used for mouse or trackpad, or screen interactions.

[0026] In one example, an XR device may manage the indicator acceleration based on the gaze of the user. When the indicator provided by the hand ray is near the center of the field of view associated with the user’s gaze, the XR device may provide a first speed associated with the movements of the indicator. As the indicator moves closer to the edges of the field of view, the XR device may provide a second speed associated with the movement of the indicator that is different from the first speed. In some implementations, the second acceleration is slower than the first acceleration.

[0027] Various embodiments of the present technology provide for at least one technical effect, advantage, and / or improvement to computing systems and components. For example, various examples may include one or more of the following technical effects, advantages, and / or improvements: 1) non-routine and unconventional use of gaze tracking to move an indicator on a computing device; 2) non-routine and unconventional operations for using a combination of gaze tracking and gestures to migrate an indicator from a first location to a second location; and 3) non-conventional operations to manage the acceleration of an indicator based on the indicator location relative to user gaze.

[0028] FIG. 1A and IB illustrate user perspectives 100 and 101 of using gaze-assisted rays according to an implementation. FIG. 1A includes displayed content 505, indicator 110, area 120, body portion 130, and content 140, 141, and 142. FIG. IB further includes gaze shift150 and area 121. The operations of FIG. lA and IB can be implemented by a computing device, such as computing system 700 of FIG. 7.

[0029] In FIG. 1 A, area 120 (which may not be displayed to the user as part of displayed content 105) corresponds to a first gaze location associated with a user. In some implementations, area 120 can represent an angularly defined region corresponding to a cone with a tip from the gaze location of the user dispersed on the display of the device (e.g., a circular region on the display). For example, indicator 110 can be located at an anchor or center of the user’s gaze, and area 120 corresponds to an area around the anchor. An anchor is a reference position in the virtual space that determines the starting point or location of the user’s cursor or indicator, enabling precise interaction with virtual objects or interfaces based on the user’s physical gestures. Thus, the anchor may start in the center of the area and move based on the user’s gestures, such as hand movement. For example, when the user moves body portion 130 (e.g., the user’s hand), the device can reflect the movement with indicator 110. As at least one technical effect, indicator 110 can be initially positioned on the display based on the user’s gaze and subsequently moved from the initial position (i.e., anchor) based on the movement of body portion 130.

[0030] Turning to FIG. IB, FIG. IB illustrates user perspective 101 of gaze-assisted rays according to an implementation. User perspective 101 is indicative of a time after user perspective 100. In user perspective 101, the user shifts their gaze (gaze shift 150) from the location associated with area 120 to area 121. Gaze can be identified using eye movement technology, which employs infrared cameras and sensors to detect the position and movement of the eyes. The device can capture reflections from the cornea and pupil to determine the direction of the gaze, allowing it to determine where a person is looking on a screen or in a physical environment. In some implementations, the device can be configured to determine when the user’s gaze vacates area 120, then can choose the next anchor when the gaze comes to fixed point. In some examples, the device can determine when the gaze fixates on a portion of displayed content 105 for a threshold period. In response to identifying the second location, the device can be configured to move the indicator, such as indicator 110 in user perspective 101. Thus, based on the gaze exceeding area 120, the indicator can be transitioned from a first anchor position associated with area 120 to a second anchor position associated with area 121 and user perspective 101. In some examples, in moving the indicator between the locations, the devicecan “teleport” or jump indicator 110 from the location associated with the first central anchor position depicted in user perspective 100 to the new central anchor position depicted in user perspective 101. In another example, indicator 110 will be eased or migrated in view from a location associated with the first central anchor to the new central anchor position. The speed and direction of the migrating indicator may be defined based on a direction and speed threshold, determined based on the rate the user’s gaze changes (i.e., glance or slow movement), or restricted based on some other factor.

[0031] FIG. 2 illustrates method 200 of using gaze-assisted input according to an implementation. Method 200 can be implemented by a device, such as computing system 700 of FIG. 7. In some implementations, method 200 can be performed by an XR or other wearable device.

[0032] Method 200 includes identifying a gaze associated with a user of a device at step 201. In some implementations, gaze can be determined using specialized hardware like infrared cameras, near-infrared LEDs, or high-resolution webcams. Using computer vision algorithms, these systems detect reflections from the user’s pupils and corneal features to calculate the gaze direction. Machine learning models may also enhance accuracy by compensating for head movement and lighting conditions.

[0033] Method 200 further includes determining a first anchor or first location for an indicator based on the gaze at step 202. In some implementations, the first location corresponds to a location of focus associated with the user’s gaze. For example, a position where the gaze remained stationary for a threshold period. In some implementations, when the first location is determined, the device can be configured to display an indicator or cursor in the first location. For example, if the user focuses on a button on the display of the device, the device can be configured to display the indicator at the focused location. Once displayed, the user can provide gesture input to move the indicator from the first location and implement desired actions, such as scrolling, object selection, and the like.

[0034] Method 200 also includes determining a second location for the indicator in response to the gaze moving outside of an area around the first location at step 203. In some examples, the gaze moving outside the area can include the gaze moving beyond a threshold distance from the first location. In some implementations, the area represents an angularly defined region corresponding to a cone with a tip from the anchor location (center of the gaze)dispersed on the display of the device (e g., a circular region on the display around the anchor location). When the gaze of the user moves outside of the area, the device can determine a new focus location associated with the user’s gaze. In some implementations, the device can determine when the user’s gaze focuses for a threshold period for a second anchor and cause a display of the indicator at the new focused location while removing the indicator from another location. For example, when anchored at the first location, the user can move the indicator to another location using gestures associated with a body portion. When the user’s gaze moves to a new location, the indicator can be removed from the gesture-based location and moved to the location associated with the gaze. In some examples, the indicator can be migrated rather than teleported between locations.

[0035] FIG. 3A, 3B, and 3C illustrate user perspectives of using gaze to move an indicator according to an implementation. FIG. 3A includes displayed content 305, indicator 310, area 320, body portion 330, and content 340, 341, and 342. FIG. 3B includes gaze shift 350 and replaces area 320 with area 321. FIG. 3C further includes body portion movement 360.

[0036] In FIG. 3A, area 320 (which may not be displayed to the user as part of displayed content 305) corresponds to a first gaze location associated with a user. In some implementations, area 320 can represent an angularly defined region corresponding to a cone with a tip from the gaze location of the user dispersed on the display of the device (e.g., a circular region on the display). For example, indicator 310 can be located at an anchor or center of the user’s gaze, and area 320 corresponds to an area around the anchor. An anchor is a reference position in the virtual space that determines the starting point or location of the user’s cursor or indicator, enabling precise interaction with virtual objects or interfaces based on the user’s physical gestures. Thus, the anchor may start in the center of the area and move based on the user’s gestures, such as hand movement. For example, when the user moves body portion 330 (e.g., the user’s hand), the device can reflect the movement with indicator 310. As at least one technical effect, indicator 310 can be initially positioned on the display based on the user’s gaze and subsequently moved from the initial position (i.e., anchor) based on the movement of body portion 330.

[0037] Turning to FIG. 3B, FIG. 3B illustrates a user perspective 301 of using gaze to move an indicator according to an implementation. User perspective 301 is indicative of a time after user perspective 300. In user perspective 301, the user shifts their gaze (gaze shift 350)from the location associated with area 320 to area 321 . Gaze can be identified using eye movement technology, which employs infrared cameras and sensors to detect the position and movement of the eyes. The device can capture reflections from the cornea and pupil to determine the direction of the gaze, allowing it to determine where a person is looking on a screen or in a physical environment. In some implementations, the device can be configured to determine when the user’s gaze vacates area 320, then can choose the next anchor when the gaze comes to fixed point. In some examples, the device can determine when the gaze fixates on a portion of displayed content 305 for a threshold period. In some examples, the new location for the user’s gaze can be the center of area 321. For example, the user can view a portion of content 342.

[0038] FIG. 3C illustrates a user perspective 302 of using gaze to move an indicator according to an implementation. After the user’s gaze vacates area 320 and establishes a new gaze location (e g., associated with area 321). The device can wait to move indicator 310 until the user provides body portion movement 360 toward the new gaze position. As at least one technical effect, the user can maintain the indicator in a first location, while changing the location of the gaze. The user could return to the first gaze location without causing movement to indicator 310 (e.g., movement away from area 320 and return to area 320).

[0039] Here, the user provides body portion movement 360 toward the new gaze location associated with area 321, causing indicator 310 to appear in area 321, which is associated with the second gaze location. In some examples, the device can be configured to consider various factors in determining the intent of the user movement and whether the movement is associated with moving indicator 310 to the new gaze location. In some examples, the device can use the direction, acceleration, hand shape, or another factor, including combinations thereof, to determine whether body portion movement 360 intends to move indicator 310 to a location associated with the user’s gaze (i.e., in area 321). When the device determines that body portion movement 360 satisfies at least one criterion to move the indicator 310, the indicator can appear (i.e., teleport to the edge of area 321 and move toward the center of the gaze location for the user. For at least technical effect, the user can change their gaze, but the indicator will move when the body portion indicates an intent to migrate the indicator.

[0040] FIG. 4 illustrates method 400 of using gaze to move an indicator according to an implementation. Method 400 can be performed by a device, such as computing system 700 ofFIG. 7. In some implementations, method 200 can be performed by an XR or other wearable device.

[0041] Method 200 includes identifying a gaze associated with a user of a device at step 401 and determining a first area based on the gaze at step 402. In some implementations, the area can correspond to a field of view around the focus or center of the user’s identified gaze. For example, the area can comprise a defined circle size around the focus of the user’s gaze. Method 200 further includes determining a second area based on the gaze moving outside of the first area at step 403. For example, the user can transition from focusing on a first location to focusing on a second location associated with a display, where the second location is an area outside of the area around the first location. Once the second location is identified, the device can determine a second area or region around the gaze location.

[0042] Method 400 further includes identifying a movement of a body portion to the second area at step 404 and displaying an indicator in the second area in in response to the movement of the body portion toward the second area at step 405. In some implementations, the device can be configured to wait to display an indicator in the gaze location until the user generates a gesture demonstrating an intent to use the indicator. For example, while the user reads content on a display, the user may not need the indicator to read the content. Instead, when the user provides a gesture or movement toward the second area, the device can display the indicator in the second area. In some implementations, the indicator can teleport or jump from a location outside the second area to a location in the second area. In some implementations, the indicator can migrate from a first location outside of the second area to the location in the second area. For example, the indicator can be in the first area associated with the gaze and migrate to the second area in response to the user movement.

[0043] In some implementations, the device can be configured to use various factors in determining whether to display the indicator in the second area. In some examples, the device can use the direction of the movement. In some examples, the device can use the acceleration of the movement. In some examples, the device can determine whether interactive elements (e.g., buttons, menus, and the like) are in the second area. In some examples, the device can use any of combination of the previously mentioned factors to determine whether the user intends an indicator to be visible in the second area.

[0044] FIG. 5A and 5B illustrate user perspectives 500 and 501 of gaze-dependent speed for an indicator according to an implementation. FIG. 5A demonstrates user perspective 500 with indicator 510, field of view 520, body portion 530, content 540, 541, and 542, and speed 550. FIG. 5B replaces speed 550 with speed 551.

[0045] In FIG. 5A and user perspective 500, indicator 510 is displayed to the user, permitting the user to move indicator 510 using body portion 530. Body portion 530 can move indicator 510 by utilizing motion sensors, cameras, or touch-sensitive surfaces to detect hand or finger movements. For example, a user can swipe in a direction to move indicator 510. Here, based on the movement of body portion 530 the indicator is moved at speed 550. Speed 550 is selected based on the distance of indicator 510 from the user’s gaze of the center of the field of view 520. In at least one example, indicator 510 may move at a first speed when at a first position relative to the center of field of view 520 and a second speed when at a second position relative to the center of field of view 520. The speed can be slower in the position that is further from the center of the field of view. For example, as demonstrated in FIG. 5B, user perspective 501 demonstrates a different speed 551 than speed 550. In some examples, the speed can be correlated to the distance.

[0046] FIG. 6 illustrates method 600 of providing gaze-dependent speed for an indicator according to an implementation. Method 600 can be performed by a device, such as computing system 700 of FIG. 7. In some implementations, method 600 can be performed by a wearable or XR device. However, method 600 can be performed by any device capable of identifying user gaze and gestures.

[0047] Method 600 includes determining a gaze associated with a user of a device at step 601. Method 600 further includes determining a field of view based on the gaze at step 602 and modifying a speed for an indicator based on the location of the indicator relative to the field of view at step 603. For example, a device can determine a gaze location associated with the user. A device can determine a user’s gaze by analyzing eye movements, pupil orientation, and facial positioning using cameras or infrared sensors. By interpreting optical reflections and applying gaze estimation algorithms, it can infer where the user is looking, which can be considered a gaze location. The user of the device can further provide body movements (e.g., hand gestures) that move an indicator on a display of the device. Based on the location of the indicator relative to the gaze, the device can adjust the speed of the indicator. For example, the indicator can havea faster speed when further away from the user’s gaze and a slower speed (e.g., for more precision) when closer to the user’s gaze location.

[0048] FIG. 7 illustrates a computing system 700 that displays an indicator according to an implementation. Computing system 700 represents any apparatus, computing system, or systems with which the various operational architectures, processes, scenarios, and sequences are disclosed herein for displaying an indicator can be implemented. Computing system 700 can be an example of an XR device, wearable device, or other computing device capable of the operations described herein. Computing system 700 can be a system of devices, such as a wearable device and a companion device (e.g., smartphone, tablet, etc.) in some examples. Computing system 700 includes storage system 745, processing system 750, communication interface 760, and input / output (VO) device(s) 770. Processing system 750 is operatively linked to communication interface 760, I / O device(s) 770, and storage system 745. In some implementations, communication interface 760 and / or I / O device(s) 770 may be communicatively linked to storage system 745. Computing system 700 may further include other components such as a battery and enclosure that are not shown for clarity.

[0049] Communication interface 760 comprises components that communicate over communication links, such as network cards, ports, radio frequency, processing circuitry (and corresponding software), or some other communication devices. Communication interface 760 may be configured to communicate over metallic, wireless, or optical links. Communication interface 760 may be configured to use Time Division Multiplex (TDM), Internet Protocol (IP), Ethernet, optical networking, wireless protocols, communication signaling, or some other communication format - including combinations thereof. Communication interface 760 may be configured to communicate with external devices, such as servers, user devices, or other computing devices.

[0050] I / O device(s) 770 may include peripherals of a computer that facilitate the interaction between the user and computing system 700. Examples of I / O device(s) 770 may include keyboards, mice, trackpads, monitors, displays, printers, cameras, microphones, external storage devices, sensors, and the like. In some implementations, I / O device(s) 770 include at least one outward-facing camera configured to capture images associated with the physical environment. In some implementations, I / O device(s) 770 consists of a see-through or video pass-through display providing a view of the physical environment. In some implementations,the computing system 700 can include a display capable of displaying an indicator or other content. In some examples, I / O device(s) 770 can include one or more sensors and cameras capable of monitoring user gaze and gestures.

[0051] Processing system 750 comprises microprocessor circuitry (e.g., at least one processor) and other circuitry that retrieves and executes operating software (i.e., program instructions) from storage system 745. Storage system 745 may include volatile and nonvolatile, removable, and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Storage system 745 may be implemented as a single storage device but may also be implemented across multiple storage devices or sub-systems. Storage system 745 may comprise additional elements, such as a controller to read operating software from the storage systems. Examples of storage media (also referred to as computer-readable storage media or a computer- readable storage medium) include random access memory, read-only memory, magnetic disks, optical disks, and flash memory, as well as any combination or variation thereof, or any other type of storage media. In some implementations, the storage media may be non-transitory. In some instances, at least a portion of the storage media may be transitory. In no case is the storage media a propagated signal.

[0052] Processing system 750 is typically mounted on a circuit board that may also hold the storage system. The operating software of storage system 745 comprises computer programs, firmware, or some other form of machine-readable program instructions. The operating software of storage system 745 comprises indicator application 724. The operating software on storage system 745 may further include an operating system, utilities, drivers, network interfaces, applications, or some other type of software. When read and executed by processing system 750 the operating software on storage system 745 directs computing system 700 to operate as described herein. In at least one implementation, the operating software can provide method 200 described in FIG. 2. The operating software can provide or cause the at least one processor to manage the location of an indicator as described herein.

[0053] In at least one implementation, indicator application 724 directs processing system 750 to identify a gaze associated with a user of a device and determine a first location for an indicator based on the gaze. Indicator application 724 further directs processing system 750 to determine a second location for the indicator in response to the gaze moving outside an areaaround the first location. In some implementations, indicator application 724 directs processing system 750 to identify movement associated with the body portion and cause the indicator to move from the first (or second) location based on the movement. For example, when the user’s gaze is at the first location, the indicator can be displayed at the first location. The user can then provide hand movements to move the indicator from the first location (e.g., to select content).

[0054] In some implementations, indicator application 724 directs processing system 750 to identify a movement associated with a body portion and cause a movement of the indicator from the first location to a third location based on the movement. Indicator application 724 further directs processing system 750 to identify a second movement associated with the body portion toward the second location in response to the gaze moving outside the area around the first location and migrate the indicator from the third location to the second location. For example, the user can move the indicator using a gesture when the gaze is in the first location and, when the gaze moves to the second location, the indicator can be migrated to the second location. In some implementations, computing system 700 can teleport to the second location instead of migrating the indicator.

[0055] In some implementations, indicator application 724 can direct processing system 750 to move the indicator based on the user’s body movement (e.g., hand gesture). The speed of the indicator can be selected based on the location of the indicator relative to the user’s gaze. In some implementations, the speed correlates to the distance. For example, as the indicator moves further away from the user’s gaze the speed of the indicator can increase.

[0056] Example claim clauses are provided below. Although these are examples, these clauses should not be considered exhaustive.

[0057] Clause 1. A method comprising: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving beyond a threshold distance from the first location.

[0058] Clause 2. The method of clause 1, further comprising: identifying a movement associated with a body portion of the user; and causing the indicator to move from the second location based on the movement.

[0059] Clause 3. The method of any one of the preceding clauses, further comprising: identifying a movement associated with a body portion of the user; and causing the indicator to move from the first location based on the movement.

[0060] Clause 4. The method of any one of the preceding clauses, further comprising: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving beyond the threshold distance from the first location, identifying a second movement associated with the body portion toward the second location, and migrating the indicator from the third location to the second location based on the second movement.

[0061] Clause 5. The method of any one of the preceding clauses, further comprising: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving beyond the threshold distance from the first location, identifying a second movement associated with the body portion toward the second location, removing the indicator from the third location based on the second movement; and displaying the indicator at the second location based on the second movement.

[0062] Clause 6. The method of any one of the preceding clauses, further comprising: displaying the indicator on a display of the device at the first location; and displaying the indicator on a display of the device at the second location.

[0063] Clause 7. The method of any one of the preceding clauses, further comprising: identifying a movement associated with a body portion of the user; and causing a migration of the indicator from the second location to a third location based on the movement, wherein a speed of the migration is based on a distance of the indicator from the second location.

[0064] Clause 8. The method of clause 7, wherein the speed correlates to the distance.

[0065] Clause 9. A computing system comprising: a computer-readable storage medium; at least one processor operatively coupled to the computer-readable storage medium; and program instructions stored on the computer-readable storage medium that, when executed by the at least one processor, direct the computing system to perform a method, the method comprising: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving outside of an area around the first location.

[0066] Clause 10. The computing system of clause 9, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing the indicator to move from the second location based on the movement.

[0067] Clause 11. The computing system of clause 9 or 10, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing the indicator to move from the first location based on the movement.

[0068] Clause 12. The computing system of any one of clauses 9 to 11, wherein the method further comprises: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving outside of the area around the first location, identifying a second movement associated with the body portion toward the second location, and migrating the indicator from the third location to the second location based on the second movement.

[0069] Clause 13. The computing system of any one of clauses 9 to 12, wherein the method further comprises: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving outside of the area around the first location, identifying a second movement associated with the body portion toward the second location, removing the indicator from the third location based on the second movement; and displaying the indicator at the second location based on the second movement.

[0070] Clause 14. The computing system of any one of clauses 9 to 13, wherein the method further comprises: displaying the indicator on a display of the computing system at the first location; and displaying the indicator on a display of the computing system at the second location.

[0071] Clause 15. The computing system of any one of clauses 9 to 14, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing a migration of the indicator from the second location to a third location based on the movement, wherein a speed of the migration is based on a distance of the indicator from the second location.

[0072] Clause 16. A computer-readable storage medium having program instructions stored thereon that, when executed by at least one processor, direct the at least one processor to perform a method, the method comprising: identifying a gaze associated with a user of a device;determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving outside of an area around the first location.

[0073] Clause 17. The computer-readable storage medium of clause 16, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing the indicator to move from the second location based on the movement.

[0074] Clause 18. The computer-readable storage medium of clause 16 or 17, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing the indicator to move from the first location based on the movement.

[0075] Clause 19. The computer-readable storage medium of any one of clauses 16 to 18, wherein the method further comprises: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving outside of the area around the first location, identifying a second movement associated with the body portion toward the second location, and migrating the indicator from the third location to the second location based on the second movement.

[0076] Clause 20. The computer-readable storage medium of any one of clauses 16 to 19, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing a migration of the indicator from the second location to a third location based on the movement, wherein a speed of the migration is based on a distance of the indicator from the second location.

[0077] In this specification and the appended claims, the singular forms “a,” “an” and “the” do not exclude the plural reference unless the context dictates otherwise. Further, conjunctions such as “and,” “or,” and “and / or” are inclusive unless the context dictates otherwise. For example, “A and / or B” includes A alone, B alone, and A with B. Further, connecting lines or connectors shown in the various figures presented are intended to represent example functional relationships and / or physical or logical couplings between the various elements. Many alternative or additional functional relationships, physical connections, or logical connections may be present in a practical device. Moreover, no item or component is essential to the practice of the implementations disclosed herein unless the element is specifically described as “essential” or “critical.”

[0078] Terms such as, but not limited to, approximately, substantially, generally, etc. are used herein to indicate that a precise value or range thereof is not required and need not be specified. As used herein, the terms discussed above will have ready and instant meaning to one of ordinary skill in the art.

[0079] Moreover, the use of terms such as up, down, top, bottom, side, end, front, back, etc. herein are used concerning a currently considered or illustrated orientation. If they are considered concerning another orientation, such terms must be correspondingly modified.

[0080] Further, in this specification and the appended claims, the singular forms “a,” “an” and “the” do not exclude the plural reference unless the context dictates otherwise. Moreover, conjunctions such as “and,” “or,” and “and / or” are inclusive unless the context dictates otherwise. For example, “A and / or B” includes A alone, B alone, and A with B.

[0081] Although certain example methods, apparatuses, and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. It is to be understood that the terminology employed herein is to describe aspects and is not intended to be limiting. On the contrary, this patent covers all methods, apparatus, and articles of manufacture fairly falling within the scope of the claims of this patent.

Claims

WHAT IS CLAIMED IS:

1. A method comprising: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving beyond a threshold distance from the first location.

2. The method of claim 1, further comprising: identifying a movement associated with a body portion of the user; and causing the indicator to move from the second location based on the movement.

3. The method of any one of the preceding claims, further comprising: identifying a movement associated with a body portion of the user; and causing the indicator to move from the first location based on the movement.

4. The method of any one of the preceding claims, further comprising: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving beyond the threshold distance from the first location, identifying a second movement associated with the body portion toward the second location, and migrating the indicator from the third location to the second location based on the second movement.

5. The method of any one of the preceding claims, further comprising: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving beyond the threshold distance from the first location,identifying a second movement associated with the body portion toward the second location, removing the indicator from the third location based on the second movement; and displaying the indicator at the second location based on the second movement.

6. The method of any one of the preceding claims, further comprising: displaying the indicator on a display of the device at the first location; and displaying the indicator on a display of the device at the second location.

7. The method of any one of the preceding claims, further comprising: identifying a movement associated with a body portion of the user; and causing a migration of the indicator from the second location to a third location based on the movement, wherein a speed of the migration is based on a distance of the indicator from the second location.

8. The method of claim 7, wherein the speed correlates to the distance.

9. A computing system comprising: a computer-readable storage medium; at least one processor operatively coupled to the computer-readable storage medium; and program instructions stored on the computer-readable storage medium that, when executed by the at least one processor, direct the computing system to perform a method, the method comprising: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving outside of an area around the first location.

10. The computing system of claim 9, wherein the method further comprises: identifying a movement associated with a body portion of the user; andcausing the indicator to move from the second location based on the movement.

11. The computing system of claim 9 or 10, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing the indicator to move from the first location based on the movement.

12. The computing system of any one of claims 9 to 11, wherein the method further comprises: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving outside of the area around the first location, identifying a second movement associated with the body portion toward the second location, and migrating the indicator from the third location to the second location based on the second movement.

13. The computing system of any one of claims 9 to 12, wherein the method further comprises: identifying a movement associated with a body portion of the user; causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving outside of the area around the first location, identifying a second movement associated with the body portion toward the second location, removing the indicator from the third location based on the second movement; and displaying the indicator at the second location based on the second movement.

14. The computing system of any one of claims 9 to 13, wherein the method further comprises:displaying the indicator on a display of the computing system at the first location; and displaying the indicator on a display of the computing system at the second location.

15. The computing system of any one of claims 9 to 14, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing a migration of the indicator from the second location to a third location based on the movement, wherein a speed of the migration is based on a distance of the indicator from the second location.

16. A computer-readable storage medium having program instructions stored thereon that, when executed by at least one processor, direct the at least one processor to perform a method, the method comprising: identifying a gaze associated with a user of a device; determining a first location for an indicator based on the gaze; and determining a second location for the indicator in response to the gaze moving outside of an area around the first location.

17. The computer-readable storage medium of claim 16, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing the indicator to move from the second location based on the movement.

18. The computer-readable storage medium of claim 16 or 17, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing the indicator to move from the first location based on the movement.

19. The computer-readable storage medium of any one of claims 16 to 18, wherein the method further comprises: identifying a movement associated with a body portion of the user;causing the indicator to move from the first location to a third location based on the movement; and in response to the gaze moving outside of the area around the first location, identifying a second movement associated with the body portion toward the second location, and migrating the indicator from the third location to the second location based on the second movement.

20. The computer-readable storage medium of any one of claims 16 to 19, wherein the method further comprises: identifying a movement associated with a body portion of the user; and causing a migration of the indicator from the second location to a third location based on the movement, wherein a speed of the migration is based on a distance of the indicator from the second location.

Images