Power control based at least in part on user eye movement

Abstract

In some aspects, the user device can receive, from a visual sensor of the user device, first image data associated with a first set of images. The user device can determine, using an image processing model, that the set of images depicts a particular type of eye activity of the user's eyes. The user device can cause a power level of a camera of the user device to be reduced based at least in part on determining that the first set of images depicts the particular type of eye activity.

Description

[0001] open field

[0002] Various aspects of this disclosure generally relate to power control of user equipment, and for example to device power control using low-power sensors to control power based on user eye movement.

[0003] background

[0004] Sensors are used within user equipment for a variety of purposes. Such sensors can be used to sense one or more characteristics associated with the user equipment. These characteristics may include one or more characteristics of the user equipment's environment and / or biometrics of the user associated with the user equipment. For example, one or more sensors may be configured to detect the presence of a user, detect user activity, detect motion, measure ambient light, capture images of the environment, and so on.

[0005] Overview

[0006] In some aspects, a method includes: receiving first image data associated with a first set of images from a vision sensor of a user device; using an image processing model to determine a first type of eye activity depicted by the first set of images of a user eye; causing a reduction in the power level of a camera of the user device based at least in part on the determination that the first set of images depicts the first type of eye activity; receiving second image data associated with a second set of images from a vision sensor; using an image processing model to determine a second type of eye activity depicted by the second set of images; and causing a reduction in the power level of a camera based at least in part on the determination that the second set of images depicts the second type of eye activity.

[0007] In some aspects, a device includes one or more memories and one or more processors communicatively coupled to the one or more memories, the one or more processors being configured to: receive first image data associated with a first image set from a vision sensor of a user device; use an image processing model to determine a first type of eye activity depicted by the first image set in the user's eye; cause a reduction in the power level of a camera of the user device, at least in part based on the determination that the first image set depicts the first type of eye activity; receive second image data associated with a second image set from a vision sensor; use an image processing model to determine a second type of eye activity depicted by the second image set in the user's eye; and cause an increase in the power level of a camera, at least in part based on the determination that the second image set depicts the second type of eye activity.

[0008] In some aspects, a non-transient computer-readable medium storing a set of instructions comprising one or more instructions that, when executed by one or more processors of a device, cause the device to: receive first image data associated with a first set of images from a vision sensor of a user device; determine, using an image processing model, that the first set of images depicts a first type of eye activity of a user's eye; cause, at least in part, based on the determination that the first set of images depicts the first type of eye activity, to reduce the power level of a camera of the user device; receive second image data associated with a second set of images from a vision sensor; determine, using an image processing model, that the second set of images depicts a second type of eye activity of the eye; and cause, at least in part, based on the determination that the second set of images depicts the second type of eye activity, to increase the power level of a camera.

[0009] In some aspects, an apparatus includes: means for receiving first image data associated with a first set of images from a vision sensor of a user device; means for determining, using an image processing model, a first type of eye activity depicted by the first set of images; and means for causing a reduction in the power level of the camera of the user device, at least in part based on the determination that the first set of images depicts the first type of eye activity.

[0010] The aspects generally include, as described substantially with reference to the accompanying drawings and description and explained as such, methods, apparatus, systems, computer program products, non-transient computer-readable media, user equipment, user gear, wireless communication equipment, and / or processing systems.

[0011] The foregoing has broadly outlined the features and technical advantages of the examples according to this disclosure in an effort to facilitate a better understanding of the following detailed description. Additional features and advantages will be described thereafter. The disclosed concepts and specific examples can be readily used as the basis for modifications or the design of other structures for implementing the same purposes as this disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The characteristics of the concepts disclosed herein, in both their organization and manner of operation, and their associated advantages, will be better understood by considering the following description in conjunction with the accompanying drawings. Each drawing is provided for illustrative and descriptive purposes and not for defining limitations on the claims. Brief description of the attached diagram

[0013] To gain a more detailed understanding of the features described above in this disclosure, reference can be made to various aspects of the above brief overview, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain typical aspects of this disclosure and should not be considered as limiting its scope, as other equivalent aspects are permissible in this description. Identical reference numerals in different drawings may identify the same or similar elements.

[0014] Figure 1 This is a diagram illustrating an example environment associated with power control based at least in part on user eye movement, according to various aspects of this disclosure.

[0015] Figure 2 This is an explanation of the various aspects based on this disclosure. Figure 1 A diagram illustrating example components of one or more devices shown.

[0016] Figure 3 These are illustrations that conceptually explain examples of user equipment including visual sensors according to various aspects of this disclosure.

[0017] Figures 4A-4C This is a diagram illustrating an example of power control associated with at least part of the user's eye movement, according to various aspects of this disclosure.

[0018] Figure 5 This is a diagram illustrating an example of power control associated with at least part of the user's eye movement, according to various aspects of this disclosure.

[0019] Figure 6 and 7 This is a flowchart of an example process associated with power control based at least in part on user eye movement, according to various aspects of this disclosure.

[0020] Detailed description

[0021] The various aspects of this disclosure are described more fully below with reference to the accompanying drawings. However, this disclosure may be implemented in many different forms and should not be construed as being limited to any specific structure or function given throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art. Based at least in part on the teachings herein, those skilled in the art will appreciate that the scope of this disclosure is intended to cover any aspect of this disclosure disclosed herein, whether implemented independently of or in combination with any other aspect of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or method of practice. Furthermore, the scope of this disclosure is intended to cover such apparatus or methods practiced using additional structures, functionalities, or structures and functionalities that complement or supplement the various aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure disclosed herein may be implemented by one or more elements of the claims.

[0022] Many user devices (e.g., mobile devices, computers, wearable devices, etc.) may include one or more cameras to allow a user to capture images using the user device. Furthermore, in some instances, the camera may be included on the display side of the user device so that it can capture images of the user as the user faces and / or interacts with the display. For example, when used in conjunction with virtual reality (VR) and / or augmented reality (AR) applications, the camera may be used in conjunction with an always-on motion detection mode to capture images of the user's eyes to determine the direction of the user's gaze and control the display settings based on that gaze direction. For example, the user device may enhance certain portions of the image to be presented on the display area aligned with the gaze direction (e.g., increasing resolution, brightness, contrast, etc.) and / or reduce certain portions of the image to be presented on the display outside the gaze direction range (e.g., to conserve resources associated with displaying relatively enhanced portions of the image that the user is not viewing). However, during certain types of eye movements, the optical characteristics of the eye are anatomically uninterpretable for the user. For example, during saccade eye movements (e.g., eye movements that occur when a user shifts focus from one object to another, or from one direction to another), the user (e.g., the user's brain) cannot interpret the optical characteristics from the user's eyes. Accordingly, during these types of movements, the power and / or computing resources (e.g., processing and / or memory resources) of the user device are wasted on processing, generating, and / or displaying relatively enhanced images (e.g., images with relatively high resolution) on the user device's display.

[0023] Some aspects described herein enable a user equipment to enter a low-power mode, at least in part, based on identifying certain types of eye movements of the user. For example, the user equipment may monitor the user's eye activity, determine that the movement corresponds to a saccade movement (or other type of movement) that the user cannot interpret from the optical characteristics of the eye during this period, and prompt the user equipment's camera to enter a low-power mode and / or the user equipment to enter a low-power mode. As described herein, the user equipment may detect saccade eye movements and reduce the power supplied to the camera (e.g., terminate power supply to the camera and / or reduce the camera's resolution). Additionally or alternatively, the user equipment may include and / or utilize a vision sensor (e.g., a low-power sensor that consumes less power than the user equipment's camera) to monitor and / or track the camera's eye movements and prompt the camera and / or the user equipment to enter a low-power mode when the low-power sensor detects saccade eye movements. In this way, when saccade eye movements are detected, the user equipment may switch to a low-power mode until the saccade eye movements cease. Once the user device detects that the saccaded eye movement has ended, the user device can return the user device and / or camera to a high-power mode to operate according to the user device's desired application (e.g., VR application, AR application, etc.).

[0024] Accordingly, as described herein, the visual sensor of the user equipment can be a low-resolution (e.g., less than one megapixel), low-power (e.g., consuming less than 10mW) sensor that enables always-on motion detection. The visual sensor can be a separate sensor of a camera and / or correspond to a camera operating in low-power and / or low-resolution modes, which consumes less user equipment power (e.g., from the user equipment's battery) and / or user equipment processing resources. In this way, the visual sensor described herein allows always-on motion detection (e.g., for eye tracking and / or gaze detection) while consuming less power and / or computational resources than a camera (and / or a camera in high-power mode).

[0025] Figure 1 This is an illustration of an example system 100 that can implement the image capture module described herein, according to various aspects of this disclosure. For example... Figure 1 As shown, system 100 may include user equipment 110, wireless communication equipment 120, and / or network 130. The devices of system 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0026] User equipment 110 includes one or more devices capable of incorporating one or more image capture modules as described herein. For example, user equipment 110 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with one or more sensors as described herein. Although Figure 1Examples shown include smartphones and VR devices, but user equipment 110 may include any type of communication and / or computing device, such as laptops, tablets, handheld computers, desktop computers, gaming devices, wearable communication devices (e.g., smartwatches, smart glasses, etc.), AR devices, or similar types of devices. As described herein, user equipment 110 (and / or the image capture module of user equipment 110) may be used to detect and analyze the user's eye movements, and / or perform one or more operations associated with tracking and / or analyzing the user's eye movements.

[0027] Similar to user equipment 110, wireless communication device 120 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with one or more sensors described herein. For example, wireless communication device 120 may include a base station, access point, etc. Additionally or alternatively, similar to user equipment 110, wireless communication device 120 may include communication devices and / or computing devices, such as mobile phones (e.g., smartphones, cordless phones, etc.), laptop computers, tablet computers, handheld computers, desktop computers, gaming devices, wearable communication devices (e.g., smartwatches, smart glasses, etc.), VR devices, AR devices, or similar types of devices.

[0028] Network 130 includes one or more wired and / or wireless networks. For example, network 130 may include cellular networks (e.g., Long Term Evolution (LTE) networks, Code Division Multiple Access (CDMA) networks, 3G networks, 4G networks, 5G networks, another type of next-generation network, etc.), Public Land Mobile Networks (PLMNs), Local Area Networks (LANs), Wide Area Networks (WANs), Metropolitan Area Networks (MANs), telephone networks (e.g., Public Switched Telephone Networks (PSTN)), private networks, ad hoc networks, intranets, the Internet, fiber-optic networks, cloud computing networks, etc., and / or combinations of these or other types of networks. In some aspects, network 130 may include data networks and / or communicate with data platforms (e.g., web platforms, cloud-based platforms, non-cloud-based platforms, etc.) capable of receiving, generating, processing, and / or providing information associated with applications of user equipment 110.

[0029] Figure 1 The number and arrangement of devices and networks shown are provided as one or more examples. In practice, there may be variations. Figure 1 The devices and / or networks shown are compared to additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged differently. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1The single device shown can be implemented as multiple distributed devices. Additionally or alternatively, the set of devices in system 100 (e.g., one or more devices) can perform one or more functions described as being performed by another set of devices in system 100.

[0030] Figure 2 This is a diagram illustrating example components of device 200. Device 200 may correspond to user equipment 110 and / or wireless communication device 120. Additionally or alternatively, user equipment 110 and / or wireless communication device 120 may include one or more devices 200 and / or one or more components of device 200. Figure 2 As shown, device 200 may include bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, and one or more sensors 240 (each referred to as "sensor 240" and collectively referred to as "sensor 240").

[0031] Bus 205 includes components that allow communication between the components of device 200. Processor 210 includes a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), digital signal processor (DSP), microprocessor, microcontroller, field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), and / or another type of processing component. Processor 210 is implemented in hardware, firmware, or a combination of hardware and software. In some aspects, processor 210 includes one or more processors that can be programmed to perform functions.

[0032] The memory 215 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by the processor 210.

[0033] Storage component 220 stores information and / or software related to the operation and use of device 200. For example, storage component 220 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state drives), compact discs (CDs), digital multi-purpose discs (DVDs), floppy disks, cassettes, magnetic tapes, and / or other types of non-transitory computer-readable media, along with corresponding drives.

[0034] Input component 225 includes components that allow device 200 to receive information, such as via user input. For example, input component 225 may be associated with a user interface as described herein (e.g., to allow the user to interact with one or more features of device 200). Input component 225 may include a touchscreen display, keyboard, keypad, mouse, buttons, switches, microphone, etc. Additionally or alternatively, input component 225 may include sensors for sensing information associated with device 200. More specifically, input component 225 may include a magnetometer (e.g., a Hall effect sensor, an anisotropic magnetoresistive (AMR) sensor, a giant magnetoresistive (GMR) sensor, etc.), a position sensor (e.g., a Global Positioning System (GPS) receiver, a Local Positioning System (LPS) device (e.g., using triangulation, multi-location, etc.), etc.), a gyroscope (e.g., a microelectromechanical system (MEMS) gyroscope or a similar device), an accelerometer, a velocity sensor, a motion sensor, an infrared sensor, a temperature sensor, a pressure sensor, etc. Output component 230 includes components (e.g., display, speaker, one or more light-emitting diodes (LEDs), etc.) that provide output from device 200.

[0035] Communication interface 235 includes a transceiver and / or separate receiver and transmitter, enabling device 200 to communicate with other devices (such as via wired connection, wireless connection, or a combination of wired and wireless connections). Communication interface 235 may allow device 200 to receive information from another device and / or provide information to another device. For example, communication interface 235 may include an Ethernet interface, optical interface, coaxial interface, infrared interface, radio frequency (RF) interface, universal serial bus (USB) interface, Wi-Fi interface, cellular network interface, wireless modem, or inter-integrated circuit (IIC) interface. 2 C), Serial Peripheral Interface (SPI), etc.

[0036] Sensor 240 includes one or more devices capable of sensing characteristics associated with the environment of device 200. Sensor 240 may include one or more passive components, such as one or more integrated circuits and / or one or more flexible circuits (e.g., on a packaged silicon die), to enable communication with one or more components of device 200. In some aspects, sensor 240 may include vision sensors (e.g., image sensors, optical sensors, etc.), cameras (e.g., low-resolution cameras, high-resolution cameras, etc.), and so on.

[0037] Sensor 240 may include a vision sensor and / or a low-resolution camera (e.g., a video graphics array (VGA)) capable of capturing low-resolution images (e.g., images less than one megapixel). Sensor 240 may be a low-power component or device (e.g., a device consuming less than 10 milliwatts (mW) of power) that is normally on when device 200 is powered on.

[0038] Device 200 can perform one or more of the processes described herein. Device 200 can perform these processes in response to processor 210 executing software instructions stored in a non-transitory computer-readable medium (such as memory 215 and / or storage component 220). As used herein, "computer-readable medium" means a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0039] Software instructions can be read from another computer-readable medium or another device into memory 215 and / or storage component 220 via communication interface 235. The software instructions stored in memory 215 and / or storage component 220, when executed, can cause processor 210 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the aspects described herein are not limited to any particular combination of hardware circuitry and software.

[0040] In some aspects, device 200 includes means for performing one or more processes described herein and / or means for performing one or more operations of the processes described herein. For example, means for performing the processes and / or operations described herein may include bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, sensor 240 and / or any combination thereof. More specifically, device 200 includes: means for receiving, from a vision sensor (received from a vision sensor of a user device), first image data associated with a first image set; means for using an image processing model to determine, the first image set depicts a first type of eye activity of a user eye; means for causing a reduction in the power level of a camera of the user device, at least in part based on the determination that the first image set depicts the first type of eye activity; means for receiving, from a vision sensor, second image data associated with a second image set; means for using an image processing model to determine, the second image set depicts a second type of eye activity of a user eye; and means for causing an increase in the power level of a camera, at least in part based on the determination that the second image set depicts the second type of eye activity.

[0041] Figure 2The number and arrangement of components shown are provided as an example. In practice, device 200 may include components with... Figure 2 The components shown are compared to additional components, fewer components, different components, or components arranged differently. Additionally or alternatively, the set of components of device 200 (e.g., one or more components) can perform one or more functions described as being performed by another set of components of device 200.

[0042] Figure 3 This is a conceptual illustration of an example of a user equipment 300 including a vision sensor 305 according to various aspects of this disclosure. Figure 3 As shown in the example, user equipment 300 (which may correspond to user equipment 110) includes an image capture module 310. Although Figure 3 The image capture module 310 includes a vision sensor 305 and a camera 315, but other configurations of the image capture module 310 and / or other configurations of the vision sensor 305 and camera 315 are also contemplated. For example, the vision sensor 305 (which has a vision sensor aperture 320) and the camera 315 (which has a camera aperture 325) may be combined or arranged as a single device (e.g., a single camera with low-power, low-resolution modes and high-power and / or high-resolution modes). In one or more of the following examples, the camera 315 may be a high-resolution camera, and the vision sensor 305 may be a low-power and / or low-resolution camera, as described herein.

[0043] The vision sensor 305 can be any suitable sensor capable of optically sensing one or more characteristics of the environment of the image capture module 310 (which, once the image capture module 310 is installed in the user equipment, can be the same environment as that of the user equipment 300). For example, the vision sensor 305 can be a low-resolution camera, an optical sensor capable of detecting light (e.g., ambient light, infrared light, optical communication signals, etc.), an infrared sensor, etc.

[0044] The vision sensor 305 of the image capture module 310 can be a low-power sensor that requires less than 10mW of power to operate. The vision sensor 305 may require less power than the camera 315 of the image capture module 310. For example, the vision sensor 305 can be a low-resolution camera that requires less than 10mW (e.g., 1mW, 2mW, 5mW, etc.) to capture images and / or video of the environment via the vision sensor aperture 320. In this way, the vision sensor 305 can implement a normally-on motion detection capability, which enables the detection of one or more objects (e.g., the user's eyes) without user interaction with the user interface of the user device 300, without the user picking up (and / or touching) the user device 300, etc. Accordingly, the vision sensor 305 can be in a normally-on mode after the user device is started, after a normally-on mode is activated, etc.

[0045] Additionally or alternatively, the vision sensor 305 may include an infrared sensor that facilitates normally-on detection when the environment of the image capture module is dark. For example, devices associated with the image capture module 310 may include a low-power emitter that emits infrared light, and the vision sensor 305 may sense reflections of the emitted light, which can be analyzed (e.g., to detect user eye movement).

[0046] As described herein, the vision sensor 305 may be configured to have a normally-on capability to analyze the movement of one or more eyes of a user associated with the user device 300 (e.g., a user within the field of view of the vision sensor 305). For example, the normally-on capability may allow the vision sensor 305 to continuously monitor the user's eye movements, at least in part, based on a specific application of the user device 300 (e.g., a VR application, an AR application, and / or other types of image viewing applications) being opened and / or activated. For example, a user may open an application to view media (e.g., images, videos, etc.) via the display 330 of the user device 300, receive a VR experience, receive an AR experience, etc. Accordingly, this normally-on capability may facilitate a normally-on motion detection capability to identify, track, and / or analyze the user's eye movements, as described herein.

[0047] In some aspects, as described herein, based at least in part on the detection of a specific eye movement (e.g., saccade eye movement) using the always-on capability of the vision sensor 305, the vision sensor 305 can analyze the eye movement (e.g., gaze direction, change in gaze direction, etc.) via an image processing model (e.g., object processing model, edge detection model, optical flow model, etc.) and perform one or more actions associated with the user device 300 (e.g., causing the camera 315 to enter a low-power mode, causing the camera 315 to enter a low-resolution mode, etc.). Furthermore, the vision sensor 305 in its always-on motion detection mode can record a sliding window of images captured by the vision sensor 305. For example, the image sliding window can be stored as pre-roll video (e.g., video captured during a time period before and / or during the detection of the eye and / or a specific eye movement). Accordingly, according to various aspects of this disclosure, the image sliding window can be saved as a pre-roll video that can be accessed to allow frame-by-frame analysis of the eye and / or eye movement.

[0048] The camera 315 of the user equipment 300 may be a high-resolution camera, which includes a camera aperture 325 and is powered by a power source (e.g., a battery) when mounted within the user equipment 300. As a high-resolution camera, the camera 315 may require 100mW or more of power to capture images and / or video. The camera 315 may be communicatively coupled to a device (e.g., processor 210, input component 225, etc.) via a communication bus (e.g., bus 205) to allow the camera 315 to be controlled and / or to be provided with the captured images to that device. While some aspects described herein may describe the vision sensor 305 as a separate device from the camera 315, the vision sensor 305 and the camera 315 may be the same device. For example, the vision sensor 305 may correspond to the camera 315 in a low-power mode (e.g., a mode using binning and / or skipping techniques, and other example techniques for reducing power consumption) and / or a low-resolution mode. In such cases, camera 315 may operate in a normally open detection mode (e.g., normally open motion detection mode, normally open gaze detection mode, etc.) to detect and / or identify one or more of the user's eyes while the user device 300 is displaying media associated with certain applications via display 330, as described herein.

[0049] In some aspects, the image capture module 310 may be formed and / or configured to include a detachable camera module for camera 315 and / or a detachable sensor module for vision sensor 305. The sensor module may be attached (e.g., fastened, fixed, connected, glued, etc.) to the camera module (e.g., a structure of the camera module) to form the image capture module 310. Additionally or alternatively, the sensor module and camera module may be attached to the image capture module 310 and / or a structure of the user equipment 300 to form the image capture module 310. In this way, the image capture module 310 may be installed within the user equipment 300 as a single assembled unit or as a detachable unit (e.g., before mounting the display surface).

[0050] In Example 300, the image capture module 310 may be mounted on the display side of the user equipment 300 (referred to herein and shown as display 330). For example, display 330 may allow a user to view media and / or images captured (in real time) by camera 315 (and / or vision sensor 305). Furthermore, one or more user interfaces (e.g., buttons, touchscreens, etc.) may be located on the display side to allow the user to control one or more features of camera 315 (e.g., zoom, focus, aspect ratio, resolution, etc.). As described herein, the user and / or user equipment 300 may control camera 315 via an application mounted on user equipment 300 that enables control of camera 315. In this manner, Figure 3 The field of view of the camera 315 and / or the vision sensor 305 may be directed towards the user operating the user equipment 300 (e.g., via the user's grip, hold, and / or support, etc., via wearable user equipment support, such as VR headsets, AR headsets, etc.). Accordingly, according to various aspects of this disclosure, the field of view of the camera 315 and / or the vision sensor 305 may be configured on the user equipment 300 to allow analysis of the eye movements of the user viewing the display 330.

[0051] In this manner, the user equipment may include an image capture module having a camera and a vision sensor, the vision sensor having a normally-on capability to allow the vision sensor to detect and / or analyze the user's eye movements and perform one or more actions associated with the camera and / or the user equipment.

[0052] As indicated above, Figure 3 This is provided as an example. Other examples may differ from the above combination. Figure 3 The example described.

[0053] Figures 4A-4C This is a diagram illustrating an example of power control associated with, at least in part, user eye movement, according to various aspects of this disclosure. Figures 4A-4C As shown, Example 400 includes a user device (shown as a VR device and corresponding to User Device 300 and / or User Device 110), which includes a controller (e.g., processor 210), a vision sensor, and a camera. As shown, the user device further includes a motion analysis module and a gaze analysis module.

[0054] While some aspects described herein are in the context of a device where the vision sensor is separate from the camera, the vision sensor and the camera can be the same device. For example, the vision sensor may correspond to a camera in a low-power mode (e.g., allowing normally-on detection capability) and / or a low-resolution mode (e.g., saving processing resources).

[0055] like Figure 4A As shown by reference numeral 410 in the accompanying drawings, the user device's vision sensor continuously captures images of the user's eyes. For example, the vision sensor may be in a normally-on mode at least in part based on the user device running an application (e.g., a VR application and / or an AR application) and / or displaying media associated with that application. In this way, the vision sensor can be configured to allow the user device to detect, track, and / or analyze the user's eyes via a motion analysis module. The motion analysis can be configured to control the user device's controller based on detected eye movements.

[0056] like Figure 4A As further illustrated by reference numeral 420 in the accompanying drawing, the camera is configured for gaze analysis. For example, the camera may provide images to the controller in a high-power mode, which can be analyzed to determine the gaze direction of the user's eyes. In high-power mode, the images may have a higher resolution than images captured in the camera's low-power mode, and the images may be provided at a relatively higher frequency than in the camera's low-power mode, among other examples.

[0057] like Figure 4A As further illustrated by reference numeral 430 in the accompanying drawing, the controller renders media for display based on gaze analysis. For example, the controller may process high-resolution images to allow it to render images on the user device's display at least in part based on the user's eye gaze direction. In this way, the user experience (e.g., VR experiences, AR experiences, etc.) can be enhanced by enabling the user device to render portions of the image at a higher resolution relative to other parts of the display that the user is not viewing.

[0058] like Figure 4BAs shown by reference numeral 440 in the accompanying drawings, the motion analysis module detects eye movements associated with media on the display of the user's device that the user cannot optically interpret. For example, as described herein, the motion analysis module may be configured to detect saccade eye movements of the user based at least in part on determining that the eye is moving at a specific rate corresponding to the user shifting focus from one object to another (or that the eye's gaze direction is changing at that rate). The motion analysis module may utilize any suitable image processing techniques to detect and / or analyze saccade eye movements and / or other eye movements that indicate optical characteristics during eye movements that the user cannot anatomically interpret.

[0059] like Figure 4B As further illustrated by reference numeral 450 in the accompanying drawings, the controller disables the camera and / or reduces the camera's capabilities. For example, the controller may turn off the camera and / or reduce the amount of power supplied to the camera. Additionally or alternatively, the controller may degrade the camera's image capture settings (e.g., reduce the camera's resolution, reduce the camera's frame rate, etc.). Accordingly, as shown by the dashed lines, the controller may disable the gaze analysis module and / or stop analyzing the user's gaze (e.g., to conserve resources associated with processing any images captured by the camera and / or vision sensors).

[0060] like Figure 4C As shown by reference numeral 460 in the accompanying drawing, the motion analysis module detects eye activity associated with media on the display that the user can optically interpret. For example, the motion analysis module may determine that the user's saccade eye movement has stopped. In some aspects, the motion analysis module may determine that the saccade eye movement has stopped at least in part based on the fact that the eye is not moving (e.g., because the eye is focused on a specific object presented on the display). Additionally or alternatively, the motion analysis module may determine that the saccade eye movement has stopped at least in part based on the rate of eye movement. For example, the motion analysis module may detect that the saccade movement has stopped even though the eye is moving because the eye is moving to track a media object moving across an area of ​​the display and / or the eye is moving slightly (or swaying) in a manner representing the user's focus on the object. In this way, the motion analysis module can detect that the saccade eye movement has stopped via an image from a vision sensor.

[0061] like Figure 4CAs further illustrated by reference numeral 470 in the accompanying drawings, the controller reactivates the camera and / or enhances its capabilities. For example, the controller may provide (or return) power to the camera and / or increase the amount of power supplied to the camera. Additionally or alternatively, the controller may improve the camera's image capture settings (e.g., increase the camera's resolution, increase the camera's frame rate, etc.). Accordingly, the controller may activate (or reactivate) the gaze analysis module to analyze user gaze in images captured by the camera (e.g., in high-power and / or high-resolution mode).

[0062] like Figure 4C As further shown by reference numeral 480 in the accompanying drawings, the controller renders media for display based on gaze analysis and / or the identified user gaze direction (such as determined by the gaze analysis module from the image of the camera).

[0063] In this way, visual sensors and / or motion analysis modules can be used to control the power of the user device during specific eye movements, thereby saving power and / or computing resources (e.g., processing and / or memory resources) that would otherwise be wasted powering the camera to capture and / or process images captured in high-power mode. These resources would also otherwise be wasted because the user cannot optically interpret the images displayed during specific eye movements.

[0064] As indicated above, Figures 4A-4C This is provided as an example. Other examples may differ from the one provided. Figures 4A-4C The example described. In practice, it is possible to have the same... Figures 4A-4C The equipment shown is compared to additional equipment, fewer equipment, different equipment, or equipment arranged differently. In practice, there may be additional processes, fewer processes, different processes, and / or these processes may be combined with... Figures 4A-4C The examples shown and / or described are executed in a different order.

[0065] Figure 5 This is an illustration of example 500 associated with power control based at least in part on user eye movement, according to various aspects of this disclosure. Example 500 provides an example of a process for analyzing images from an image set performed by a motion analysis module (e.g., the motion analysis module of example 400). The image set has a resolution defined by a pixel array 510 (shown as a 16×12 pixel array). In some aspects, the image set may depict one or more of the user's eyes to allow the motion analysis module to detect eye movement and / or the rate of eye movement.

[0066] As shown in Example 500, the pixel array is divided into subsets of the pixel array (shown herein and referred to as “Subset 1”, “Subset 2”, “Subset 3”, and “Subset 4”, and collectively as “Subsets”). Although the configuration of the subsets of the pixel array in Example 500 is shown as equally distributed pixels, sub-pixel arrays having the same set of dimensions, and / or sub-pixel arrays having the same number of pixels, one or more configurations of the subsets may vary and / or the configurations between the subsets may vary.

[0067] As shown in process 520, the motion analysis module can iteratively analyze the pixel array subsets of the images relative to the corresponding pixel array subsets of the previous images in the image set (shown herein and referred to as "previous subset 1", "previous subset 2", "previous subset 3", and "previous subset 4"). As described herein, previous subset 1 may have the same set of pixel positions as subset 1, previous subset 2 may have the same set of pixel positions as subset 2, previous subset 3 may have the same set of pixel positions as subset 3, and previous subset 4 may have the same set of pixel positions as subset 4, relative to the coordinates of the image set.

[0068] In Example 500, the motion analysis module may detect motion at least partially based on the fact that one or more pixel values ​​of subset 1 differ from those of the previous subset 1. If the motion analysis module detects motion at least partially based on subset 1 and the previous subset 1, the motion analysis module may control the settings of the camera (e.g., the camera of Example 400). For example, the motion analysis module may cause the controller to deactivate the camera, reduce the power supplied to the camera, and / or lower the camera's image capture settings. Additionally or alternatively, the motion analysis module may forgo processing and / or comparing additional subsets of images. In this way, the motion analysis module may save resources associated with comparing subset 2 and the previous subset 2, subset 3 and the previous subset 3, and / or subset 4 and the previous subset 4 in Example 500.

[0069] As shown in the figure, if the motion analysis module does not detect motion at least partially based on subset 1 and the previous subset 1, then the motion analysis module may sequentially analyze subset 2 and the previous subset 2, subset 3 and the previous subset 3 (unless motion is detected in the comparison of subset 2 and the previous subset 2), and subset 4 and the previous subset 4 (unless motion is detected in the comparison of subset 3 and the previous subset 3). If no motion is detected at least partially based on subset 4 and the previous subset 4, then the motion analysis module may iteratively perform process 520 on subsequent images.

[0070] In this way, the motion analysis module can iteratively compare similar portions of an image that have a lower resolution than the image itself. Accordingly, the motion analysis module can save computational resources by using process 520 to avoid processing every pixel of the image set.

[0071] As indicated above, Figure 5 This is provided as an example. Other examples may differ from the one provided. Figure 5 The example described.

[0072] Figure 6 This is a flowchart of an example process 600 associated with power control based at least in part on user eye movement. In some respects, Figure 6 One or more process frames may be executed by a user device (e.g., user device 110, user device 300, and / or the user device of example 400). Additionally or alternatively, Figure 6 One or more process frames may be executed by one or more components of device 200 (such as processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235 and / or sensor 240).

[0073] like Figure 6 As shown, process 600 may include receiving first image data associated with a first image set from a vision sensor of the user equipment (block 610). For example, the user equipment may receive the first image data associated with the first image set from a vision sensor of the user equipment, as described above.

[0074] like Figure 6 As further shown, process 600 may include: using an image processing model to determine a first type of eye activity depicted by the first image set (box 620). For example, a user device may use an image processing model to determine the first type of eye activity depicted by the first image set, as described above.

[0075] like Figure 6 As further shown, process 600 may include: causing a reduction in the power level of the user device's camera based at least in part on determining that the first image set depicts a first type of eye activity (box 630). For example, the user device may cause a reduction in the power level of the user device's camera based at least in part on determining that the first image set depicts a first type of eye activity, as described above.

[0076] like Figure 6 As further shown, process 600 may include: receiving second image data associated with the second image set from a vision sensor (box 640). For example, a user device may receive second image data associated with the second image set from a vision sensor, as described above.

[0077] like Figure 6As further shown, process 600 may include: using an image processing model to determine a second type of eye activity depicting the eye from the second set of images (box 650). For example, a user device may use an image processing model to determine the second type of eye activity depicting the eye from the second set of images, as described above.

[0078] like Figure 6 As further shown, process 600 may include: causing an increase in the power level of the camera based at least in part on determining that the second set of images depicts a second type of eye activity (box 660). For example, the user equipment may cause an increase in the power level of the camera based at least in part on determining that the second set of images depicts a second type of eye activity, as described above.

[0079] Process 600 may include additional aspects, such as any single aspect or any combination of aspects described below and / or in conjunction with one or more other processes described elsewhere herein.

[0080] In a first aspect, the first type of eye activity corresponds to saccadic eye movements. In a second aspect, either alone or in conjunction with the first aspect, the first type of eye activity corresponds to eye activities that do not allow the eye to interpret optical properties anatomically, and the second type of eye activity corresponds to eye activities that allow the eye to interpret optical properties anatomically.

[0081] In the third aspect, either alone or in combination with one or more of the first and second aspects, at least one of the following is true: the first image set is associated with a larger pixel count than the second image set, or the first image set has a higher resolution than the second image set.

[0082] In a fourth aspect, either alone or in combination with one or more of the first to third aspects, the vision sensor includes a low-power sensor that enables a normally open motion detection mode to detect at least one of a first type of eye activity or a second type of eye activity. In a fifth aspect, either alone or in combination with one or more of the first to fourth aspects, the normally open motion detection mode enables the vision sensor to record an image sliding window corresponding to a second image set.

[0083] In a sixth aspect, determining that the first image set depicts a first type of eye activity, either alone or in combination with one or more of the first to fifth aspects, includes: processing a first subset of pixels in the images of the first image set to determine whether the eye has moved relative to a previous image in the first image set; determining that the eye has moved in association with the first type of eye activity based at least in part on the first subset of pixels having values ​​different from corresponding pixels in the previous images; and determining that the first image set depicts the first type of eye activity based at least in part on the determination that the eye has moved in association with the first type of eye activity without processing a second subset of pixels in the images. In a seventh aspect, either alone or in combination with one or more of the first to sixth aspects, the second subset of pixels does not include pixels from the first subset of pixels.

[0084] In an eighth aspect, determining that the first image set depicts a first type of eye activity, either alone or in combination with one or more of the first to seventh aspects, includes: processing a first subset of pixels in the images of the first image set to determine whether the eye has moved relative to a previous image in the first image set; determining, at least in part, that the eye has not moved in association with the first type of eye activity, based on the first subset of pixels having values ​​identical to corresponding pixels in the previous image; processing a second subset of pixels in the images of the first image set to determine whether the eye has moved relative to a previous image in the first image set; determining, at least in part, that the eye has moved in association with the first type of eye activity, based on the second subset of pixels having values ​​different from corresponding pixels in the previous image; and determining that the first image set depicts the first type of eye activity, based at least in part on the determination that the eye has moved in association with the first type of eye activity. In a ninth aspect, either alone or in combination with one or more of the first to eighth aspects, the second subset of pixels does not include pixels from the first subset of pixels.

[0085] In the tenth aspect, the image processing model is configured, either alone or in combination with one or more of the first to ninth aspects, to detect a first type of eye activity based at least in part on a change in eye gaze direction and to detect a second type of eye activity based at least in part on a fixed eye gaze direction for a threshold time period.

[0086] In the eleventh aspect, either alone or in combination with one or more of the first to tenth aspects, the camera includes a high-resolution camera, and the vision sensor includes a low-resolution camera. In the twelfth aspect, either alone or in combination with one or more of the first to eleventh aspects, the vision sensor is configured to use less power than the camera.

[0087] In the thirteenth aspect, either alone or in combination with one or more of the first to twelfth aspects, the visual sensor includes at least one of: a camera in a low-power mode, or a camera in a low-resolution mode. In the fourteenth aspect, either alone or in combination with one or more of the first to thirteenth aspects, the camera is configured to capture a third set of images to track the user's eye gaze, wherein virtual reality rendering or augmented reality rendering is presented on the display of the user device based at least in part on the tracked user's eye gaze.

[0088] In the fifteenth aspect, alone or in combination with one or more of the first to fourteenth aspects, process 600 includes: causing a camera to capture a third set of images, processing the third set of images to track eye gaze, and rendering graphics via a display of a user interface based at least in part on eye gaze.

[0089] although Figure 6 An example box of process 600 is shown, but in some respects, process 600 may include... Figure 6 The boxes depicted in the process are compared to additional boxes, fewer boxes, different boxes, or boxes arranged differently. Additionally or alternatively, two or more boxes in process 600 can be executed in parallel.

[0090] Figure 7 This is a flowchart of an example process 700 associated with power control based at least in part on user eye movement. In some respects, Figure 7 One or more process boxes may be executed by a user device (e.g., user device 110, user device 300, and / or the user device of example 400).

[0091] like Figure 7 As shown, process 700 may include receiving first image data associated with a first image set from a vision sensor of the user equipment (block 710). For example, the user equipment may receive the first image data associated with the first image set from a vision sensor of the user equipment, as described above.

[0092] like Figure 7 As further shown, process 700 may include: using an image processing model to determine a first type of eye activity depicted by the first image set (box 720). For example, a user device may use an image processing model to determine the first type of eye activity depicted by the first image set, as described above.

[0093] like Figure 7As further shown, process 700 may include: causing a reduction in the power level of the user device's camera based at least in part on determining that the first image set depicts a first type of eye activity (box 730). For example, the user device may cause a reduction in the power level of the user device's camera based at least in part on determining that the first image set depicts a first type of eye activity, as described above.

[0094] Process 700 may include additional aspects, such as any single aspect or any combination of aspects described below and / or in conjunction with one or more other processes described elsewhere herein.

[0095] In a first aspect, process 700 includes: receiving second image data associated with a second set of images from a vision sensor; using an image processing model to determine a second type of eye activity depicted by the second set of images; and causing an increase in the power level of a camera based at least in part on the determination that the second set of images depicts the second type of eye activity.

[0096] In a second aspect, either alone or in combination with the first aspect, determining that the first image set depicts a first type of eye activity includes: processing a first subset of pixels of the images in the first image set to determine whether the eye has moved relative to a previous image in the first image set; determining that the eye has moved in association with the first type of eye activity based at least in part on the fact that the first subset of pixels has values ​​that are different from the corresponding pixels in the previous images; and determining that the first image set depicts the first type of eye activity based at least in part on the determination that the eye has moved in association with the first type of eye activity without processing a second subset of pixels of the images.

[0097] In a third aspect, determining that the first image set depicts a first type of eye activity, either alone or in combination with one or more of the first and second aspects, includes: processing a first subset of pixels in the images of the first image set to determine whether the eye has moved relative to a previous image in the first image set; determining, at least in part, that the eye has not moved in association with the first type of eye activity, based on the first subset of pixels having the same value as a corresponding pixel in the previous image; processing a second subset of pixels in the images of the first image set to determine whether the eye has moved relative to a previous image in the first image set; determining, at least in part, that the eye has moved in association with the first type of eye activity, based on the second subset of pixels having a different value than a corresponding pixel in the previous image; and determining that the first image set depicts the first type of eye activity, based at least in part on the determination that the eye has moved in association with the first type of eye activity.

[0098] although Figure 7 An example box of process 700 is shown, but in some respects, process 700 may include... Figure 7The boxes depicted in the process are compared to additional boxes, fewer boxes, different boxes, or boxes arranged differently. Additionally or alternatively, two or more boxes in process 700 can be executed in parallel.

[0099] The foregoing disclosure provides explanations and descriptions, but is not intended to be exhaustive or to limit the aspects to the precise forms disclosed. Modifications and variations may be made in light of the foregoing disclosure or may be obtained through practice.

[0100] As used herein, the term "component" is intended to be interpreted broadly as hardware, firmware, and / or a combination of hardware and software. As used herein, a processor is implemented using hardware, firmware, and / or a combination of hardware and software. It will be apparent that the systems and / or methods described herein can be implemented in various forms as hardware, firmware, and / or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limiting in any way. Therefore, the operation and behavior of these systems and / or methods are described herein without reference to any specific software code—it is understood that software and hardware can be designed to implement these systems and / or methods, at least in part, based on the description herein.

[0101] As used in this article, depending on the context, a threshold can refer to a value greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, not equal to the threshold, etc.

[0102] Although specific combinations of features are described in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of aspects. In fact, many of these features can be combined in ways not specifically described in the claims and / or not disclosed in the specification. Although each dependent claim listed below may be directly dependent on only one claim, the disclosure of aspects includes each dependent claim being combined with each other claim in this set of claims. The phrase “at least one of” refers to any combination of these items, including single members. As an example, “at least one of a, b, or c” is intended to cover: a, b, c, ab, ac, bc, and abc, as well as any combination having multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0103] The elements, actions, or instructions used herein should not be construed as critical or necessary unless explicitly stated otherwise. Furthermore, as used herein, the articles “a” and “a certain” are intended to include one or more items and may be used interchangeably with “one or more.” Additionally, as used herein, the article “the” is intended to include one or more items referenced in conjunction with the article “the” and may be used interchangeably with “one or more.” Furthermore, as used herein, the terms “set” and “group” are intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with “one or more.” Where only one item is intended, the phrase “only one” or similar language is used. Moreover, as used herein, the terms “have,” “contain,” “include,” etc., are intended to be open-ended terms. Furthermore, the phrase “at least partially based on” is intended to mean “at least partially based on” unless otherwise explicitly stated. Moreover, as used herein, the term “or” is intended to be inclusive when used in a sequence and may be used interchangeably with “and / or” unless otherwise explicitly stated (e.g., in combination with “either of” or “only one of”).

Claims

1. A method comprising: Receive first image data associated with the first image set from the visual sensor of the user device; Process a first subset of pixels from the images in the first image set to determine whether the eye has moved relative to a previous image in the first image set; In response to determining that the eye is not moving relative to the previous image, a second subset of pixels in the first image set is processed to determine whether the eye is moving relative to the previous image in the first image set, wherein the second subset of pixels does not include pixels in the first subset of pixels; Based on the eye movement determined relative to the previous image, an image processing model is used to determine a first type of eye activity depicted by the first image set; The power level of the user device's camera is reduced, at least in part, based on the determination that the first image set depicts the first type of eye activity; Receive second image data associated with the second image set from the visual sensor; The image processing model is used to determine the second type of eye activity depicted by the second image set; and The power level of the camera is increased, at least in part, based on the determination that the second set of images depicts the second type of eye activity.

2. The method as described in claim 1, wherein, The first type of eye activity corresponds to the saccadic movement of the eye.

3. The method as described in claim 1, wherein, The first type of eye activity corresponds to eye activity that does not allow the eye to interpret its optical properties anatomically, and the second type of eye activity corresponds to eye activity that allows the eye to interpret its optical properties anatomically.

4. The method of claim 1, wherein, At least one of the following: The first image set is associated with a larger pixel count than the second image set; or The first image set has a higher resolution than the second image set.

5. The method of claim 1, wherein, The vision sensor includes a low-power sensor that enables a normally open motion detection mode to detect at least one of the following: The first type of eye movement, or The second type of eye activity.

6. The method of claim 5, wherein, The normally open motion detection mode enables the visual sensor to record an image sliding window corresponding to the second image set.

7. The method of claim 1, wherein, Determining that the first image set depicts the first type of eye activity includes: The movement of the eye in association with the first type of eye activity is determined at least in part based on the fact that the second subset of pixels has values ​​different from those of corresponding pixels in the previous image; and The first image set is determined to depict the first type of eye activity based at least in part on determining the movement of the eye in association with the first type of eye activity.

8. The method of claim 1, wherein, The image processing model is configured to detect the first type of eye activity at least in part based on the eye changing its gaze direction, and to detect the second type of eye activity at least in part based on the eye maintaining a fixed gaze direction for a threshold time period.

9. The method of claim 1, wherein, The camera includes a high-resolution camera, and the vision sensor includes a low-resolution camera.

10. The method of claim 1, wherein, The vision sensor is configured to use less power than the camera.

11. The method of claim 1, wherein, The visual sensor includes at least one of the following: The camera in low power mode, or The camera in low-resolution mode.

12. The method of claim 1, wherein, The camera is configured to capture a third set of images to track eye gaze. Virtual reality or augmented reality rendering is presented on the display of the user device, at least in part, based on the tracked eye gaze.

13. The method of claim 1, further comprising: The camera is prompted to capture a third set of images; The third image set is processed to track the eye's gaze; as well as Graphics are rendered via a user interface display, at least in part, based on the eye gaze.

14. An apparatus comprising: One or more memory units; as well as One or more processors communicatively coupled to the one or more memories, the one or more processors being configured to: Receive first image data associated with a first image set from a vision sensor; Process a first subset of pixels from the images in the first image set to determine whether the eye has moved relative to a previous image in the first image set; In response to determining that the eye is not moving relative to the previous image, a second subset of pixels in the first image set is processed to determine whether the eye is moving relative to the previous image in the first image set, wherein the second subset of pixels does not include pixels in the first subset of pixels; Based on the eye movement determined relative to the previous image, an image processing model is used to determine a first type of eye activity depicted by the first image set; The camera's power level is reduced, at least in part, based on the determination that the first image set depicts the first type of eye activity; Receive second image data associated with the second image set from the visual sensor; The image processing model is used to determine the second type of eye activity depicted by the second image set; and The power level of the camera is increased, at least in part, based on the determination that the second set of images depicts the second type of eye activity.

15. The device as claimed in claim 14, wherein, The first type of eye activity corresponds to the saccadic movement of the eye.

16. The device as claimed in claim 14, wherein, The first type of eye activity corresponds to eye activity that does not allow the eye to interpret its optical properties anatomically, and the second type of eye activity corresponds to eye activity that allows the eye to interpret its optical properties anatomically.

17. The device as claimed in claim 14, wherein, At least one of the following: The first image set is associated with a larger pixel count than the second image set; or The first image set has a higher resolution than the second image set.

18. The device as claimed in claim 14, wherein, The vision sensor includes a low-power sensor that enables a normally open motion detection mode to detect at least one of the following: The first type of eye movement, or The second type of eye activity.

19. The device as claimed in claim 18, wherein, The normally open motion detection mode enables the visual sensor to record an image sliding window corresponding to the second image set.

20. The device as claimed in claim 14, wherein, To determine that the first image set depicts the first type of eye activity, the one or more processors are further configured to: The movement of the eye in association with the first type of eye activity is determined at least in part based on the fact that the second subset of pixels has values ​​that are different from the corresponding pixels in the previous image; as well as The first image set is determined to depict the first type of eye activity based at least in part on determining the movement of the eye in association with the first type of eye activity.

21. The device as claimed in claim 14, wherein, The image processing model is configured to detect the first type of eye activity at least in part based on the eye changing its gaze direction, and to detect the second type of eye activity at least in part based on the eye maintaining a fixed gaze direction for a threshold time period.

22. The device as claimed in claim 14, wherein, The camera includes a high-resolution camera, and the vision sensor includes a low-resolution camera.

23. The device as claimed in claim 14, wherein, The vision sensor is configured to use less power than the camera.

24. The device as claimed in claim 14, wherein, The visual sensor includes at least one of the following: The camera in low power mode, or The camera in low-resolution mode.

25. The device as claimed in claim 14, wherein, The camera is configured to capture a third set of images to track eye gaze. Virtual reality or augmented reality rendering is presented on the display of the user device, at least in part, based on the tracked eye gaze.

26. The device as claimed in claim 14, wherein, The one or more processors are further configured to: The camera is prompted to capture a third set of images; Process the third image set to track the eye's gaze; and Graphics are rendered via a user interface display, at least in part, based on the eye gaze.

27. A non-transient computer-readable medium storing an instruction set, the instruction set comprising: One or more instructions that, when executed by one or more processors of the device, cause the device to perform the following operations: Receive first image data associated with a first image set from a vision sensor; Process a first subset of pixels from the images in the first image set to determine whether the eye has moved relative to a previous image in the first image set; In response to determining that the eye is not moving relative to the previous image, a second subset of pixels in the first image set is processed to determine whether the eye is moving relative to the previous image in the first image set, wherein the second subset of pixels does not include pixels in the first subset of pixels; Based on the eye movement determined relative to the previous image, an image processing model is used to determine a first type of eye activity depicted by the first image set; The camera's power level is reduced, at least in part, based on the determination that the first image set depicts the first type of eye activity; Receive second image data associated with the second image set from the visual sensor; The image processing model is used to determine the second type of eye activity depicted by the second image set; and The power level of the camera is increased, at least in part, based on the determination that the second set of images depicts the second type of eye activity.

28. The non-transient computer-readable medium of claim 27, wherein, The first image set has a higher resolution than the second image set.

29. The non-transient computer-readable medium of claim 27, wherein, The camera includes a high-resolution camera, and the vision sensor includes a low-resolution camera.

30. The non-transient computer-readable medium of claim 27, wherein, The vision sensor is configured to use less power than the camera.

31. The non-transient computer-readable medium of claim 27, wherein, The visual sensor includes at least one of the following: The camera in low power mode, or The camera in low-resolution mode.

32. An equipment comprising: A means for receiving first image data associated with a first image set from a vision sensor; A means for processing a first subset of pixels of images in the first image set to determine whether an eye has moved relative to a previous image in the first image set; A means for processing a second subset of pixels of the first image set in response to determining that the eye is not moving relative to the previous image, to determine whether the eye is moving relative to the previous image of the first image set, wherein the second subset of pixels does not include pixels in the first subset of pixels; A means for determining eye movement based on the previous image and using an image processing model to determine a first type of eye activity depicted by the first image set; as well as A means for causing a reduction in the power level of a camera based at least in part on determining that the first set of images depicts the first type of eye activity.

33. The equipment as claimed in claim 32, further comprising: A means for receiving second image data associated with a second image set from the visual sensor; A means for using the image processing model to determine a second type of eye activity depicted by the second image set; as well as A means for causing an increase in the power level of the camera based at least in part on determining that the second set of images depicts the second type of eye activity.

Images