Transitions between power states based on environmental context

Abstract

Some examples in this disclosure relate to systems and methods for transitioning between power states based on environmental context. [Solution] In some examples, the first electronic device includes a first power state and a second power state, the second power state being a higher power state than the first power state. In some examples, while in the first power state, the first electronic device detects information that satisfies one or more criteria, and as a result, the first electronic device transitions from the first power state to the second power state. In some examples, the first electronic device activates one or more displays while in the second power state. In some examples, by activating one or more displays while in the second power state, the first electronic device can reduce its power consumption by displaying information only for the relevant time.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 699,637, filed Sep. 26, 2024, and U.S. Patent Application No. 19 / 256,671, filed Jul. 1, 2025, the contents of which are hereby incorporated by reference in their entirety for all purposes.

[0002] The present invention generally relates to systems and methods for transitioning between power states based on an environmental context. Specifically, it relates to activating one or more input devices or one or more displays based on an environmental context.

Background Art

[0003] Some computer - graphical environments provide computer - generated two - dimensional and / or three - dimensional environments in which at least some of the objects presented to the user are virtual. In some examples, the objects include one or more user - interface elements that are presented in response to one or more environmental - context information. [[ID=二十]]

Summary of the Invention

[0004] Some examples of this disclosure relate to systems and methods for transitioning between power states based on environmental context. Specifically, this relates to activating one or more input devices or one or more displays based on environmental context. In some examples, a first electronic device includes a first power state and a second power state, the second power state being a higher power state than the first power state. In some examples, while in the first power state, the first electronic device detects information that satisfies one or more criteria, and as a result, the first electronic device transitions from the first power state to the second power state. In some examples, the first electronic device activates one or more displays while in the second power state. In some examples, by activating one or more displays after detecting information that satisfies one or more criteria for transitioning the first electronic device to the second power state, the first electronic device is able to turn on one or more displays and / or display relevant content only at appropriate times, thereby reducing the power consumption of the first electronic device.

[0005] A complete description of these examples is provided in the "Drawings" and "Modes for Carrying Out the Invention," and it is understood that this "Summary of the Invention" does not in any way limit the scope of this disclosure.

[0006] To improve the understanding of the various examples described herein, refer to the following “Modes for Carrying Out the Invention” together with the following drawings. Similar reference numbers often refer to corresponding parts throughout the drawings. [Brief explanation of the drawing]

[0007] [Figure 1] This disclosure illustrates several examples of electronic devices that present an extended reality environment.

[0008] [Figure 2A]The following are block diagrams illustrating the exemplary architecture of a device, based on several examples of this disclosure. [Figure 2B] The following are block diagrams illustrating the exemplary architecture of a device, based on several examples of this disclosure.

[0009] [Figure 3A] Several examples of this disclosure illustrate how a first electronic device transitions between different power states based on information collected from one or more input devices. [Figure 3B] Several examples of this disclosure illustrate how a first electronic device transitions between different power states based on information collected from one or more input devices. [Figure 3C] Several examples of this disclosure illustrate how a first electronic device transitions between different power states based on information collected from one or more input devices. [Figure 3D] Several examples of this disclosure illustrate how a first electronic device transitions between different power states based on information collected from one or more input devices.

[0010] [Figure 4A] Several examples of this disclosure illustrate different cases in which an electronic device transitions from a first power state to a second power state based on information collected from one or more input devices. [Figure 4B] Several examples of this disclosure illustrate different cases in which an electronic device transitions from a first power state to a second power state based on information collected from one or more input devices. [Figure 4C] Several examples of this disclosure illustrate different cases in which an electronic device transitions from a first power state to a second power state based on information collected from one or more input devices.

[0011] [Figure 5A]Several examples of this disclosure illustrate how an electronic device transitions from a first power state to a second power state based on non-contextual information collected from one or more input devices. [Figure 5B] Several examples of this disclosure illustrate how an electronic device transitions from a first power state to a second power state based on non-contextual information collected from one or more input devices.

[0012] [Figure 6] The following flowcharts illustrate exemplary processes for transitioning an electronic device from a first power state to a second power state, using several examples from this disclosure. [Modes for carrying out the invention]

[0013] Some examples of this disclosure relate to systems and methods for transitioning between power states based on environmental context. Specifically, this relates to activating one or more input devices or one or more displays based on environmental context. In some examples, a first electronic device includes a first power state and a second power state, the second power state being a higher power state than the first power state. In some examples, while in the first power state, the first electronic device detects information that satisfies one or more criteria, and as a result, the first electronic device transitions from the first power state to the second power state. In some examples, the first electronic device activates one or more displays while in the second power state. In some examples, by activating one or more displays after detecting information that satisfies one or more criteria for transitioning the first electronic device to the second power state, the first electronic device is able to turn on one or more displays and / or display relevant content only at appropriate times, thereby reducing the power consumption of the first electronic device.

[0014] Figure 1 shows an electronic device 101 presenting an extended reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects) in some examples of the present disclosure. In some examples, as shown in Figure 1, the electronic device 101 is a head-mounted display or other head-mountable device configured to be worn on the head of the user of the electronic device 101. Examples of the electronic device 101 are described below with reference to the architectural block diagram in Figure 2A. As shown in Figure 1, the electronic device 101 and the table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floor, wall) or physical objects (e.g., table, lamp, etc.). In some examples, the electronic device 101 may be configured to detect and / or capture an image of the physical environment, including the table 106 (shown within the field of view of the electronic device 101).

[0015] In some examples, as shown in Figure 1, the electronic device 101 includes one or more internal image sensors 114a (e.g., an eye-tracking camera described below with reference to Figures 2A-2B) oriented toward the user's face. In some examples, the internal image sensors 114a are used for eye tracking (e.g., detecting the user's gaze). The internal image sensors 114a are optionally positioned in the left and right portions of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c facing outward from the user to detect and / or capture the movement of the physical environment of the electronic device 101 and / or the user's hands or other body parts.

[0016] In some examples, the display 120 has a field of view visible to the user, which may or may not correspond to the field of view of external image sensors 114b and 114c. Since the display 120 is optionally part of a head-mounted device, the field of view of the display 120 is optionally the same as or similar to the field of view of the user's eyes. In other examples, the field of view of the display 120 may be smaller than the field of view of the user's eyes. In some examples, the electronic device 101 may be an optical see-through device in which the display 120 is a transparent or translucent display through which a portion of the physical environment can be directly viewed. In some examples, the display 120 may be contained within a transparent lens, or overlap the entire transparent lens or only a portion thereof. In other examples, the electronic device may be a video pass-through device in which the display 120 is an opaque display configured to display an image of the physical environment captured by external image sensors 114b and 114c. Although a single display 120 is shown, it should be understood that the display 120 may include a stereo pair of displays. In Figures 2A and 2B, the display 120 includes or corresponds to a transparent or translucent surface (e.g., a lens) that does not have display capabilities (e.g., therefore cannot generate and display virtual objects 104), and instead presents a direct view of the physical environment within the user's field of view (e.g., the user's field of vision).

[0017] In some examples, the electronic device 101 is configured to display a virtual object 104 in a three-dimensional environment (for example, in response to a trigger). The virtual object 104 is represented by a cube, as shown in Figure 1, which does not exist in the physical environment but is displayed in the three-dimensional environment by being placed on a table 106 (for example, a real-world table or its representation). Optionally, the virtual object 104 is displayed on the surface of the table 106 in the three-dimensional environment, as shown via the display 120 of the electronic device 101, in response to the detection of a plane of the table 106 in the physical environment 100.

[0018] The virtual object 104 is a representative virtual object, and it is understood that one or more different virtual objects (such as two-dimensional or other three-dimensional virtual objects with various numbers of dimensions) can be included in the three-dimensional environment and rendered. For example, the virtual object can represent an application or a user interface displayed within the three-dimensional environment. In some examples, the virtual object can represent the content corresponding to the application and / or the content displayed via the user interface in the three-dimensional environment. In some examples, the virtual object 104 is optionally interactive so that the user can interact with the virtual object 104 by virtually touching, tapping, moving, rotating, or otherwise, and is configured to respond to user input (such as air gestures such as air pinch gestures, air tap gestures, and / or air touch gestures).

[0019] As described herein, one or more air pinch gestures performed by the user (such as using the hand 103 in FIG. 1) are detected by one or more input devices of the electronic device 101 and interpreted as one or more user inputs directed to the content displayed by the electronic device 101. Additionally or alternatively, in some examples, one or more user inputs interpreted by the electronic device 101 as being directed to the content (such as the virtual object 104) displayed by the electronic device 101 are detected via one or more hardware input devices (such as a controller, touch pad, proximity sensor, button, slider, knob, etc.) rather than via one or more input devices configured to detect air gestures such as one or more air pinch gestures performed by the user. Such depictions are intended to be illustrative rather than limiting, and the user can optionally provide user input using different air gestures and / or other forms of input.

[0020] In some examples, the electronic device 101 may be configured to communicate with a second electronic device, such as a companion device. For example, as shown in FIG. 1, the electronic device 101 optionally communicates with the electronic device 160. In some examples, the electronic device 160 corresponds to a mobile electronic device such as a smartphone, a tablet computer, a smartwatch, a laptop computer, or other electronic device. In some examples, the electronic device 160 corresponds to a non-mobile electronic device that is generally stationary and not easily moved within a physical environment (e.g., a desktop computer, a server, etc.). Additional examples of the electronic device 160 are described below with reference to the architecture block diagram of FIG. 2B. In some examples, the electronic device 101 and the electronic device 160 are associated with the same user. For example, in FIG. 1, the electronic device 101 may be disposed (e.g., worn on the head) on the user's head, and the electronic device 160 may be disposed near the electronic device 101, such as in the user's hand 103 (e.g., the hand 103 holds the electronic device 160), the user's pocket or bag, or on a surface near the user. The electronic device 101 and the electronic device 160 are optionally associated with the same user account of the user (e.g., the user is logged in to user accounts on the electronic device 101 and the electronic device 160). Additional details regarding the communication between the electronic device 101 and the electronic device 160 are provided below with reference to FIGS. 2A-2B.

[0021] In some examples, displaying an object within a three-dimensional environment is triggered by or enables interaction with one or more user interface objects within the three-dimensional environment. For example, initiating the display of an object within a three-dimensional environment may involve interaction with one or more virtual options / affordances displayed within the three-dimensional environment. In some examples, the user's gaze may be tracked by an electronic device as input to identify one or more virtual options / affordances to be selected when initiating the display of an object within the three-dimensional environment. For example, gaze can be used to identify one or more virtual options / affordances to be selected using another selection input. In some examples, virtual options / affordances may be selected using hand-tracking input detected via an input device communicating with an electronic device. In some examples, an object displayed within a three-dimensional environment may be moved and / or oriented within the three-dimensional environment according to a movement input detected via an input device.

[0022] The following description describes an electronic device that communicates with one or more displays and one or more input devices. The electronic device is understood to optionally communicate with one or more other physical user interface devices, such as a touch-sensitive surface, a physical keyboard, a mouse, a joystick, a hand-tracking device, an eye-tracking device, or a stylus. Furthermore, as described above, the described electronic device, display, and touch-sensitive surface are understood to optionally be distributed between two or more devices. Therefore, as used in this disclosure, information displayed on or by an electronic device is used optionally to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on an electronic device (e.g., touch input received on the touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) is used optionally to describe input received on an input device separate from the input device from which the electronic device receives input information.

[0023] The device typically supports a variety of applications, including drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, workout support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications, one or more of these.

[0024] Figures 2A and 2B show block diagrams of exemplary architectures of electronic devices according to some examples of the present disclosure. In some examples, electronic device 201 and / or electronic device 260 comprises one or more electronic devices. For example, each electronic device 201 could be a portable device, an auxiliary device for communicating with another device, a head-mounted display, a head-mounted speaker, and so on. In some examples, electronic device 201 corresponds to electronic device 101 described with reference to Figure 1. In some examples, electronic device 260 corresponds to electronic device 160 described with reference to Figure 1.

[0025] As shown in Figure 2A, the electronic device 201 optionally includes one or more sensors, such as one or more hand tracking sensors 202, one or more location sensors 204A, one or more image sensors 206A (optionally corresponding to internal image sensors 114a and / or external image sensors 114b, 114c in Figure 1), one or more touch-sensing surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye tracking sensors 212, one or more microphones 213A or other sound sensors, one or more body tracking sensors (e.g., torso and / or head tracking sensors). The electronic device 201 optionally includes one or more output devices, such as one or more display generating components 214A (optionally corresponding to the display 120 in Figure 1), one or more speakers 216A, one or more tactile output devices (not shown). The electronic device 201 optionally includes one or more processors 218A, one or more memories 220A, and / or communication circuits 222A. Optionally, one or more communication buses 208A are used for communication between the aforementioned components of the electronic device 201.

[0026] In addition, the electronic device 260 optionally includes the same or similar components as the electronic device 201. For example, as shown in Figure 2B, the electronic device 260 optionally includes one or more location sensors 204B, one or more image sensors 206B, one or more touch-sensing surfaces 209B, one or more orientation sensors 210B, one or more microphones 213B, one or more display generation components 214B, one or more speakers 216B, one or more processors 218B, one or more memories 220B, and / or communication circuits 222B. Optionally, one or more communication buses 208B are used for communication between the aforementioned components of the electronic device 260.

[0027] Electronic devices 201 and 260 are optionally configured to communicate via a wired or wireless connection between the two electronic devices (for example, via communication circuits 222A and 222B). For example, as shown in Figure 2A, electronic device 260 may function as a companion device to electronic device 201. For example, in some examples, electronic device 260 processes sensor inputs from electronic devices 201 and 260 and / or generates content for display using the display generation component 214A of electronic device 201.

[0028] Communication circuits 222A and 222B optionally include circuits for communicating with electronic devices and networks such as the Internet, intranets, wired networks and / or wireless networks, cellular networks and wireless local area networks (LANs). Communication circuits 222A and 222B optionally include circuits for communicating using short-range communication such as near-field communication (NFC) and / or Bluetooth®. In some examples, communication circuits 222A and 222B include Wi-Fi (e.g., 802.11 protocol), Ethernet, ultra-wideband ("UWB"), high-frequency systems (e.g., 900MHz, 2.4GHz, and 5.6GHz communication systems), or any other communication protocol, or any combination thereof.

[0029] One or more processors 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, one or more processors 218A, 218B include one or more microprocessors, one or more central processing units, one or more application-specific integrated circuits, one or more field-programmable gate arrays, one or more programmable logic devices, or a combination of such devices. In some examples, memories 220A and / or 220B are non-temporary computer-readable storage media (e.g., flash memory, random-access memory, or other volatile or non-volatile memory or storage device) that store computer-readable instructions executed by one or more processors 218A, 218B and configured to perform the techniques, processes, and / or methods described herein. In some examples, memories 220A and / or 220B may include two or more non-temporary computer-readable storage media. Non-temporary computer-readable storage media can be any (non-signal) medium capable of tangibly storing or storing computer-executable instructions used by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is temporary computer-readable storage media. In some examples, the storage medium is non-temporary computer-readable storage media. Non-temporary computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, compact discs (CDs), digital versatile discs (DVDs), or optical discs based on Blu-ray® technology, as well as persistent solid-state memory such as flash drives and solid-state drives.

[0030] In some examples, one or more display generating components 214A, 214B include a single display (e.g., a liquid-crystal display (LCD), an organic light-emitting diode (OLED), or another type of display). In some examples, one or more display generating components 214A, 214B include multiple displays. In some examples, one or more display generating components 214A, 214B may include touch-enabled displays (e.g., touchscreens), projectors, holographic projectors, retinal projectors, transparent or translucent displays, etc. In some examples, the electronic device does not include one or more display generating components 214A or 214B. For example, instead of one or more display generating components 214A or 214B, some electronic devices include transparent or translucent lenses or other surfaces that are not configured to display or present virtual content. However, in such cases, it should be understood that electronic device 201 and / or electronic device 260 may optionally be equipped with one or more of the other components shown in Figures 2A and 2B and described herein, such as one or more hand tracking sensors 202, one or more eye tracking sensors 212, one or more image sensors 206A, and / or one or more motion and / or orientation sensors 210A. Alternatively, in some examples, one or more display generating components 214A or 214B are provided separately from electronic devices 201 and / or 260. For example, one or more display generating components 214A, 214B communicate with electronic device 201 (and / or electronic device 260) but are not integrated with electronic devices 201 and / or electronic devices 260 (e.g., within the housing of electronic devices 201, 260).In some examples, electronic devices 201 and 260 each include one or more touch-sensitive surfaces 209A and 209B for receiving user input such as tap input and swipe input or other gestures (e.g., hand gestures or finger gestures). In some examples, one or more display generating components 214A, 214B and one or more touch-sensitive surfaces 209A, 209B form one or more touch-sensitive displays (e.g., touchscreens integrated with each of electronic devices 201 and 260, or external touchscreens for each of electronic devices 201 and 260 that communicate with each of electronic devices 201 and 260).

[0031] Each electronic device 201 and 260 optionally includes one or more image sensors 206A and 206B. The one or more image sensors 206A, 206B optionally include one or more visible light image sensors, such as charge-coupled device (CCD) sensors and / or complementary metal-oxide-semiconductor (CMOS) sensors, capable of operating to acquire images of physical objects from a real-world environment. The one or more image sensors 206A, 206B also optionally include one or more IR sensors, such as passive or active infrared (IR) sensors, for detecting infrared light from a real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The one or more image sensors 206A, 206B also optionally include one or more cameras configured to capture the movement of physical objects in a real-world environment. One or more image sensors 206A, 206B also optionally include one or more depth sensors configured to detect the distance of a physical object from electronic devices 201, 260. In some examples, information from one or more depth sensors can enable the device to identify an object in a real-world environment and distinguish it from other objects in the real-world environment. In some examples, one or more depth sensors can enable the device to determine the texture and / or topography of an object in a real-world environment. In some examples, one or more image sensors 206A or 206B are contained within a different electronic device than electronic devices 201 and / or 260. For example, one or more image sensors 206A, 206B communicate with electronic devices 201, 260 but are not integrated with electronic devices 201, 260 (e.g., within the housing of electronic devices 201, 260).In particular, in some examples, one or more cameras with one or more image sensors 206A, 206B are integrated and / or coupled with one or more devices separate from electronic devices 201 and / or 260, such as one or more input and / or output devices containing one or more image sensors 206A, 206B (e.g., one or more speakers such as earphones or headphones and / or one or more microphones) (e.g., but communicating with electronic devices 201 and / or 260). In some examples, electronic device 201 or electronic device 260 corresponds to a head-worn speaker (e.g., headphones or earbuds). In such cases, electronic device 201 or electronic device 260 is equipped with a subset of other components shown in Figures 2A and 2B and described herein. In some such examples, the electronic device 201 or electronic device 260 is equipped with one or more image sensors 206A, 206B, one or more motion and / or orientation sensors 210A, 210B, and / or speakers 216A, 216B.

[0032] In some examples, electronic devices 201, 260 use a combination of CCD sensors, event cameras, and depth sensors to detect the physical environment surrounding the electronic devices 201, 260. In some examples, one or more image sensors 206A, 206B include a first image sensor and a second image sensor. The first and second image sensors work together and are configured to selectively capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor and the second image sensor is a depth sensor. In some examples, electronic devices 201, 260 use one or more image sensors 206A, 206B to detect the position and orientation of electronic devices 201, 260 and / or one or more display generating components 214A, 214B in the real-world environment. For example, electronic devices 201, 260 use one or more image sensors 206A, 206B to track the position and orientation of one or more display generating components 214A, 214B relative to one or more stationary objects in a real-world environment.

[0033] In some examples, electronic devices 201 and 260 each include one or more microphones 213A and 213B, or other sound sensors. Electronic devices 201 and 260 optionally use one or more microphones 213A and 213B to detect sounds from the user and / or the user's real-world environment. In some examples, one or more microphones 213A and 213B optionally include an array of microphones (multiple microphones) working together to identify ambient noise or to locate sound sources in the space of the real-world environment.

[0034] Each electronic device 201 and 260 includes one or more location sensors 204A and 204B for detecting the location of electronic device 201 and / or one or more display generating components 214A, and the location of electronic device 260 and / or one or more display generating components 214B, respectively. For example, one or more location sensors 204A, 204B may include a global positioning system (GPS) receiver that receives data from one or more satellites, enabling electronic devices 201, 260 to determine the absolute position of the devices in the physical world.

[0035] Electronic devices 201 and 260 each include one or more orientation sensors 210A and 210B for detecting the orientation and / or movement of electronic device 201 and / or one or more display generating components 214A, and the orientation and / or movement of electronic device 260 and / or one or more display generating components 214B, respectively. For example, electronic devices 201 and 260 use one or more orientation sensors 210A and 210B to track changes in the position and / or orientation of electronic devices 201 and 260 and / or one or more display generating components 214A and 214B relative to physical objects in a real-world environment. One or more orientation sensors 210A and 210B optionally include one or more gyroscopes and / or one or more accelerometers.

[0036] In some examples, the electronic device 201 includes one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212. Although referred to as hand tracking sensors or eye tracking sensors, it is understood that the electronic device 201 optionally includes, in addition to or instead, one or more other body tracking sensors, such as one or more leg tracking sensors, one or more torso tracking sensors, and / or one or more head tracking sensors. One or more hand tracking sensors 202 are configured to track the position and / or location of one or more parts of the user's hand, as well as / or the motion of one or more parts of the user's hand, relative to a three-dimensional environment, one or more display generating components 214A, and / or another defined coordinate system. One or more eye tracking sensors 212 are configured to track the position and movement of the user's gaze (e.g., more commonly, the user's attention, including the eyes, face, or head) relative to the real world or a three-dimensional environment and / or one or more display generating components 214A. In some examples, one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212 are integrated with one or more display generating components 214A. In some examples, one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212 are implemented separately from one or more display generating components 214A. In some examples, the electronic device 201 does not include one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212.In some examples, one or more display generation components 214A may be utilized by an electronic device 260 to provide a three-dimensional environment, and the electronic device 260 may utilize inputs and other data collected via one or more other sensors of the electronic device 201 (e.g., one or more location sensors 204A, one or more image sensors 206A, one or more touch-sensing surfaces 209A, one or more motion and / or orientation sensors 210A, and / or one or more microphones 213A or other sound sensors) as inputs and data processed by one or more processors 218B of the electronic device 260. In addition or instead, the electronic device 260 may optionally not include other components shown in Figure 2B, such as one or more location sensors 204B, one or more image sensors 206B, one or more touch-sensing surfaces 209B. In some such examples, one or more display generating components 214A may be utilized by an electronic device 260 to provide a three-dimensional environment, and the electronic device 260 may utilize inputs and other data collected via one or more motion and / or orientation sensors 210A (and / or one or more microphones 213A) of the electronic device 201 as input.

[0037] In some examples, one or more hand tracking sensors 202 (and / or other body tracking sensors such as leg tracking sensors, torso tracking sensors, and / or head tracking sensors) may use one or more image sensors 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that capture three-dimensional information from the real world, including one or more body parts (e.g., the hands, legs, or torso of a human user). In some examples, the hand may be resolved with sufficient resolution to distinguish the fingers and the position of each of them. In some examples, one or more image sensors 206A are positioned relative to the user so as to define the field of view of the one or more image sensors 206A and the interaction space in which the position, orientation, and / or movement of the fingers / hands captured by the image sensors are used as input (e.g., to distinguish the user's stationary hand from the other hands of other people in the real world environment). Tracking fingers / hands for input (e.g., gestures, touches, taps, etc.) may have advantages in that it does not require the user to touch, hold, or wear any kind of beacon, sensor, or other marker.

[0038] In some examples, one or more eye-tracking sensors 212 include at least one eye-tracking camera (e.g., an IR camera) and / or an illumination source (e.g., an IR light source such as an LED) that emits light toward the user's eye. The eye-tracking camera may be directed toward the user's eye so as to receive reflected IR light from the light source directly or indirectly through the eye. In some examples, both eyes are tracked separately by their respective eye-tracking cameras and illumination sources, and the focus / line of sight can be determined from tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more respective eye-tracking cameras / illumination sources.

[0039] Electronic devices 201 and 260 are not limited to the components and configurations shown in Figures 2A and 2B, and may include fewer, other, or additional components in multiple configurations. In some examples, electronic device 201 and / or electronic device 260 can each be implemented (e.g., as a system) among multiple electronic devices. In some such examples, each (or more) electronic device may include one or more of the same components described above, such as various sensors, one or more display generating components, one or more speakers, one or more processors, one or more memories, and / or communication circuits. Persons(s) who use electronic device 201 and / or electronic device 260 are optionally referred to herein as users(s).

[0040] Here, we draw our attention to transitioning a first electronic device (e.g., electronic device 201) from a first power state to a second power state based on contextual information provided in the examples of this disclosure. In some examples, the first electronic device transitions states based on contextual information from the first electronic device or from a second electronic device (e.g., electronic device 260) communicating with the first electronic device. In some examples, the first electronic device is a wearable device having one or more output devices (e.g., a display and / or a speaker), and the second electronic device includes one or more sensors that can be used to provide contextual information to the first electronic device. As will be described in more detail herein, the contextual information is optionally based on one or more sensors of the first electronic device and one or more sensors of the second electronic device. Acquiring information from one or more sensors of a second electronic device to determine contextual information for the first electronic device can improve the user experience (e.g., displaying information to the user of the first electronic device at the right time with little or no additional user input) and / or reduce the power consumption, weight, and cost of the first electronic device.

[0041] In some examples, a first electronic device transitions between different power states to conserve power. For example, the first electronic device receives data providing contextual information from a sensor of a second electronic device, which allows the first electronic device to remain in a relatively low-power state (e.g., one or more output devices are off or in a low-power operating state, and one or more sensors are off or in a low-power operating state). The first electronic device can transition to a higher power state if the contextual information meets one or more criteria. In some examples, a higher power state includes turning on an output device or sensor, or entering a higher power state, such as turning on one or more displays, processing at higher power (e.g., more frequently than while in a low-power state), having a higher refresh rate, and / or running background applications. In some examples, transitioning to a higher power state (e.g., a second power state, as described below) includes turning on an additional processor. For example, while in a low-power state, an electronic device uses a low-power processor to monitor one or more sensors / input devices operating in the low-power state and / or determine the environmental context of the electronic device. Transitioning to a higher power state involves activating an additional processor (e.g., a higher-power processor) to monitor / operate additional input devices. The power states of electronic devices described or used herein are characterized by the on / off state or operating speed of one or more output devices of a first electronic device (e.g., a display, speaker, haptic driver, etc.) and by the on / off state or operating rate of one or more sensors (e.g., one or more input devices). As used herein, on / off states refer to whether a component (e.g., an input or output device) is powered (on) or not (off), and power is supplied when the component receives a threshold voltage and / or current from the power supply.In some examples, more components of the first electronic device are turned on while it is in a high-power state (e.g., the second state) than while it is in a low-power state (e.g., the first state).

[0042] Figures 3A to 3D illustrate an example in which a first electronic device transitions between different power states based on information collected from one or more input devices. Figures 3A to 3D are used to illustrate the processes described below, including process 600 shown in Figure 6. In some examples, as shown in Figures 3A to 3D, a user may have a routine of consuming media (e.g., listening to music) on their way to work. Using one or more input devices in the first power state, the electronic device 101 may be able to determine the environmental context and initiate a process for the user to consume the respective media. In some examples, as described below, the electronic device 101 may transition to a second power state to display one or more user interface elements to facilitate the user's consumption of media.

[0043] Figure 3A shows an electronic device 101 presenting a portion 300 of the physical environment 350 from the user's perspective via a display 120. In Figure 3A, the electronic device 101 (corresponding, for example, to either electronic device 201 / 260) optionally presents the portion 300 via a video pass-through or optical see-through display. Figure 3A shows that the user 317 of the electronic device 101 is facing a bus stop 312 and a bus 313 is approaching the bus stop.

[0044] In some examples, the user's viewpoint influences which content (e.g., physical and / or virtual objects) is visible in the viewport (e.g., a view of the physical environment 350 visible to the user via one or more displays 120, or a pair of display modules providing stereoscopic content to different eyes of the same user). In some examples, the (virtual) viewport has a viewport boundary that defines the extent of the physical environment 350 visible to the user via the displays 120 in Figures 3A–3D. In some examples, the area defined by the viewport boundary is smaller in one or more dimensions than the user's field of view (based on, for example, the user's field of view, the size of one or more displays, optical properties or other physical characteristics, and / or the location and / or orientation of one or more displays relative to the user's eyes). In some examples, the area defined by the viewport boundary is larger in one or more dimensions than the user's field of view (based on, for example, the user's field of view, the size of one or more displays, optical properties or other physical characteristics, and / or the location and / or orientation of one or more displays relative to the user's eyes). Viewports and viewport boundaries typically move as one or more displays move (for example, with the user's head in the case of a head-mounted device, or with the user's hand in the case of a handheld device such as a tablet or smartphone). The user's viewpoint influences which content is visible within the viewport, and the viewpoint generally specifies the location and orientation relative to the physical environment 350, so that as the viewpoint shifts, the view of the physical environment 350 also shifts within the viewport. In the case of a head-mounted device, the viewpoint typically provides a perceptually accurate view of the three-dimensional environment based on the location and orientation of the user's head, face, and / or eyes, providing an immersive experience when the user is using the head-mounted device.For handheld or stationary devices, the viewpoint shifts as the handheld or stationary device moves and / or as the user's position relative to the handheld or stationary device changes (e.g., as the user moves toward or away from the device, above or below the device, to the right of the device, and / or to the left of the device). For devices with displays that have video passthrough, the portion of the physical environment visible (e.g., displayed and / or projected) through one or more displays is typically based on the field of view of one or more cameras communicating with the display, as the user's viewpoint moves as the field of view of one or more cameras moves (and the appearance of one or more virtual objects displayed through one or more displays is updated based on the user's viewpoint (e.g., the displayed position and orientation of virtual objects are updated based on the user's viewpoint)), thus moving with the display (e.g., moving with the user's head in the case of a head-mounted device, or moving with the user's hand in the case of a handheld device such as a tablet or smartphone). In the case of displays with optical see-through, parts of the physical environment visible through one or more displays (e.g., optically visible through one or more partially or fully transparent parts of the display-generating components) are based on the user's field of view through the partially or fully transparent parts of the display-generating components (e.g., moving with the user's head in the case of a head-mounted device, or moving with the user's hand in the case of a handheld device such as a tablet or smartphone), as the user's viewpoint moves as the user's field of view moves through the partially or fully transparent parts of the display (and the appearance of one or more virtual objects is updated based on the user's viewpoint).

[0045] In Figure 3A, electronic device 101 communicates with one or more additional devices (e.g., electronic device(s) 303 and / or 305). The combination of devices communicating with electronic device 101 is called a computer system (e.g., electronic devices(s) 101, 303, and 305). In some examples, electronic devices(s) 303 and / or 305 have one or more characteristics of electronic devices(s) 201 and / or 260. In some examples, electronic device 303 is a smartphone, and electronic device 305 is a smartwatch that communicates with electronic device 101. For example, one or more electronic devices 101, 303, and / or 305 communicate wirelessly (e.g., via Bluetooth, Wi-Fi, and / or wireless networks) or via wired communication (e.g., via wires and / or cables such as Universal Serial Bus A (USB-A), Universal Serial Bus C (USB-C), and / or Ethernet). In some examples, the one or more electronic devices 101, 303, and / or 305 share a common user account and / or user (e.g., a common user / account uses the devices). For example, a user optionally logs into one or more electronic devices using one user account (e.g., one username and password).

[0046] In some examples, one or more electronic devices 101, 303, and / or 305 each include one or more sensors and / or one or more displays. For example, an electronic device may include an accelerometer, a global positioning sensor (GPS), an image sensor (image sensors 206A and / or 206B), an orientation sensor (e.g., orientation sensors 210A and / or 210B), and / or a location sensor (e.g., location sensors 204A and / or 204B). In some examples, while in a first power state, electronic device 101 does not include a powered-on display, as shown in Figure 3A. For example, in Figure 3A, electronic device 101 does not use display 120 to display content, but instead presents a passthrough of the physical environment 350. While in the first power state, electronic devices 101, 303, and / or 305 operate a first subset of input devices, such as microphones, GPS, location sensors, compass sensors, accelerometers, and other background sensors. In some examples, while in the first power state, a second set of input devices requiring more battery power, such as image sensors, high-power processors, and displays, are inactive. In some examples, while in the first power state, electronic device 101 uses the first subset of input devices to sense and receives sensor data regarding sound, location, and orientation from the input devices of electronic devices 303 and / or 305.

[0047] Figure 3A shows an example where a user is at a bus stop to go to work. Representation 352 shows a top-down view of the physical environment 350 as user 317 walks towards bus stop 312 to board bus 313. As shown in Figure 3A, electronic device 101 presents a portion 300 of the physical environment 350, including bus stop 312 and bus 313, via display 120. In some examples, electronic devices 101, 303, and / or 305 capture one or more sounds 310a of the physical environment 350 using microphones. For example, the sounds are related to the urban outdoor environment (e.g., bird sounds, car sounds, bus sounds, airplane sounds, and / or other urban and outdoor sounds). In some examples, electronic devices 101, 303, and / or 305 also capture location data using one or more location sensors indicating that the user is at bus stop 312. In some examples, the electronic device 101 and / or computer system may store data from one or more sensors as historical data that can be used to inform one or more future actions of the device. For example, the electronic device(s) 101, 303, and / or 305 may use sound and / or location data to transition power states, as will be explained in more detail in Figures 3B to 3D.

[0048] In the examples described herein, a computer system may be described as one that performs a function (for example, a computer system stores historical data as described above). However, it should be understood that any electronic device described herein that communicates with electronic device 101 can perform any or all of the steps that constitute the individual functions. For example, a computer system consists of electronic devices 101, 303, and 305, and any electronic device within the computer system can perform any or all of the steps that constitute the individual functions described herein.

[0049] In Figure 3A, the electronic device 101 and / or the computer system detect that one or more criteria for transitioning to a power state are not met. In some examples, one or more criteria for transitioning to a power state are based on one or more previous patterns of the user. For example, the electronic device 101 and / or the computer system have not detected (e.g., via sensors on the electronic device 101 or via sensors on one or more electronic devices 303 and / or 305 communicating with the electronic device 101) that meet the criteria for a sound, a location, and / or other criteria. As described below, in Figure 3B, the electronic devices 101, 303, and / or 305 detect data that matches a previous pattern of the user (e.g., that meets one or more criteria) that requires the electronic device 101 to transition from a first power state to a second power state.

[0050] In Figure 3B, the user is seated on bus 313, and the electronic device 101 presents a portion 300 of the physical environment 350 corresponding to the user seated on bus 313. In Figure 3B, representation 352 indicates that user 317 is inside bus 313. In addition, after detecting that the electronic device 101 (and the user of the electronic device 101) is inside bus 313, the electronic device 101 transitions to a second power state and displays user interface elements 304 and 306 on display 120. In some examples, one or more criteria are met when the electronic device 101 and / or the computer system detect that the electronic device 101 is located inside bus 313. In some examples, electronic devices 101, 303, and / or 305 detect a change in sound (e.g., from sound 310a to sound 310b) corresponding to a user boarding bus 313 and being inside the bus, so electronic device 101 and / or the computer system detect that electronic device 101 is inside bus 313. Sound 310b may include sounds associated with being inside the bus (e.g., people talking and the sound of the bus). In some examples, sound 310b may be quieter than sound 310a because being inside the bus is a more enveloping environment. In some examples, electronic devices 101, 303, and / or 305 use accelerometers, GPS, location sensors, and / or compass sensors to detect changes in acceleration / movement (e.g., increases in speed and / or acceleration, changes in location, changes in orientation) that match being on the vehicle. In addition, in some examples, electronic device 101 detects that electronic device 101 is in a specific location (e.g., on bus 313) when electronic device(s) 101, 303, 305 connect to a Wi-Fi network corresponding to a specific location (e.g., a bus Wi-Fi network). Alternatively or in addition, one or more criteria may be met based on time and / or date. For example, based on previous user activity (e.g., user location data, sound data, movement data, calendar data, or other data), electronic device 101 determines that the user is on bus 313 at a specific time and / or date (e.g., 8 a.m. Monday through Friday).In some examples, the electronic device 101 may use position and / or orientation sensors to determine the position of the user 317's head. In some examples, the electronic device 101 may use eye-tracking and / or hand-tracking sensors to determine the position of one or more of the user 317's eyes. For example, the electronic device 101 may use the head-tracking sensor, hand-tracking sensor, and eye-tracking sensor to determine the environmental context (e.g., that the user is sitting inside the bus 313). In some examples, the electronic device 101 may use one or more image sensors to capture an image of the physical environment 300 and determine the environmental context. In some examples, data from a first set of input devices of electronic device(s) 303 and / or 305, as well as data from a first set of input devices of electronic device 101, are used to determine whether one or more criteria are met.

[0051] In some examples, the electronic device 101 displays user interface elements 304 and 306 depending on whether it detects that one or more criteria have been met. In some examples, user interface elements 304 and 306 correspond to the volume and play buttons of a music application, respectively. In some examples, the electronic device 101 displays user interface elements 304 and 306 depending on whether it detects that the user is on bus 313 and / or based on historical data. In some examples, the electronic device 101 receives one or more inputs corresponding to a request to play music while on bus 313 (e.g., a specific bus used at a particular time and place, and / or any bus used by a user of the electronic device 101). In some examples, as described above, the electronic device 101 and / or the computer system store sensor data (e.g., from the sensors of the electronic device 101, or from the sensors of electronic device(s) 303 and / or 305) and determine a pattern based on the user's previous actions and the corresponding respective sensor data. In some examples, one or more criteria are dynamic based on the environmental context captured by a first set of one or more sensors. For example, the electronic device 101 and / or computer system may recognize one or more patterns of user behavior (e.g., playing music in bus 313, checking tasks in a certain location (e.g., an office location), turning on the do not disturb state at a specific time (e.g., bedtime), or other patterns). Each pattern may include a set of criteria (e.g., a time criterion, a sound criterion, a location criterion, and / or other criteria) that must be met to transition the electronic device 101 from a first power state to a second power state.

[0052] In some examples, one or more criteria are based on one or more patterns of the user, as described above. Alternatively, or in addition, in some examples, one or more criteria are based on the user's preferences. For example, the user may configure one or more actions that electronic device 101 (or electronic devices (one or more) 303 and / or 305) perform (e.g., in a second power state). For example, the user may request the activation of a first application (e.g., running a background application and / or displaying the user interface of the first application) when they are in a particular location.

[0053] In Figure 3B, while operating in a second power state and displaying user interface elements 304 and 306, the electronic device 101 receives a selection input directed to user interface element 306. In some examples, the input may be gaze input using the eye 320, as shown in Figure 3B. Alternatively or in addition, in some examples, the input may be air pinch input using the hand 315, as shown in Figure 3B. Alternatively or in addition, in some examples, the input may be a combination of gaze input using the eye 320 and a predetermined movement of the hand 315. In response to receiving the input shown in Figure 3B, the electronic device 101 begins displaying user interface element 310, as shown in Figure 3C.

[0054] In some examples, depending on whether one or more criteria are met, the electronic device 101 remains in a first power state while running background applications. For example, instead of displaying user interface elements 304 and 306 in Figure 3B, the electronic device 101 automatically starts playing music depending on whether one or more criteria are met. In some examples, the electronic device 101 may reduce power consumption by refraining from activating the display 120 while performing an action based on historical data and / or user preferences.

[0055] Figure 3C shows an electronic device 101 displaying user interface elements 310, 304, and 306 on a display 120. In Figure 3C, the electronic device 101 updates user interface element 306 to include a pause button icon instead of the play button shown in Figure 3B, indicating that music is being played (for example, using one or more speakers of the electronic device 101 or one or more speakers of a device communicating with the electronic device 101 (e.g., one or more electronic devices 303 and / or 305)) and that further input directed to user interface element 306 will cause the electronic device 101 to pause playback of the content. In Figure 3C, the electronic device 101 also displays user interface element 310 containing a representation of the currently playing playlist. In some examples, electronic device 101 plays a curated playlist, resumes playback of previously paused music, or plays a previously determined playlist (e.g., music in a music library, music in a favorites playlist, or other playlists downloaded or saved to a music application / electronic device(s) 101, 303, or 305). In some examples, the curated playlist is based on historical data of the types of music played in the same or similar context, such as at a given time and / or location while in the bus 313. In some examples, the historical music data is based on metadata of previously played music. For example, electronic device 101 stores historical music data indicating that happy pop music is frequently played while electronic device 101 is in the bus 313. Based on this historical music data, electronic device 101 creates a playlist containing happy pop music that will be played, as shown in Figure 3C.

[0056] In Figure 3D, the user arrives at the desired location and leaves the bus. Representation 352 shows a top-down view of the physical environment 350, indicating that the user is in front of building 311 and no longer inside bus 313. In some examples, the electronic device 101 and / or computer system determine, while in a first power state, that user 317 has disembarked from the bus and is at or within a threshold distance (e.g., 1m, 5m, 10m, 100m, or 500m) from building 311. For example, before displaying user interface element 302 and after displaying user interface elements 310, 304, and 306 shown in Figure 3C, the electronic device 101 transitions back to the first power state. In some examples, the electronic device 101 transitions from a second power state to a first power state when a threshold amount of time (e.g., 5 seconds, 10 seconds, 30 seconds, 1 minute, or 5 minutes) has elapsed without detecting any input directed to the electronic device 101 (e.g., the electronic device 101 times out). In some examples, the electronic device 101 transitions from a second power state to a first power state when one or more criteria are no longer met (e.g., the electronic device(s) 101, 303, and / or 305 are no longer in a particular location, a certain amount of time has elapsed, one or more sounds are no longer detected, and / or the user 317 rejects a user interface element and / or turns off the display 120).

[0057] In some examples, as described above, while in the first power state, the electronic device 101 uses data from microphones, location sensors, and / or orientation sensors (e.g., of electronic device 101, or of electronic devices (single or multiple) 303 and / or 305) to determine the environmental context of user 317. In some examples, the electronic device 101 determines that the environmental sound has changed from sound 310b (e.g., sound from inside bus 313) to sound 310c (e.g., sound from outside building 311). In some examples, the electronic device 101 also determines that sound 310c matches historical sound data associated with the current location of electronic device 101 (e.g., outside building 311). In some examples, a change in location, a change in movement, and / or a change in sound of one or more electronic devices (e.g., electronic devices (single or multiple) 101, 303, and / or 305) satisfies one or more criteria. In addition, or instead, in some examples, the electronic device 101 uses one or more head tracking sensors, hand tracking sensors, and / or eye tracking sensors to determine the user 317's environmental context. For example, the electronic device 101 uses the head tracking sensor to determine the position of the user's head (e.g., looking down to leave a bus). In some examples, the electronic device 101 uses one or more image sensors to determine the user's environmental context. For example, the electronic device 101 uses the image sensor to capture an image of the physical environment 350, such as a portion of the physical environment 300 shown in Figure 3D. In some examples, while in a first power state, the electronic device 101 operates the image sensor, head tracking sensor, hand tracking sensor, and eye tracking sensor at a lower frequency than while in a second power state. Depending on whether one or more criteria are determined to be met, the electronic device 101 transitions from the first power state to the second power state and displays the user interface element 302 shown in Figure 3D.

[0058] In some examples, as described above, one or more criteria are based on historical data. For example, building 311 is a frequently visited location (e.g., user 317's workplace) based on historical location data. For example, electronic devices 101, 303, and / or 305 include historical location data based on one or more location sensors (e.g., GPS) indicating that the user frequently visits building 311. In some examples, the historical location data includes visit time data (e.g., user 317 visits building 311 from Monday through Friday, starting around 8 a.m. and leaving around 5 p.m.). In some examples, electronic devices 101, 303, and / or 305 detect that user 317 checks an email application, a texting application, and / or a reminder application using one or more devices after a given time, a given location, and / or after a specific action (e.g., at building 311 at 8 a.m., after user 317 is no longer in bus 313). Depending on the history data, the electronic device 101 displays user interface elements 302 containing instructions for notifications from one or more frequently visited applications while the user is in front of the building 311. In some examples, the electronic device 101 displays user interface elements 302 based on the user's preferences. For example, as described above, user 317 may decide which instructions to display on the user interface elements 302.

[0059] In some examples, if one or more criteria are not met, the electronic device 101 will not transition from the first power state to the second power state. For example, if the electronic device(s) 101, 303, and / or 305 do not detect that user 317 is inside bus 313 in Figure 3B (for example, using one or more sensors as described above), the electronic device 101 will not transition to the second power state and will not display user interface elements 304 and 306. Similarly, if the electronic device 101 detects that the user has arrived at a location other than the location containing building 311 shown in Figure 3D at a different time, the electronic device 101 will not transition to the second power state and will not display user interface element 302. For example, user 317 boards a bus in the morning and arrives at a second location different from the location of building 311 at 8:00 a.m. In some examples, if the second location is associated with one or more history patterns and one or more criteria are met, the electronic device 101 transitions to a second power state and displays content associated with the second location.

[0060] In some cases, one or more criteria change as user preferences and / or historical data change. For example, if the user frequently rejects / closes user interface elements 304 and 306 while on bus 313, the electronic device 101 may transition to a second power state while on bus 313 and cease displaying elements of the music application.

[0061] Figures 4A to 4C illustrate different examples of how the electronic device 101 transitions from a first power state to a second power state based on information collected from one or more input devices. Figures 4A to 4C are used to illustrate the processes described below, including the process 600 shown in Figure 6. In some examples, the user may be in a situation such as a problem with the car, in which case it is advantageous to display additional information on the display 120 of the electronic device 101. In some examples, using one or more input sensors, the electronic device 101 may determine the environmental context and consequently display relevant information.

[0062] Figure 4A shows an electronic device 101 presenting a portion 400 of the physical environment 450 from the user's perspective via a pass-through or optical see-through display 120. Figure 4A shows the user of the electronic device 101 while driving a car 420. The electronic device 101 communicates with one or more additional devices, such as one or more electronic devices 303 and / or 305, shown in Figure 4A and described in more detail above. In Figure 4A, the sensors of electronic devices 101, 303, and / or 305 (e.g., accelerometers, and / or other location sensors, compass sensors, or motion sensors) detect that each device is moving at 70 km / h with an acceleration of 0 m / s² (e.g., since the user is driving at 70 km / h). In Figure 4A, sensors of electronic devices (one or more) 101, 303, and / or 305 detect that a user is inside the car (e.g., sound 402a).

[0063] In Figure 4B, the sensors of one or more electronic devices 101, 303, and / or 305 detect changes in speed such that each device has a speed of 2 km / h and a deceleration of 5 m / s² (for example, car 420 is slowing down so that it is no longer moving). In some examples, the detected acceleration profile (e.g., change in speed) may correspond to a known acceleration profile, such as the acceleration profile of a tire puncture event. In addition, in some examples, the sensors of one or more electronic devices 101, 303, and / or 305 detect a change in sound from sound 402a to sound 402b. In some examples, sound 402b includes a loud sound that matches a tire puncture. In some examples, electronic device 101 detects a tire puncture event using the characteristic sound from sound 402b that corresponds to a tire puncture. In addition or alternatively, the user searches for "how to change a tire" on the internet browser user interface 410 on electronic device 303, for example. In some examples, the user may, in addition to or instead, perform a search on the internet browser user interface of electronic device 101 or electronic device 305. In some examples, using head tracking sensors, eye tracking sensors, and / or hand tracking sensors, electronic device 101 may detect a sudden change in gaze, head movement, or hand movement that may match a tire puncture.

[0064] In Figure 4C, the electronic device 101 transitions from a first power state to a second power state after determining that one or more criteria are met. In some examples, one or more criteria are met when the electronic device 101 determines that the confidence level for transitioning from the first power state to the second power state exceeds a confidence threshold (e.g., 51%, 60%, 75%, 90%, 95%, or 99%). In some examples, the confidence level is based on the likelihood that the user is in a context in which the electronic device 101 can display contextual information. For example, in Figure 4C, the electronic device 101 and / or the computer system detect, via the electronic device 303, that the user searched for "how to change a tire" on a web browser application. In addition, the electronic device 101 and / or the computer system detect, via the sensor in the electronic device 305, that the user is in a crouched position. For example, the electronic device 305 and / or the computer system uses one or more location sensors and orientation sensors to determine that the user is crouching. In addition, the electronic device 101 and / or the computer system detects, via microphones (e.g., on electronic device 101, electronic device 303, and / or electronic device 305), that the ambient sound has changed to sound 402c, which corresponds to an outdoor sound. In addition, in some examples, the electronic device 101 and / or the computer system uses image sensors, head tracking sensors, eye tracking sensors, and / or hand tracking sensors to increase the confidence level for transitioning to a second power state. For example, the image sensor may indicate that the user is outside the car and / or looking at a flat tire. The eye tracking sensor and / or head tracking sensor may indicate that the user is staring at the flat tire. In some examples, a combination of information from sensor data indicates that the electronic device 101 is in a context where the confidence level exceeds a threshold confidence level, and therefore the electronic device 101 should transition from a first power state to a second power state.In some examples, contexts in which the confidence level exceeds the threshold confidence level include contexts in which the user may require additional information (e.g., a video on how to change a tire, as shown in Figure 4C), contexts based on historical information (e.g., the contexts described in Figures 3A to 3D), and / or contexts in which the electronic device 101 receives instructions from a second electronic device (e.g., a notification from an application) that should be displayed on the electronic device 101.

[0065] In Figure 4C, the electronic device 101 displays a user interface element 411 via the display 120, which includes a video about how to change a tire, based on information from a combination of sensor data from electronic device 101, electronic device 303, and electronic device 305. In some examples, the electronic device 101 displays a user interface element 411 that improves user device interaction, so that the user can view information about changing a tire without having to use electronic device 303 and / or without having to provide additional input to electronic device 101 to search for information about changing a tire.

[0066] In some examples, the electronic device 101 does not display the user interface element 411 in Figure 4B because the confidence level has not yet exceeded the confidence threshold. For example, in Figure 4B, the electronic device 101 and / or the computer system detect changes in speed and data from the electronic device 303 from the web browser application. However, the electronic device 101 does not detect changes in sound or actions such as crouching down by the user. For example, the user may be changing a tire with the help of another person, and therefore the electronic device 101 does not need to display the user interface element 411.

[0067] Figures 4A to 4C illustrate an example of a tire puncture, but similar actions may occur if one or more electronic devices 101, 303, and / or 305 detect a collision. For example, using one or more sensors, electronic device 101 and / or a computer system may detect a collision (e.g., a change in acceleration, an increase in force on electronic device 101, sound, a sudden change in head position). In response to detecting a collision, electronic device 101 may transition from a first power state to a second power state and display relevant information about the collision (e.g., displaying insurance information and / or prompts including post-collision procedures (e.g., collecting information from other parties, taking pictures, and / or calling the police)).

[0068] Figures 5A and 5B illustrate an example in which the electronic device 101 transitions from a first power state to a second power state based on non-contextual information collected from one or more input devices. Figures 5A and 5B are used to illustrate the processes described below, including the process 600 shown in Figure 6. In some examples, while the electronic device 101 is in the second power state, it may opportunistically collect additional information about the user's environment to signal future actions.

[0069] Figure 5A shows an electronic device 101 presenting a portion 500 of the physical environment 550 from the user's perspective via a pass-through or optical see-through display 120. Figure 5A shows the user of the electronic device 101 viewing a storm. The electronic device 101 communicates with one or more additional devices, such as one or more electronic devices 303 and / or 305 shown in Figure 5A and described in more detail above.

[0070] In Figure 5A, the electronic device 101 and / or the computer system receive instructions for a text message from a messaging application. Upon receiving the instructions, the electronic device 101 transitions from a first power state to a second power state and displays the visual instructions 503 of the text message via the display 120. In some examples, as shown in Figures 3A-3D and 4A-4C, the electronic device 101 transitions from a first power state to a second power state based on environmental context and / or historical data. In some examples, the electronic device 101 also transitions from a first power state to a second power state upon receiving instructions to be displayed on the electronic device 101 (e.g., normal operation of the electronic device 101). While the electronic device 101 is in the second power state, it opportunistically uses a second subset of one or more input devices operating in the second power state to collect additional information. For example, while operating in a second power state, the electronic device 101 activates one or more image sensors (as described in more detail, for example, in Figure 2). In Figure 5A, one or more image sensors capture characteristics of the physical environment 550, such as the fact that it is raining. In some examples, the electronic device 101 and / or computer system use the additional processing power available while in the second power state to access the internet and collect weather data.

[0071] Furthermore, as shown in Figure 5A, the electronic device 303 includes a first wallpaper 502a. In some examples, the wallpaper 502a is computer-generated and / or selected by the user.

[0072] Figure 5B shows electronic device 101 updating the wallpaper from wallpaper 502a to wallpaper 502b in response to information captured while electronic device 303 was in the second power state of Figure 5A. As described in Figure 5A, electronic device 101 captures context information using an additional input device while electronic device 101 is in the second power state. Electronic device 101 transmits the context information to one or more devices (e.g., electronic device(s) 303 and / or 305) that communicate with electronic device 101. In some examples, electronic device 303 updates the wallpaper in response to receiving the context information. In some examples, electronic device 303 updates the wallpaper based on user preferences or as a feature of computer-generated wallpapers. For example, the user may decide which wallpapers to use for each weather, or electronic device 303 may decide which wallpaper to use for each weather.

[0073] In Figure 5B, the electronic device 101 ceases displaying the visual instruction 503 and transitions back to the first power state. As described above, the electronic device 101 may transition back from the second power state to the first power state after a time threshold has elapsed without detecting user interaction with the displayed element, or in response to user input.

[0074] Figures 5A and 5B illustrate an example in which opportunistic sensing triggers an action on electronic device 303. In some examples, opportunistic sensing of data while in a second power state may cause one or more actions to be performed on electronic device 101, electronic device 303, and / or electronic device 305. While electronic device 101 is displaying non-contextual information on electronic device 101 while in a second power state, electronic device 101 may capture additional data that satisfies one or more criteria, as described above. In response to satisfying one or more criteria, electronic device 101 may remain in the second power state and perform additional actions corresponding to satisfying one or more criteria. In some examples, electronic devices 303 and / or 305 may perform opportunistic sensing that transitions from a first power state to a second power state and consequently capture additional data that satisfies one or more criteria for transitioning electronic device 101 from the first power state to the second power state. For example, the electronic device 303 may receive and display a notification from the first application. While displaying the notification from the first application, the electronic device 303 may detect additional data that causes the electronic device 101 to transition from a first power state to a second power state.

[0075] Figure 6 shows a flowchart illustrating an exemplary process for transitioning an electronic device from a first power state to a second power state, according to some examples of the present disclosure. In some examples, process 600 begins in a first electronic device that communicates with a second electronic device, which has one or more first displays and one or more input devices. In some examples, the second electronic device includes one or more second displays and one or more second input devices. In some examples, the first electronic device is optionally a head-mounted display similar to or corresponding to electronic device 201 or 260 in Figure 2, and electronic device 101 in Figures 3A-3D, 4A-4C, and 5A-5B. As shown in Figure 6, in some examples, in 602, while the first electronic device is in a first power state, the first electronic device detects a first set of information via a first subset of the first input devices among one or more first input devices. In some examples, a first subset of input devices includes a location sensor, an orientation sensor, a microphone, and other low-power sensors, as shown in Figure 3A. In some examples, a first set of information includes information that informs the electronic device 101 of the environmental context.

[0076] In some examples, in 604, the first electronic device (e.g., electronic device 101) receives from the second electronic device a second set of information different from the first set of information, which is detected using one or more second input devices of the second electronic device. In some examples, electronic device 101 receives sensor data from input devices of one or more electronic devices communicating with electronic device 101, such as electronic device 303 and / or electronic device 305, as described in more detail in Figures 3A and 4B. In some examples, electronic device 101 may receive motion data from a smartwatch, location data from a phone, or other data to inform the environmental context, as described above.

[0077] In some examples, in 606, a first electronic device (e.g., electronic device 101) transitions the first electronic device from a first power state to a second power state, where the second power state is associated with a higher power state than the first power state, according to the determination that one or more first criteria are met, which include criteria that are met based on a first set of information and a second set of information. In some examples, if the environmental context based on the first set of information and the second set of information meets one or more criteria based on historical data, current sensor data, and / or user preferences, the electronic device 101 transitions from a first power state to a second power state. As shown in Figures 3A to 3D, if the previous historical data indicates a user pattern and the environmental context corresponds to one or more of the user patterns, the electronic device 101 transitions to a second power state and performs one or more actions. In some examples, as shown in Figures 4A to 4C, if the confidence level based on the environmental context (e.g., one or more sensor data) exceeds the confidence threshold, the electronic device 101 transitions to a second power state and displays the relevant information. In some examples, in 608, in accordance with the determination that one or more first criteria are not met, the first electronic device (e.g., electronic device 101) refrains from transitioning the first electronic device from a first power state to a second power state.

[0078] Process 600 is an example, and it should be understood that more, fewer, or different operations may be performed in the same or different order. In addition, the operations of Process 600 described above may optionally be performed by running one or more functional modules of an information processing device, such as a general-purpose processor or application-specific chip (as described with respect to Figure 2, for example), and / or by other components of Figure 2. In addition, in some examples, one or more operations described in Process 600 may optionally be performed by any of the electronic devices in the computer system (e.g., one or more electronic devices 101, 303, and / or 305).

[0079] Accordingly, according to the foregoing, some examples of the present disclosure include a first electronic device having one or more first displays and one or more first input devices that communicates with a second electronic device having one or more second input devices, wherein while the first electronic device is in a first power state, a first set of information is detected via a first subset of the first input devices of the one or more first input devices, and a second set of information different from the first set of information is detected using one or more second input devices of the second electronic device. The following are included: receiving from an electronic device; transitioning a first electronic device from a first power state to a second power state in accordance with a determination that one or more first criteria are met, including criteria that are met based on a first set of information and a second set of information, wherein the second power state is associated with a higher power state than the first power state; and ceasing to transition the first electronic device from the first power state to the second power state in accordance with a determination that one or more first criteria are not met. In addition to or instead of one or more of the examples disclosed above, in some examples, the first power state includes operating the first electronic device without operating one or more first displays, and the second power state includes operating the first electronic device, including operating one or more first displays. In addition to or instead of one or more of the examples disclosed above, in some examples, operating one or more first displays while in a second power state includes displaying a first user interface on one or more displays of a first electronic device, wherein the first user interface is based on a first set of information and a second set of information.In addition to or instead of one or more of the examples disclosed above, in some examples, the transition from a first power state to a second power state includes activating a first application and a second subset of one or more first input devices, which is different from a first subset of one or more first input devices. In addition to or instead of one or more of the examples disclosed above, in some examples, while in the first and second power states, the first electronic device deactivates one or more first displays. In addition to or instead of one or more of the examples disclosed above, in some examples, transitioning a first electronic device from a first power state to a second power state includes activating a second subset of one or more first input devices that were inactive in the first power state, and, after activating the second subset of one or more first input devices, detecting a third set of information through the second subset of one or more first input devices, the third set of information being used to inform of one or more functions of the first and second electronic devices. In addition to or instead of one or more of the examples disclosed above, in some examples, one or more first criteria include a second criterion that is satisfied based on one or more history patterns of the user of the first electronic device.In addition to or instead of one or more of the examples disclosed above, some examples further include: transitioning a first electronic device from a first power state to a second power state by displaying, via one or more first displays, one or more first user interface elements corresponding to the respective contextual information from the first and second information while the device is in the second power state, in accordance with the determination that the first and second information correspond to a first context; and by displaying, via one or more first displays, one or more second user interface elements corresponding to the respective contextual information from the first and second information while the device is in the second power state, in accordance with the determination that the first and second information correspond to a second context. In addition to or instead of one or more of the examples disclosed above, in some examples, transitioning a first electronic device from a first power state to a second power state further includes, in accordance with the determination that the first and second information correspond to a first context, displaying one or more first user interface elements corresponding to the respective non-contextual information from the first and second information via one or more first displays while the device is in the second power state, and in accordance with the determination that the first and second information correspond to a second context, displaying one or more second user interface elements corresponding to the respective contextual information from the first and second information via one or more first displays while the device is in the second power state. In addition to or instead of one or more of the examples disclosed above, in some examples, a first subset of the first input devices among one or more first input devices includes an image sensor, a hand tracking sensor, and / or a head tracking sensor.

[0080] Some examples of the present disclosure relate to an electronic device comprising one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the programs including instructions for performing any of the methods described above.

[0081] Some examples of the present disclosure relate to a non-temporary computer-readable storage medium storing one or more programs, wherein the one or more programs, when executed by one or more processors of an electronic device, include instructions causing the electronic device to perform one of the methods described above.

[0082] Some examples of the present disclosure relate to an electronic device comprising one or more processors, memory, and means for performing any of the methods described above.

[0083] Some examples of the present disclosure relate to information processing devices for use in electronic devices, which include means for performing any of the methods described above.

[0084] This disclosure intends that, in some cases, the data used may include personal data that uniquely identifies a particular person, or personal data that can be used to contact a particular person or to locate them. Such personal data may include demographic data, content consumption activity, location-based data, telephone numbers, email addresses, Twitter® IDs, home addresses, data or records relating to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), birth dates, or any other identifying or personal information. Specifically, as described herein, one aspect of this disclosure is tracking a user's location and / or sound using a microphone.

[0085] This disclosure acknowledges that such use of personal data in this technology may be in the user's best interest. For example, personal data may be used to display suggestion text that changes based on changes in the user's biometric data. For instance, the suggestion text may be updated based on changes in the user's age, height, weight, and / or health history.

[0086] This disclosure aims to ensure that entities involved in the collection, analysis, disclosure, transfer, storage, or other use of such personal data adhere to robust privacy policies and / or privacy practices. Specifically, such entities shall implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for the strict confidentiality of personal data. Such policies should be readily accessible to users and should be updated as data collection and / or use changes. Personal data from users shall be collected for the lawful and legitimate use of the entity and shall not be shared or sold outside the scope of such lawful use. Furthermore, such collection / sharing should be carried out only after obtaining informed consent from the user. In addition, such entities shall take all necessary measures to protect and secure access to such personal data and ensure that others with access to such personal data adhere to those privacy policies and procedures. Furthermore, such entities may undergo third-party evaluations to demonstrate their compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be tailored to the specific types of personal data collected and / or accessed, and should comply with applicable laws and standards, including jurisdiction-specific considerations. For example, in the United States, the collection or access to certain health data may be subject to federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA). Health data in other countries, on the other hand, may be subject to other regulations and policies and should be addressed accordingly. Therefore, each country should maintain different privacy practices with respect to different types of personal data.

[0087] Notwithstanding the foregoing, this disclosure also intends to provide examples of how users can selectively prevent the use of or access to personal data. Specifically, this disclosure intends to provide hardware and / or software elements to prevent or block access to such personal data. For example, the technology could be configured to allow a user to choose to “opt in” or “opt out” of participating in the collection of personal data during or at any time thereafter when registering for the service. In another example, a user could choose not to enable the recording of personal data in a particular application (e.g., Application 1 and / or Application 2). In addition to providing “opt-in” and “opt-out” options, this disclosure intends to provide notices regarding access to or use of personal data. For example, a user should be notified when the collection of personal data begins, and then notified again immediately before the personal data is accessed by a device(s).

[0088] Furthermore, the intent of this disclosure is that personal data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data when it is no longer needed. In addition, where applicable, data anonymization can be used in certain health-related applications to protect user privacy. Anonymization can be facilitated, where appropriate, by removing certain identifiers (e.g., birth dates), controlling the amount or specificity of data stored (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data across all users), and / or by other means.

[0089] The above description of the purpose of this explanation is based on specific examples. However, the above illustrative description is not intended to be exhaustive or to limit this disclosure to the exact form disclosed. Many modifications and variations are possible in light of the above teachings. The examples have been selected and described to best illustrate the principles of this disclosure and their practical application, thereby enabling those skilled in the art to best use this disclosure and the various described examples with various modifications suitable for the specific use intended.

Claims

1. While the first electronic device is in a first power state, a first set of information is obtained that is detected using a first subset of one or more first input devices of the first electronic device. Obtaining a second set of information different from the first set of information, which is detected using a first subset of the second input devices of the second electronic device, wherein the second electronic device communicates with the first electronic device. Transitioning the first electronic device from the first power state to the second power state according to a determination that one or more first criteria are met, which include criteria that are met based on the first set of information and the second set of information, wherein the second power state is associated with a power state higher than the first power state, This includes, in accordance with a determination that one or more of the above first criteria are not met, ceasing to transition the first electronic device from the first power state to the second power state, method.

2. The method according to claim 1, wherein the first power state includes operating the first electronic device without operating one or more first displays of the first electronic device, and the second power state includes operating the first electronic device including operating one or more first displays.

3. The method according to claim 1, wherein operating the one or more first displays while in the second power state is to display a first user interface on the one or more displays of the first electronic device, the first user interface being based on a first set of information and a second set of information.

4. The method according to claim 1, wherein transitioning from the first power state to the second power state includes activating a first application and a second subset of the one or more first input devices, which is different from the first subset of the one or more first input devices.

5. The method according to claim 1, wherein the first electronic device refrains from activating one or more first displays of the first electronic device while in the first power state and the second power state.

6. Transitioning the first electronic device from the first power state to the second power state is Activating a second subset of the one or more first input devices that were inactive in the first power state, The method according to claim 1, further comprising activating the second subset of the one or more first input devices, and then detecting a third set of information through the second subset of the one or more first input devices, wherein the third set of information is used to indicate one or more functions of the first electronic device and the second electronic device.

7. The method according to claim 1, wherein the one or more first criteria include a second criterion that is satisfied based on one or more history patterns of the user of the first electronic device.

8. Transitioning the first electronic device from the first power state to the second power state is In accordance with the determination that the first set of information and the second set of information correspond to a first context, while in the second power state, one or more first user interface elements corresponding to the respective context information from the first set of information and the second set of information are displayed via one or more first displays of the first electronic device, The method according to claim 1, further comprising, in accordance with the determination that the first set of information and the second set of information correspond to a second context, displaying one or more second user interface elements corresponding to the respective context information from the first set of information and the second set of information via the one or more first displays while in the second power state.

9. Transitioning the first electronic device from the first power state to the second power state is In accordance with the determination that the first set of information and the second set of information correspond to a first context, while in the second power state, one or more first user interface elements corresponding to the respective non-contextual information from the first set of information and the second set of information are displayed via one or more first displays, The method according to claim 1, further comprising, in accordance with the determination that the first set of information and the second set of information correspond to a second context, displaying one or more second user interface elements corresponding to the respective context information from the first set of information and the second set of information via the one or more first displays while in the second power state.

10. The method according to claim 1, wherein the first subset of the first input devices among the one or more first input devices includes an image sensor, a hand tracking sensor, and / or a head tracking sensor.

11. One or more processors, Memory and The system comprises one or more programs stored in the memory and configured to be executed by the one or more processors, each program including instructions for performing the method according to any one of claims 1 to 10. Electronic devices.

12. A non-temporary computer-readable storage medium storing one or more programs, wherein when the one or more programs are executed by one or more processors of an electronic device, the electronic device is provided with instructions to cause the electronic device to execute the method according to any one of claims 1 to 10.

13. A program for causing a computer to execute a process, wherein, when the process is executed by one or more processors of an electronic device, the program includes instructions for causing the electronic device to execute the method according to any one of claims 1 to 10.

14. One or more input devices, One or more displays, One or more processors, Memory and The system comprises one or more programs stored in the memory and configured to be executed by the one or more processors, each program including instructions for performing the method according to any one of claims 1 to 10. Electronic devices.

Images