Transitioning between power states based on environmental context

Abstract

Some examples of the present disclosure relate to systems and methods for transitioning between power states based on environmental context. In some examples, the first electronic device includes a first power state and a second power state, where the second power state is a higher power state than the first power state. In some examples, when in a first power state, the first electronic device detects information that meets one or more criteria, and thus, the first electronic device transitions from the first power state to a second power state. In some examples, the first electronic device activates the one or more displays while in the second power state. In some examples, activating the one or more displays while in the second power state allows the first electronic device to reduce power usage of the first electronic device by displaying information only at related times.

Description

Cross-references to related applications

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 699,637, filed September 26, 2024, and U.S. Patent Application No. 19 / 256,671, filed July 1, 2025, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0002] This disclosure relates in its entirety to systems and methods for transitioning between power states based on environmental context. Specifically, this disclosure relates to activating one or more input devices or one or more displays based on environmental context. Background Technology

[0003] Some computer graphics environments provide two-dimensional and / or three-dimensional environments in which at least some of the objects displayed for user viewing are virtual and computer-generated. In some examples, objects include one or more user interface elements that are displayed in response to one or more environmental context information. Summary of the Invention

[0004] Some examples of this disclosure relate to systems and methods for transitioning between power states based on environmental context. Specifically, this disclosure relates to activating one or more input devices or one or more displays based on environmental context. In some examples, a first electronic device includes a first power state and a second power state, wherein the second power state is a higher power state than the first power state. In some examples, when in the first power state, the first electronic device detects information that one or more criteria are met, and therefore, the first electronic device transitions from the first power state to the second power state. In some examples, the first electronic device activates one or more displays while in the second power state. In some examples, activating one or more displays after detecting information that one or more criteria are met to transition the first electronic device to the second power state allows the first electronic device to turn on one or more displays and / or display relevant content only at appropriate times, thereby reducing the power usage of the first electronic device.

[0005] A full description of these examples is provided in the accompanying drawings and detailed embodiments, and it should be understood that the content of this invention does not limit the scope of this disclosure in any way. Attached Figure Description

[0006] To better understand the various examples described herein, reference should be made to the following detailed embodiments and the accompanying drawings. Throughout the drawings, similar reference numerals generally refer to corresponding parts.

[0007] Figure 1Examples of electronic devices that present extended real-world environments according to some examples of this disclosure are illustrated.

[0008] Figures 2A to 2B A block diagram illustrating an example architecture for a device according to some examples of this disclosure is shown.

[0009] Figures 3A to 3D Examples of first electronic devices according to this disclosure are illustrated, which switch between different power states based on information collected from one or more input devices.

[0010] Figures 4A to 4C Examples of various embodiments of an electronic device according to this disclosure are illustrated, in which the electronic device transitions from a first power state to a second power state based on information collected from one or more input devices.

[0011] Figures 5A to 5B Examples of electronic devices according to this disclosure are illustrated, wherein an electronic device transitions from a first power state to a second power state based on non-contextual information collected from one or more input devices.

[0012] Figure 6 A flowchart illustrating an example process for transitioning an electronic device from a first power state to a second power state, according to some examples of this disclosure, is shown. Detailed Implementation

[0013] Some examples of this disclosure relate to systems and methods for transitioning between power states based on environmental context. Specifically, this disclosure relates to activating one or more input devices or one or more displays based on environmental context. In some examples, a first electronic device includes a first power state and a second power state, wherein the second power state is a higher power state than the first power state. In some examples, when in the first power state, the first electronic device detects information that one or more criteria are met, and therefore, the first electronic device transitions from the first power state to the second power state. In some examples, the first electronic device activates one or more displays while in the second power state. In some examples, activating one or more displays after detecting information that one or more criteria are met to transition the first electronic device to the second power state allows the first electronic device to turn on one or more displays and / or display relevant content only at appropriate times, thereby reducing the power usage of the first electronic device.

[0014] Figure 1 An electronic device 101 is illustrated according to some examples of this disclosure, which presents an extended reality (XR) environment (e.g., a computer-generated environment that optionally includes representations of physical and / or virtual objects). In some examples, such as Figure 1As shown, electronic device 101 is a head-mounted display or other head-mountable device configured to be worn on the head of a user of electronic device 101. See below for reference. Figure 2A An example of an architecture block diagram to describe electronic device 101. For example... Figure 1 As shown, electronic device 101 and table 106 are located in a physical environment. The physical environment may include physical features such as physical surfaces (e.g., floor, wall) or physical objects (e.g., table, lamp, etc.). In some examples, electronic device 101 may be configured to detect and / or capture images of the physical environment including table 106 (exemplified in the field of view of electronic device 101).

[0015] In some examples, such as Figure 1 As shown, electronic device 101 includes one or more internal image sensors 114a oriented toward the user's face (e.g., referred to below). Figures 2A to 2B (The described eye-tracking camera). In some examples, an internal image sensor 114a is used for eye tracking (e.g., detecting the user's gaze). The internal image sensor 114a is optionally arranged on the left and right portions of the display 120 to enable eye tracking of the user's left and right eyes. In some examples, the electronic device 101 also includes external image sensors 114b and 114c facing outwards from the user to detect and / or capture the physical environment of the electronic device 101 and / or movement of the user's hand or other body parts.

[0016] In some examples, display 120 has a field of view visible to the user (e.g., it may or may not correspond to the field of view of external image sensors 114b and 114c). Because display 120 is optionally part of a head-mounted device, the field of view of display 120 may be the same as or similar to the field of view of the user's eyes. In other examples, the field of view of display 120 may be smaller than the field of view of the user's eyes. In some examples, electronics 101 may be an optical pass-through device, through which display 120 is a transparent or translucent display through which parts of the physical environment can be directly viewed. In some examples, display 120 may be included within a transparent lens and may overlap with all or only a portion of the transparent lens. In other examples, electronics may be a video pass-through device, through which display 120 is an opaque display configured to display images of the physical environment captured by external image sensors 114b and 114c. Although a single display 120 is shown, it should be understood that display 120 may include a pair of stereoscopic displays. Figures 2A to 2BIn this context, the display 120 includes or corresponds to a transparent or translucent surface (e.g., a lens) that is not equipped with display capabilities (e.g., and therefore cannot generate and display virtual objects 104), and alternatively presents a direct view of the physical environment in the user's field of view (e.g., the user's eye's field of view).

[0017] In some examples, electronic device 101 is configured to display (e.g., in response to a trigger) virtual object 104 in a three-dimensional environment. Virtual object 104 is... Figure 1 The cube illustrated represents a cube that does not exist in the physical environment but is displayed in the three-dimensional environment as positioned on top of table 106 (e.g., a real-world table or a representation thereof). Optionally, in response to detecting a flat surface of table 106 in the physical environment 100, virtual object 104 is displayed on the surface of table 106 in a three-dimensional environment displayed via display 120 of electronic device 101.

[0018] It should be understood that virtual object 104 is a representative virtual object and may include and render one or more different virtual objects (e.g., virtual objects with various dimensions, such as two-dimensional or other three-dimensional virtual objects) in a three-dimensional environment. For example, a virtual object may represent an application or user interface displayed in a three-dimensional environment. In some examples, a virtual object may represent content corresponding to an application and / or displayed via a user interface in a three-dimensional environment. In some examples, virtual object 104 may optionally be configured to be interactive and responsive to user input (e.g., air gestures, such as air pinch gestures, air tap gestures, and / or air touch gestures), allowing the user to virtually touch, tap, move, rotate, or otherwise interact with virtual object 104.

[0019] As discussed in this article, by the user (e.g., in Figure 1 One or more air pinch gestures performed by the middle hand (103) are detected by one or more input devices of electronic device 101 and interpreted as one or more user inputs pointing to content displayed by electronic device 101. Additionally or alternatively, in some examples, one or more user inputs interpreted by electronic device 101 as pointing to content displayed by electronic device 101 (e.g., virtual object 104) are detected via one or more hardware input devices (e.g., controllers, touchpads, proximity sensors, buttons, sliders, knobs, etc.) rather than via one or more input devices configured to detect air gestures (such as one or more air pinch gestures) performed by the user. This description is intended to be exemplary and not limiting; the user may optionally use different air gestures and / or other forms of input to provide user input.

[0020] In some examples, electronic device 101 may be configured to communicate with a second electronic device, such as a companion device. For example, as Figure 1 As illustrated, electronic device 101 may optionally communicate with electronic device 160. In some examples, electronic device 160 corresponds to a mobile electronic device (such as a smartphone, tablet computer, smartwatch, laptop computer, or other electronic device). In some examples, electronic device 160 corresponds to a non-mobile electronic device that is typically stationary and not easily moved within a physical environment (e.g., a desktop computer, server, etc.). See below for reference. Figure 2B The following are additional examples of architectural diagrams used to describe electronic device 160. In some examples, electronic device 101 and electronic device 160 are associated with the same user. For example, in Figure 1 In this configuration, electronic device 101 may be positioned on the user's head (e.g., mounted on the user's head), and electronic device 160 may be positioned near electronic device 101, such as in the user's hand 103 (e.g., a hand 103 holding electronic device 160), in the user's pocket or bag, or on a surface near the user. Electronic devices 101 and 160 may optionally be associated with the same user account (e.g., a user account logged into on both electronic devices 101 and 160). See below for further details. Figures 2A to 2B Additional details regarding the communication between electronic device 101 and electronic device 160 are provided.

[0021] In some examples, the display of an object in a 3D environment is triggered or implemented by interaction with one or more user interface objects in the 3D environment. For example, initiating the display of an object in a 3D environment may include interaction with one or more virtual option / function representations displayed in the 3D environment. In some examples, when initiating the display of an object in a 3D environment, the electronic device may track the user's gaze as input for identifying one or more virtual option / function representations as a target for selection. For example, a gaze may be used to identify one or more virtual option / function representations as a target for selection using another selection input. In some examples, a virtual option / function representation may be selected using hand-tracking input detected via an input device communicating with the electronic device. In some examples, an object displayed in a 3D environment may move and / or reorient itself in the 3D environment based on movement input detected via an input device.

[0022] In the following description, an electronic device that communicates with one or more displays and one or more input devices is described. It should be understood that the electronic device may optionally communicate with one or more other physical user interface devices, such as touch-sensitive surfaces, physical keyboards, mice, joysticks, hand-tracking devices, eye-tracking devices, styluses, etc. Furthermore, as described above, it should be understood that the described electronic device, display, and touch-sensitive surface may optionally be distributed among two or more devices. Therefore, as used in this disclosure, information on or displayed by the electronic device may optionally be used to describe information output by the electronic device for display on a separate display device (touch-sensitive or non-touch-sensitive). Similarly, as used in this disclosure, input received on the electronic device (e.g., touch input received on a touch-sensitive surface of the electronic device, or touch input received on the surface of a stylus) may optionally be used to describe input received on a separate input device from which the electronic device receives input information.

[0023] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, TV channel browsing applications, and / or digital video player applications.

[0024] Figures 2A to 2B Block diagrams illustrating example architectures for electronic devices according to some examples of this disclosure are shown. In some examples, electronic device 201 and / or electronic device 260 include one or more electronic devices. For example, electronic device 201 may be a portable device, an auxiliary device for communicating with another device, a head-mounted display, a head-mounted speaker, etc. In some examples, electronic device 201 corresponds to the above reference. Figure 1 The described electronic device 101. In some examples, electronic device 260 corresponds to the above reference. Figure 1 The described electronic device 160.

[0025] like Figure 2A As illustrated, electronic device 201 may optionally include one or more sensors, such as one or more hand tracking sensors 202, one or more position sensors 204A, and one or more image sensors 206A (optionally corresponding to...). Figure 1The electronic device 201 may include an internal image sensor 114a and / or external image sensors 114b and 114c, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, one or more eye-tracking sensors 212, one or more microphones 213A or other audio sensors, one or more body tracking sensors (e.g., torso tracking sensors and / or head tracking sensors), etc. The electronic device 201 may optionally include one or more output devices, such as one or more display generation components 214A (which may optionally correspond to...) Figure 1 The electronic device 201 may include a display 120, one or more speakers 216A, one or more tactile output devices (not shown), etc. The electronic device 201 may optionally include one or more processors 218A, one or more memories 220A, and / or communication circuitry 222A. One or more communication buses 208A may optionally be used for communication between the components of the electronic device 201 mentioned above.

[0026] Additionally, electronic device 260 may optionally include components that are the same as or similar to those of electronic device 201. For example, such as Figure 2B As shown, the electronic device 260 optionally includes one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, one or more orientation sensors 210B, one or more microphones 213B, one or more display generating components 214B, one or more speakers 216B, one or more processors 218B, one or more memories 220B, and / or communication circuitry 222B. One or more communication buses 208B are optionally used for communication between the components of the electronic device 260 mentioned above.

[0027] Electronic devices 201 and 260 are optionally configured to communicate via a wired or wireless connection between the two electronic devices (e.g., via communication circuits 222A, 222B). For example, as Figure 2A As indicated, electronic device 260 can be used as an accessory device to electronic device 201. For example, in some examples, electronic device 260 processes sensor inputs from electronic devices 201 and 260 and / or uses the display generation component 214A of electronic device 201 to generate content for display.

[0028] Communication circuits 222A and 222B optionally include circuitry for communicating with electronic devices and networks such as the Internet, intranets, wired and / or wireless networks, cellular networks, and wireless local area networks (LANs). Communication circuits 222A and 222B optionally include circuitry for using near-field communication (NFC) and / or short-range communication (such as Bluetooth). ®The communication circuits 222A and 222B communicate with each other. In some examples, the communication circuits 222A and 222B include or support Wi-Fi (e.g., the 802.11 protocol), Ethernet, ultra-wideband (“UWB”), high-frequency systems (e.g., 900MHz, 2.4GHz, and 5.6GHz communication systems) or any other communication protocol or any combination thereof.

[0029] One or more processors 218A, 218B include one or more general-purpose processors, one or more graphics processors, and / or one or more digital signal processors. In some examples, one or more processors 218A, 218B include one or more microprocessors, one or more central processing units, one or more application-specific integrated circuits, one or more field-programmable gate arrays, one or more programmable logic devices, or combinations of such devices. In some examples, memories 220A and / or 220B are non-transitory computer-readable storage media (e.g., flash memory, random access memory, or other volatile or non-volatile memory or storage device) that store computer-readable instructions configured to be executed by one or more processors 218A, 218B to perform the techniques, processes, and / or methods described herein. In some examples, memories 220A and / or 220B may include more than one non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium can be any medium (e.g., excluding signals) that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, and device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on compact disc (CD), digital versatile optical disc (DVD), or Blu-ray technology, and persistent solid-state storage devices such as flash memory, solid-state drives, etc.

[0030] In some examples, one or more display generating components 214A, 214B include a single display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED), or other types of display). In some examples, one or more display generating components 214A, 214B include multiple displays. In some examples, one or more display generating components 214A, 214B may include a touch-enabled display (e.g., a touchscreen), a projector, a holographic projector, a retinal projector, a transparent or translucent display, etc. In some examples, the electronic device does not include one or more display generating components 214A or 214B. For example, some electronic devices do not include one or more display generating components 214A or 214B, but instead include a transparent or translucent lens or other surface not configured to display or present virtual content. However, it should be understood that in such cases, electronic device 201 and / or electronic device 260 may optionally be equipped with Figure 2A and Figure 2B One or more of the other components illustrated and described herein, such as one or more hand-tracking sensors 202, one or more eye-tracking sensors 212, one or more image sensors 206A, and / or one or more motion and / or orientation sensors 210A. Alternatively, in some examples, one or more display generation components 214A or 214B are provided separately from electronic devices 201 and / or 260. For example, one or more display generation components 214A, 214B communicate with electronic device 201 (and / or electronic device 260) but are not integrated with electronic device 201 and / or electronic device 260 (e.g., not integrated within the housing of electronic devices 201, 260). In some examples, electronic devices 201 and 260 respectively include one or more touch-sensitive surfaces 209A and 209B for receiving user input, such as tap input and swipe input or other gestures (e.g., hand-based gestures or finger-based gestures). In some examples, one or more display generating components 214A, 214B and one or more touch-sensitive surfaces 209A, 209B form one or more touch-sensitive displays (e.g., touchscreens integrated with each of the electronic devices 201 and 260 or touchscreens external to each of the electronic devices 201 and 260 that communicate with each of the electronic devices 201 and 260).

[0031] Electronic devices 201 and 260 optionally include one or more image sensors 206A and 206B, respectively. The image sensors 206A and 206B optionally include one or more visible light image sensors (such as charge-coupled device (CCD) sensors) and / or complementary metal-oxide-semiconductor (CMOS) sensors operable to obtain images of physical objects from a real-world environment. The image sensors 206A and 206B also optionally include one or more infrared (IR) sensors, such as passive or active IR sensors, for detecting infrared light from the real-world environment. For example, an active IR sensor includes an IR emitter for emitting infrared light into the real-world environment. The image sensors 206A and 206B also optionally include one or more cameras configured to capture movement of physical objects in the real-world environment. The image sensors 206A and 206B also optionally include one or more depth sensors configured to detect the distance between the physical object and the electronic devices 201 and 260. In some examples, information from one or more depth sensors allows a device to identify objects in a real-world environment and distinguish them from other objects in the real-world environment. In some examples, one or more depth sensors allow a device to determine the texture and / or shape of objects in a real-world environment. In some examples, one or more image sensors 206A or 206B are included in an electronic device different from electronic devices 201 and / or 260. For example, one or more image sensors 206A, 206B communicate with electronic devices 201, 260 but are not integrated with electronic devices 201, 260 (e.g., not integrated within the housing of electronic devices 201, 260). Specifically, in some examples, one or more cameras of one or more image sensors 206A, 206B are integrated with and / or coupled to electronic devices 201 and / or 260 and to one or more devices separate from these electronic devices (e.g., but communicating with electronic devices 201 and / or 260), such as one or more input devices and / or output devices (e.g., one or more speakers and / or one or more microphones, such as headphones or headsets) that include one or more image sensors 206A, 206B. In some examples, electronic device 201 or electronic device 260 corresponds to a headphone speaker (e.g., headphones or earbuds). In such cases, electronic device 201 or electronic device 260 is equipped with Figure 2A and Figure 2B A subset of other components illustrated and described herein. In some such examples, electronic device 201 or electronic device 260 is equipped with one or more image sensors 206A, 206B, one or more motion and / or orientation sensors 210A, 210B, and / or speakers 216A, 216B.

[0032] In some examples, electronic devices 201 and 260 combine a CCD sensor, an event camera, and a depth sensor to detect the physical environment surrounding them. In some examples, one or more image sensors 206A and 206B include a first image sensor and a second image sensor. The first and second image sensors work cooperatively and are optionally configured to capture different information about physical objects in the real-world environment. In some examples, the first image sensor is a visible light image sensor, and the second image sensor is a depth sensor. In some examples, electronic devices 201 and 260 use one or more image sensors 206A and 206B to detect the location and orientation of electronic devices 201 and 260 and / or one or more display generation components 214A and 214B in the real-world environment. For example, electronic devices 201 and 260 use one or more image sensors 206A and 206B to track the location and orientation of one or more display generation components 214A and 214B relative to one or more stationary objects in the real-world environment.

[0033] In some examples, electronic devices 201 and 260 include one or more microphones 213A and 213B or other audio sensors, respectively. Electronic devices 201 and 260 may optionally use one or more microphones 213A and 213B to detect sound from a user and / or the user's real-world environment. In some examples, the one or more microphones 213A and 213B include microphone arrays (e.g., multiple microphones) that optionally operate in cooperation to identify ambient noise or locate sound sources in the space of a real-world environment.

[0034] Electronic devices 201 and 260 each include one or more position sensors 204A and 204B, which are used to detect the positions of electronic device 201 and / or one or more display generating components 214A and the positions of electronic device 260 and / or one or more display generating components 214B, respectively. For example, the one or more position sensors 204A, 204B may include a Global Positioning System (GPS) receiver that receives data from one or more satellites and allows electronic devices 201, 260 to determine the absolute position of the electronic devices in the physical world.

[0035] Electronic devices 201 and 260 each include one or more orientation sensors 210A and 210B for detecting the orientation and / or movement of electronic device 201 and / or one or more display generation components 214A and the orientation and / or movement of electronic device 260 and / or one or more display generation components 214B, respectively. For example, electronic devices 201 and 260 use one or more orientation sensors 210A and 210B to track changes in the positioning and / or orientation of electronic devices 201 and 260 and / or one or more display generation components 214A and 214B, such as changes relative to physical objects in a real-world environment. The one or more orientation sensors 210A and 210B may optionally include one or more gyroscopes and / or one or more accelerometers.

[0036] In some examples, electronic device 201 includes one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212. It should be understood that, although referred to as hand tracking sensors or eye tracking sensors, electronic device 201 additionally or optionally includes one or more other body tracking sensors, such as one or more leg tracking sensors, one or more torso tracking sensors, and / or one or more head tracking sensors. One or more hand tracking sensors 202 are configured to track the localization and / or position of one or more portions of a user's hand, and / or the movement of one or more portions of the user's hand relative to a three-dimensional environment, relative to one or more display generation components 214A, and / or relative to another defined coordinate system. One or more eye tracking sensors 212 are configured to track the localization and movement of a user's gaze (e.g., the user's attention, more generally including the eyes, face, or head) relative to the real world or three-dimensional environment and / or relative to one or more display generation components 214A. In some examples, one or more hand tracking sensors 202 and / or one or more eye tracking sensors 212 are implemented together with one or more display generation components 214A. In some examples, one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212 are implemented separately from one or more display generation components 214A. In some examples, the electronic device 201 alternatively does not include one or more hand-tracking sensors 202 and / or one or more eye-tracking sensors 212. In some examples, one or more display generation components 214A may be utilized by the electronic device 260 to provide a three-dimensional environment, and the electronic device 260 may utilize input and other data collected via other one or more sensors of the electronic device 201 (e.g., one or more position sensors 204A, one or more image sensors 206A, one or more touch-sensitive surfaces 209A, one or more motion and / or orientation sensors 210A, and / or one or more microphones 213A or other audio sensors) as input and data processed by one or more processors 218B of the electronic device 260. Additionally or alternatively, the electronic device 260 may optionally not include... Figure 2B Other components shown include one or more position sensors 204B, one or more image sensors 206B, one or more touch-sensitive surfaces 209B, etc. In some such examples, one or more display generation components 214A may be utilized by electronic device 260 to provide a three-dimensional environment, and electronic device 260 may utilize input and other data collected via one or more motion and / or orientation sensors 210A (and / or one or more microphones 213A) of electronic device 201 as input.

[0037] In some examples, one or more hand-tracking sensors 202 (and / or other body-tracking sensors, such as leg-tracking sensors, torso-tracking sensors, and / or head-tracking sensors) may use one or more image sensors 206 (e.g., one or more IR cameras, 3D cameras, depth cameras, etc.) that capture 3D information from the real world, including one or more body parts (e.g., a human user's hand, leg, torso). In some examples, sufficient resolution is available to distinguish the hand to differentiate the fingers and their corresponding positions. In some examples, one or more image sensors 206A are positioned relative to the user to define a field of view and interaction space for one or more image sensors 206A, in which the finger / hand positions, orientations, and / or movements captured by the image sensors are used as input (e.g., to differentiate from the user's resting hand or other hands of other people in the real-world environment). Tracking the fingers / hands used for input (e.g., gestures, touches, taps, etc.) may be advantageous because it does not require the user to touch, hold, or wear any type of beacon, sensor, or other marker.

[0038] In some examples, one or more eye-tracking sensors 212 include at least one eye-tracking camera (e.g., an IR camera) and / or an illumination source (e.g., an IR light source, such as an LED) that emits light toward the user's eyes. The eye-tracking camera may be pointed at the user's eyes to receive reflected IR light from the light source directly or indirectly from the eyes. In some examples, both eyes are tracked separately by the respective eye-tracking camera and illumination source, and focus / gaze can be determined by tracking both eyes. In some examples, one eye (e.g., the dominant eye) is tracked by one or more respective eye-tracking cameras / illumination sources.

[0039] Electronic devices 201 and 260 are not limited to Figures 2A to 2B The components and configurations may include, but are not limited to, a few components, other components, or additional components in various configurations. In some examples, electronic device 201 and / or electronic device 260 may be implemented in various ways among multiple electronic devices (e.g., as a system). In some such examples, each electronic device (or more electronic devices) in the electronic device may include one or more of the same components discussed above, such as various sensors, one or more display generation components, one or more speakers, one or more processors, one or more memories, and / or communication circuitry. One or more persons using electronic device 201 and / or electronic device 260 may optionally be referred to herein as one or more users of the device.

[0040] Attention now turns to the context-based transition of a first electronic device (e.g., electronic device 201) from a first power state to a second power state according to examples of this disclosure. In some examples, the first electronic device transitions its state based on context information from the first electronic device or from a second electronic device (e.g., electronic device 260) communicating with the first electronic device. In some examples, the first electronic device is a wearable device with one or more output devices (e.g., a display and / or a speaker), and the second electronic device includes one or more sensors that can be used to provide context information to the first electronic device. The context information may optionally be based on one or more sensors of the first electronic device and one or more sensors of the second electronic device, as described in more detail herein. Obtaining information from one or more sensors of the second electronic device to determine the context information of the first electronic device can improve the user experience (e.g., revealing information to the user of the first electronic device at the right time with little or no additional user input) and / or reduce the power consumption, weight, and cost of the first electronic device.

[0041] In some examples, a first electronic device transitions between different power states to conserve power. For instance, the first electronic device receives data from sensors of a second electronic device to provide contextual information, enabling it to remain in a relatively low-power state (e.g., where one or more output devices are off or operating at low power, and where one or more sensors are off or operating at low power). The first electronic device may transition to a higher-power state when the contextual information meets one or more criteria. In some examples, a higher-power state includes turning on or entering a higher-power state of an output device or sensor, such as turning on one or more displays, processing at higher power (e.g., at a higher frequency than in a low-power state), having a higher refresh rate, and / or operating a background application. In some examples, transitioning to a higher-power state (e.g., a second power state, as described below) includes activating an additional processor. For example, when in a low-power state, the electronic device uses a low-power processor to monitor one or more sensors / input devices operating at low power and / or determine the environmental context of the electronic device. Transitioning to a higher-power state includes activating an additional processor (e.g., a higher-power processor) to monitor / operate an additional input device. As described or used herein, the power states of an electronic device are characterized by the on / off states or operating rates of one or more output devices (e.g., displays, speakers, haptic drivers, etc.) of the first electronic device, and the on / off states or operating rates of one or more sensors (e.g., one or more input devices). As used herein, an on-off state refers to whether a component (e.g., an input or output device) is supplied with power to operate (in the on state) or not to operate (in the off state), wherein power is supplied when the component receives a threshold voltage and / or current from a power source. In some examples, more components of the first electronic device are on when in a high-power state (e.g., the second state) rather than in a low-power state (e.g., the first state).

[0042] Figures 3A to 3D An example is illustrated in which a first electronic device transitions between different power states based on information collected from one or more input devices. Figures 3A to 3D Used to illustrate the process described below, including Figure 6 The process 600 is shown in some examples. And as in some other examples... Figures 3A to 3D As shown, a user may have a routine for consuming media (e.g., listening to music) during their commute. Using one or more input devices in a first power state, electronic device 101 may be able to determine the environmental context and initiate the process of the user consuming the corresponding media. In some examples, and as described below, electronic device 101 may transition to a second power state to display one or more user interface elements, thereby facilitating the user's media consumption.

[0043] Figure 3A This illustrates a portion 300 of an electronic device 101 presenting the physical environment 350 from the user's viewpoint via a display 120. Figure 3A In this context, electronic device 101 (e.g., corresponding to any of electronic devices 201 / 260) may optionally present portion 300 via a video pass-through or optical pass-through display. Figure 3A The user 317 of the electronic device 101 is shown facing the bus stop 312, with the bus 313 approaching the bus stop.

[0044] In some examples, the user's viewpoint affects the content (e.g., physical and / or virtual objects) visible in the viewport (e.g., a view of the same user-visible physical environment 350 via one or more displays 120, a monitor, or a pair of display modules providing stereoscopic content to different eyes of the user). In some examples, the (virtual) viewport has a defined viewpoint via... Figures 3A to 3DThe viewport 120 defines the range of the physical environment 350 visible to the user. In some examples, the area defined by the viewport boundary is smaller than the user's visual range in one or more dimensions (e.g., based on the user's visual range, the size of one or more displays, optical properties or other physical characteristics, and / or the position and / or orientation of one or more displays relative to the user's eyes). In some examples, the area defined by the viewport boundary is larger than the user's visual range in one or more dimensions (e.g., based on the user's visual range, the size of one or more displays, optical properties or other physical characteristics, and / or the position and / or orientation of one or more displays relative to the user's eyes). The viewport and viewport boundary typically move with the movement of one or more displays (e.g., with the user's head movement in a head-mounted device, or with the user's hand movement in a handheld device such as a tablet or smartphone). The user's viewpoint affects what is visible in the viewport; the viewpoint generally specifies the position and orientation relative to the physical environment 350, and as the viewpoint shifts, the view of the physical environment 350 also shifts within the viewport. For head-mounted devices, the viewpoint is typically based on the position and orientation of the user's head, face, and / or eyes to provide a perceptually accurate view of a three-dimensional environment that provides an immersive experience when the user is using the head-mounted device. For handheld or fixed devices, the viewpoint shifts with the movement of the handheld or fixed device and / or with changes in the user's positioning relative to the handheld or fixed device (e.g., the user moves towards, away from, up, down, right, and / or left). For devices including displays with video pass-through, a portion of the physical environment visible (e.g., displayed and / or projected) via one or more displays is based on the field of view of one or more cameras communicating with the displays, which typically move with the displays (e.g., with the user's head for head-mounted devices, or with the user's hands for handheld devices such as tablets or smartphones), because the user's viewpoint moves with the field of view of one or more cameras (and updates the appearance of one or more virtual objects displayed via one or more displays based on the user's viewpoint (e.g., the display position and pose of virtual objects are updated based on the movement of the user's viewpoint)).For displays with optical perspective, the portion of the physical environment visible through one or more displays (e.g., optically visible through one or more portions or fully transparent portions of the display generating components) is based on the user's field of view through the portions or fully transparent portions of the display generating components (e.g., moving with the user's head for head-mounted devices, or moving with the user's hand for handheld devices such as tablets or smartphones), because the user's viewpoint moves with the user's field of view through the portions or fully transparent portions of the display (and the appearance of one or more virtual objects is updated based on the user's viewpoint).

[0045] exist Figure 3A In this system, electronic device 101 communicates with one or more additional devices (e.g., electronic devices 303 and / or 305). The combination of devices communicating with electronic device 101 is referred to as a computer system (e.g., electronic devices 101, 303, and 305). In some examples, electronic devices 303 and / or 305 have one or more characteristics of electronic devices 201 and / or 260. In some examples, electronic device 303 is a smartphone, and electronic device 305 is a smartwatch communicating with electronic device 101. For example, electronic devices 101, 303, and / or 305 communicate wirelessly (e.g., Bluetooth, Wi-Fi, and / or wireless networks) or wiredly (e.g., via wires and / or cables, such as Universal Serial Bus A (USB-A), Universal Serial Bus C (USB-C), and / or Ethernet). In some examples, electronic devices 101, 303, and / or 305 share a common user account and / or user (e.g., common users / accounts use these devices). For example, a user may optionally log in to one or more electronic devices using a user account (e.g., a username and password).

[0046] In some examples, one or more electronic devices 101, 303, and / or 305 each include one or more sensors and / or one or more displays. For example, the electronic devices may include an accelerometer, a global positioning sensor (GPS), an image sensor (image sensor 206A and / or 206B), an orientation sensor (e.g., orientation sensor 210A and / or 210B), and / or a position sensor (e.g., position sensor 204A and / or 204B). In some examples, when in a first power state, electronic device 101 does not include a powered display, such as... Figure 3A As shown. For example, in Figure 3AIn this embodiment, electronic device 101 does not use display 120 to display content, but instead presents a transparent view of the physical environment 350. When in a first power state, electronic devices 101, 303, and / or 305 operate a first subset of input devices, such as microphones, GPS, position sensors, orientation sensors, accelerometers, and other background sensors. In some examples, a second set of input devices requiring more battery power (such as image sensors, higher-power processors, and displays) is inactive when in the first power state. In some examples, when in the first power state, electronic device 101 uses the first subset of input devices for sensing and receives sensor data related to sound, position, and orientation from the input devices of electronic devices 303 and / or 305.

[0047] Figure 3A This example illustrates a user going to work at a bus stop. Figure 352 shows a top-down view of the physical environment 350, where user 317 is walking towards bus stop 312 to catch bus 313. (See figure 352.) Figure 3A As shown, electronic device 101 presents a portion 300 of physical environment 350 via display 120, which includes bus stop 312 and bus 313. In some examples, electronic devices 101, 303, and / or 305 use microphones to capture one or more sounds 310a of physical environment 350. For example, the sounds are related to urban outdoor environments (e.g., bird calls, car sounds, bus sounds, airplane sounds, and / or other urban and outdoor sounds). In some examples, electronic devices 101, 303, and / or 305 also use one or more position sensors to capture location data indicating a user's location at bus stop 312. In some examples, electronic device 101 and / or the computer system can store data from one or more sensors as historical data, which can be used to notify the device of one or more future actions. For example, electronic devices 101, 303, and / or 305 can use sound and / or location data to transition power states, such as... Figures 3B to 3D More detailed descriptions are available in the text.

[0048] In the examples described herein, the computer system may be described as performing a function (e.g., the computer system stores historical data, as described above). However, it should be understood that any of the electronic devices communicating with electronic device 101 as described herein may perform any or all of the steps constituting the corresponding function. For example, the computer system comprises electronic devices 101, 303, and 305, and any of the electronic devices within the computer system may perform any or all of the steps constituting the corresponding function described herein.

[0049] exist Figure 3AIn this context, electronic device 101 and / or computer system detect one or more criteria for a state of transition power that are not met. In some examples, the one or more criteria for the state of transition power are based on one or more previous patterns of the user. For example, electronic device 101 and / or computer system do not detect (e.g., via sensors on electronic device 101 or via sensors on electronic devices 303 and / or 305 communicating with electronic device 101) sounds that meet the criteria, locations that meet the criteria, and / or other criteria that meet the criteria. As described below, in Figure 3B In this process, electronic devices 101, 303 and / or 305 detect data consistent with the user's previous modes (e.g., meeting one or more criteria), which require electronic device 101 to transition from a first power state to a second power state.

[0050] exist Figure 3B In the image, the user is sitting on bus 313, and electronic device 101 displays a portion 300 of the physical environment 350 corresponding to the user's seating position on bus 313. Figure 3BIn the example shown, 352 illustrates a user 317 on bus 313. Additionally, after detecting that electronic device 101 (and the user of electronic device 101) is on bus 313, electronic device 101 transitions to a second power state and displays user interface elements 304 and 306 on display 120. In some examples, one or more criteria are met when electronic device 101 and / or the computer system detects that electronic device 101 is on bus 313. In some examples, electronic device 101 and / or the computer system detect that electronic device 101 is on bus 313 because electronic devices 101, 303, and / or 305 detect a sound change corresponding to the user's entry and presence on bus 313 (e.g., from sound 310a to sound 310b). Sound 310b may include sounds associated with being on the bus (e.g., people talking and the sound of the bus). In some examples, sound 310b may be quieter than sound 310a because the environment on the bus is more pleasant. In some examples, electronic devices 101, 303, and / or 305 use accelerometers, GPS, position sensors, and / or orientation sensors to detect changes in acceleration / movement consistent with that on a vehicle (e.g., increased speed and / or acceleration, changes in position, changes in orientation). Additionally, in some examples, when electronic devices 101, 303, and 305 are connected to a Wi-Fi network corresponding to a specific location (e.g., a bus Wi-Fi network), electronic device 101 detects that it is at that specific location (e.g., on bus 313). Alternatively or additionally, one or more criteria are met based on time and / or date. For example, based on previous user activity (e.g., user location data, voice data, mobility data, calendar data, or other data), electronic device 101 determines that a user entered bus 313 at a specific time and / or date (e.g., 8:00 AM, Monday through Friday). In some examples, electronic device 101 may use positioning and / or orientation sensors to determine the position of user 317's head. In some examples, electronic device 101 may use eye-tracking sensors and / or hand-tracking sensors to determine one or more locations of the user 317's eyes. For example, electronic device 101 may use head-tracking sensors, hand-tracking sensors, and eye-tracking sensors to determine the environmental context (e.g., the user is sitting in a bus 313). In some examples, electronic device 101 may use one or more image sensors to capture images of the physical environment 300 to determine the environmental context. In some examples, data from a first set of input devices of electronic devices 303 and / or 305 and data from a first set of input devices of electronic device 101 are used to determine whether one or more criteria are met.

[0051] In some examples, electronic device 101 displays user interface elements 304 and 306 in response to detecting that one or more criteria are met. In some examples, user interface elements 304 and 306 (respectively) correspond to the volume and play buttons of a music application. In some examples, electronic device 101 displays user interface elements 304 and 306 based on detecting that a user is on bus 313 and / or based on historical data. In some examples, electronic device 101 receives one or more inputs corresponding to a request to play music while on bus 313 (e.g., a specific bus photographed at a specific time and location and / or any bus photographed by the user of electronic device 101). In some examples, and as described above, electronic device 101 and / or the computer system store sensor data (e.g., sensors from electronic device 101 or electronic devices 303 and / or 305) and determine patterns based on the user's previous actions and corresponding sensor data. In some examples, one or more criteria are dynamic based on the environmental context captured by a first set of one or more sensors. For example, electronic device 101 and / or computer system may recognize one or more user patterns (e.g., playing music on bus 313, checking tasks at a location (e.g., office location), activating Do Not Disturb mode or other modes at a specific time (e.g., bedtime)). The corresponding pattern may include a corresponding criterion (e.g., time criterion, sound criterion, location criterion, and / or other criterion) that must be met for electronic device 101 to transition from a first power state to a second power state.

[0052] In some examples, one or more criteria are based on one or more user patterns, as described above. Alternatively or additionally, in some examples, one or more criteria are based on user preferences. For example, a user may set one or more actions for electronic device 101 (or electronic devices 303 and / or 305) to perform (e.g., in a second power state). For example, when in a particular location, a user may request activation of a first application (e.g., running a background application and / or displaying the user interface of the first application).

[0053] exist Figure 3B In the second power state, when operating and displaying user interface elements 304 and 306, the electronic device 101 receives selection input directed to user interface element 306. In some examples, and such as Figure 3B As shown, the input can be gaze input using eyes 320. Alternatively or additionally, in some examples, the input can be air pinch input using hands 315, such as... Figure 3B As shown in the diagram. Alternatively or additionally, in some examples, the input may be a combination of gaze input using eye 320 and predefined movement of hand 315. In response to receiving Figure 3BThe input shown in the image indicates that the electronic device 101 begins to display. Figure 3C User interface element 310 is shown.

[0054] In some examples, in response to the detection that one or more criteria are met, electronic device 101 remains in a first power state while also running background applications. For example, electronic device 101 does not display... Figure 3B Instead of using user interface elements 304 and 306, the device automatically starts playing music in response to determining that one or more criteria are met. In some examples, the electronic device 101 can reduce power consumption by forgoing activation of the display 120 while also performing actions based on historical data and / or user preferences.

[0055] Figure 3C An example is an electronic device 101 that displays user interface elements 310, 304, and 306 on a display 120. Figure 3C In the middle, electronic device 101 updates user interface element 306 to include an icon for the pause button instead of... Figure 3B The play button shown indicates that music is playing (e.g., using one or more speakers of electronic device 101 or one or more speakers of a device communicating with electronic device 101 (e.g., electronic devices 303 and / or 305)), and further input to user interface element 306 will cause electronic device 101 to pause playback of the content. Figure 3C In this context, electronic device 101 also displays a user interface element 310 that includes a representation of the currently playing playlist. In some examples, electronic device 101 plays a featured playlist, resumes previously paused music, or plays a previously selected playlist (e.g., music from a music library, music from a favorite songs playlist, or other playlists downloaded or saved to the music app / electronic device 101, 303, or 305). In some examples, the featured playlist is based on historical data of music types played in the same or similar context, such as music played on bus 313, at a given time, and / or at a given location. In some examples, the historical music data is based on metadata of previously played music. For example, electronic device 101 stores historical music data indicating that upbeat pop music was frequently played when electronic device 101 was on bus 313. Based on this historical music data, electronic device 101's featured playlist includes a selection of upbeat pop music to play, such as... Figure 3C As shown in the image.

[0056] exist Figure 3DIn this scenario, the user arrives at their desired location and disembarks from the bus. Figure 352 illustrates a top-down view of the physical environment 350, showing the user in front of building 311 and no longer on bus 313. In some examples, in a first power state, electronic device 101 and / or computer system determine that user 317 has disembarked from the bus and is at or within a threshold distance (e.g., 1m, 5m, 10m, 100m, or 500m) of building 311. For example, before displaying user interface element 302 and before displaying... Figure 3C Following the user interface elements 310, 304, and 306 shown, electronic device 101 transitions back to the first power state. In some examples, electronic device 101 transitions from the second power state to the first power state after a threshold time amount (e.g., 5 seconds, 10 seconds, 30 seconds, 1 minute, or 5 minutes) without detecting any input to electronic device 101 (e.g., electronic device 101 times out). In some examples, electronic device 101 transitions from the second power state to the first power state when one or more criteria are no longer met (e.g., electronic devices 101, 303, and / or 305 are no longer in a specific location, a specific time has elapsed, one or more sounds are no longer detected, and / or user 317 deactivates the user interface elements and / or turns off display 120).

[0057] In some examples, and as described above, when in a first power state, electronic device 101 uses data from (e.g., electronic device 101 or electronic devices 303 and / or 305) microphones, position sensors, and / or orientation sensors to determine the environmental context of user 317. In some examples, electronic device 101 determines that the ambient sound has changed from sound 310b (e.g., sound from inside bus 313) to sound 310c (e.g., sound from outside building 311). In some examples, electronic device 101 also determines that sound 310c is consistent with historical sound data associated with the current location of electronic device 101 (e.g., outside building 311). In some examples, changes in location, changes in movement of one or more electronic devices (e.g., electronic devices 101, 303, and / or 305), and / or changes in sound satisfy one or more criteria. Additionally or alternatively, in some examples, electronic device 101 uses one or more head tracking sensors, hand tracking sensors, and / or eye tracking sensors to determine the environmental context of user 317. For example, electronic device 101 uses a head-tracking sensor to determine the location of the user's head (e.g., looking down as they exit a bus). In some examples, electronic device 101 uses one or more image sensors to determine the user's environmental context. For example, electronic device 101 uses image sensors to capture the physical environment 350 (such as...). Figure 3DThe image shows a portion 300 of the physical environment. In some examples, when in a first power state, the electronic device 101 operates the image sensor, head tracking sensor, hand tracking sensor, and eye tracking sensor at a lower frequency than when in a second power state. In response to determining that one or more criteria are met, the electronic device 101 transitions from the first power state to the second power state and displays user interface elements 302, such as... Figure 3D As shown in the image.

[0058] In some examples, and as described above, one or more criteria are based on historical data. For example, building 311 is a frequently visited location based on historical location data (e.g., user 317's workplace). For example, electronic devices 101, 303, and / or 305 include historical location data based on one or more location sensors (e.g., GPS) that indicate frequent user visits to building 311. In some examples, the historical location data includes time data of visits (e.g., user 317 visits building 311 from approximately 8:00 AM Monday through Friday and leaves around 5:00 PM). In some examples, electronic devices 101, 303, and / or 305 detect that at a given time, at a given location, and / or after a specific action (e.g., at 8:00 AM, at building 311, after user 317 is no longer on bus 313), user 317 uses one or more devices to check an email application, a text messaging application, and / or an alert application. In response to historical data, when electronic device 101 is in front of building 311, it displays a user interface element 302 that includes indications of notifications from one or more frequently visited applications. In some examples, electronic device 101 displays user interface element 302 based on user preferences. For example, and as described above, user 317 can determine what indications to display on user interface element 302.

[0059] In some examples, electronic device 101 will not transition from a first power state to a second power state if one or more criteria are not met. For example, if electronic devices 101, 303, and / or 305 do not detect user 317... Figure 3B On bus 313 (e.g., using one or more sensors as described above), electronic device 101 will not transition to the second power state and will not display user interface elements 304 and 306. Similarly, if electronic device 101 detects that a user has arrived at the corresponding time, in addition to including... Figure 3DIf the location is outside the location of building 311, the electronic device 101 will not switch to the second power state and will not display user interface element 302. For example, user 317 takes a bus in the morning and arrives at the second location at 8:00 AM, which is different from the location of building 311. In some examples, if the second location is associated with one or more historical patterns and meets one or more criteria, the electronic device 101 switches to the second power state and displays content related to the second location.

[0060] In some examples, one or more criteria change as user preferences and / or historical data change. For example, if the user frequently toggles user interface elements 304 and 306 while on bus 313, the electronic device 101 may stop transitioning to a second power state to display elements of the music application while on bus 313.

[0061] Figures 4A to 4C Different examples are illustrated in which electronic device 101 transitions from a first power state to a second power state based on information collected from one or more input devices. Figures 4A to 4C Used to illustrate the process described below, including Figure 6 The process 600 is shown in the diagram. In some examples, the user may be in a situation such as a car malfunction, where it is advantageous to display additional information on the display 120 of the electronic device 101. In some examples, using one or more input sensors, the electronic device 101 may determine the environmental context and display relevant information as a result.

[0062] Figure 4A An example is shown of a portion 400 of the physical environment 450 presented from the user's viewpoint via a transparent or optically perceptible display by an electronic device 101 via a display 120. Figure 4A The illustration shows the user of electronic device 101 when operating vehicle 420. Electronic device 101 interacts with one or more additional devices (such as...) Figure 4A The electronic devices 303 and / or 305 shown and described in more detail above communicate. Figure 4A In this context, sensors (e.g., accelerometers and / or other position, orientation, or movement sensors) of electronic devices 101, 303, and / or 305 detect that the corresponding device is moving at 70 km / h with an acceleration of 0 m / s^2 (e.g., because the user is driving at 70 km / h). Figure 4A In this context, the sensors of electronic devices 101, 303 and / or 305 detect that the user is inside the vehicle (e.g., sound 402a).

[0063] exist Figure 4BIn this embodiment, sensors of electronic devices 101, 303, and / or 305 detect a change in speed, such that the corresponding device has a speed of 2 km / h and a deceleration of 5 m / s² (e.g., car 420 is decelerating to stop moving). In some examples, the detected acceleration curve (e.g., speed change) may correspond to a known acceleration curve, such as the acceleration curve of a tire blowout event. Additionally, in some examples, sensors of electronic devices 101, 303, and / or 305 detect a change in sound from sound 402a to sound 402b. In some examples, sound 402b includes a loud sound consistent with a tire blowout. In some examples, electronic device 101 uses a characteristic sound from sound 402b corresponding to a tire blowout to detect a tire blowout event. Additionally or alternatively, for example, a user searches for "how to change a tire" on an internet browser user interface 410 on electronic device 303. In some examples, additionally or alternatively, a user may perform a search on an internet browser user interface of electronic device 101 or electronic device 305. In some examples, electronic device 101 may use head tracking sensors, eye tracking sensors and / or hand tracking sensors to detect sudden changes in gaze, head movement or hand movement that may coincide with a tire blowout.

[0064] exist Figure 4C In this embodiment, electronic device 101 transitions from a first power state to a second power state after determining that one or more criteria are met. In some examples, one or more criteria are met when electronic device 101 determines that the confidence level for transitioning from the first power state to the second power state exceeds a confidence threshold (e.g., 51%, 60%, 75%, 90%, 95%, or 99%). In some examples, the confidence level is based on the probability that the user is in a context in which electronic device 101 can display contextual information. For example, in… Figure 4CIn this embodiment, electronic device 101 and / or computer system detect via electronic device 303 that the user has searched for "how to change a tire" on a web browser application. Additionally, electronic device 101 and / or computer system detect via sensors of electronic device 305 that the user is in a crouching position. For example, electronic device 305 and / or computer system use one or more position and orientation sensors to determine whether the user is crouching. Additionally, electronic device 101 and / or computer system detect via microphones (e.g., on electronic device 101, electronic device 303, and / or electronic device 305) that the ambient sound has changed to sound 402c, which corresponds to an outdoor sound. Additionally, in some examples, electronic device 101 and / or computer system use image sensors, head tracking sensors, eye tracking sensors, and / or hand tracking sensors to increase the confidence level of the transition to the second power state. For example, an image sensor may indicate that the user is outside their car and / or is looking at a flat tire. An eye tracking sensor and / or head tracking sensor may indicate that the user is looking at a flat tire. In some examples, a combination of information from sensor data indicates that electronic device 101 is in a context where the confidence level exceeds a threshold confidence level, and therefore electronic device 101 should transition from a first power state to a second power state. In some examples, the context where the confidence level exceeds the threshold confidence level includes a context where the user may require additional information (e.g., such as...). Figure 4C The video shown is about how to change a tire, and the context is based on historical information (e.g., Figures 3A to 3D (the context described in the document) and / or the context in which electronic device 101 receives from a second electronic device an instruction to be displayed on electronic device 101 (e.g., a notification from an application).

[0065] exist Figure 4C In this embodiment, electronic device 101 displays user interface element 411 via display 120. This user interface element includes a video showing how to change a tire based on a combination of sensor data from electronic devices 101, 303, and 305. In some examples, electronic device 101 displays user interface element 411, which improves user device interaction because the user can view information about changing tires without having to use electronic device 303 and / or provide additional input to electronic device 101 to search for information about changing tires.

[0066] In some examples, electronic device 101 is not displayed. Figure 4B User interface element 411 in the example is excluded because the confidence level has not yet exceeded the confidence threshold. For example, in... Figure 4BIn this system, electronic device 101 and / or computer system detect changes in speed and data from electronic device 303 via a web browser application. However, electronic device 101 does not detect changes in sound or the user's crouching motion. For example, the user may receive assistance from someone else to change a tire, and therefore electronic device 101 does not need to display user interface element 411.

[0067] Although Figures 4A to 4C An example of a tire blowout has been described, but similar actions could occur if electronic devices 101, 303, and / or 305 detect a collision. For example, using one or more sensors, electronic device 101 and / or the computer system can detect a collision (e.g., a change in acceleration, an increase in force on electronic device 101, sound, a sudden change in head positioning). In response to detecting a collision, electronic device 101 can transition from a first power state to a second power state and display relevant collision information (e.g., displaying insurance information and / or displaying prompts including post-collision indications (e.g., collecting information from other parties, taking photos, and / or issuing alarms)).

[0068] Figures 5A to 5B An example is illustrated in which electronic device 101 transitions from a first power state to a second power state based on non-contextual information collected from one or more input devices. Figures 5A to 5B Used to illustrate the process described below, including Figure 6 The process 600 is shown in the diagram. In some examples, when the electronic device 101 is in a second power state, the electronic device 101 may opportunistically collect additional information about the user environment to inform future actions.

[0069] Figure 5A This illustrates a portion 500 of the physical environment 550 presented from the user's viewpoint via a transparent or optically perceptible display by an electronic device 101, through a display 120. Figure 5A The image shows a user of electronic device 101 watching a rainstorm. Electronic device 101 is connected to one or more additional devices (such as...). Figure 5A The electronic devices 303 and / or 305 shown and described in more detail above communicate.

[0070] exist Figure 5A In this context, electronic device 101 and / or computer system receive an instruction for a text message from a messaging application. In response to receiving the instruction, electronic device 101 transitions from a first power state to a second power state to display a visual instruction 503 of the text message via display 120. In some examples, and as such... Figures 3A to 3D and Figures 4A to 4CAs described, electronic device 101 transitions from a first power state to a second power state based on environmental context and / or historical data. In some examples, electronic device 101 also transitions from the first power state to the second power state in response to receiving an instruction to be displayed on electronic device 101 (e.g., normal operation of electronic device 101). When electronic device 101 is in the second power state, electronic device 101 opportunistically uses a second subset of one or more input devices operating in the second power state to collect additional information. For example, electronic device 101 activates one or more image sensors (e.g., described in more detail in Figure 2) when operating in the second power state. Figure 5A In this system, one or more image sensors capture characteristics of the physical environment 550, such as the fact that it is raining. In some examples, electronic devices 101 and / or computer systems use additional processing power available in a second power state to access the Internet to collect weather data.

[0071] Additionally, such as Figure 5A As illustrated, the electronic device 303 includes a first wallpaper 502a. In some examples, the wallpaper 502a is computer-generated and / or user-selected.

[0072] Figure 5B An example is given of electronic device 303 responding to electronic device 101 being in a state of... Figure 5A The information captured during the second power state is used to update the wallpaper from wallpaper 502a to wallpaper 502b. For example... Figure 5A As described, when electronic device 101 is in a second power state, electronic device 101 uses an additional input device to capture context information. Electronic device 101 sends the context information to one or more devices communicating with electronic device 101 (e.g., electronic devices 303 and / or 305). In some examples, electronic device 303 updates the wallpaper in response to receiving the context information. In some examples, electronic device 303 updates the wallpaper based on user preferences or according to a computer-generated wallpaper. For example, a user may determine one or more wallpapers to use during a given weather period, or electronic device 303 may determine different wallpapers for a given weather condition.

[0073] exist Figure 5B In this state, electronic device 101 stops displaying visual indication 503 and transitions back to the first power state. As described above, electronic device 101 can transition from the second power state back to the first power state after a time threshold without detecting user interaction with the displayed elements or responding to user input.

[0074] Although Figures 5A to 5BAn example is illustrated where opportunistic sensing causes action on electronic device 303, but in some examples, opportunistically sensing data while in a second power state may cause one or more actions to be performed on electronic device 101, electronic device 303, and / or electronic device 305. When non-contextual information is displayed on electronic device 101 while it is in a second power state, electronic device 101 may capture additional data that satisfies one or more criteria, as described above. In response to the satisfaction of one or more criteria, electronic device 101 may remain in the second power state to perform additional actions corresponding to the satisfaction of one or more criteria. In some examples, electronic devices 303 and / or 305 may transition from a first power state to a second power state and perform opportunistic sensing that results in the capture of additional data that satisfies one or more criteria to transition electronic device 101 from the first power state to the second power state. For example, electronic device 303 may receive and display a notification from a first application. When displaying the notification from the first application, electronic device 303 detects additional data that causes electronic device 101 to transition from the first power state to the second power state.

[0075] Figure 6 Flowcharts illustrating example processes for transitioning an electronic device from a first power state to a second power state, according to some examples of this disclosure, are shown. In some examples, process 600 begins at a first electronic device having one or more first displays and one or more input devices, which communicates with a second electronic device. In some examples, the second electronic device includes one or more second displays and one or more second input devices. In some examples, the first electronic device may optionally be the electronic device 201 or 260 of FIG2 and... Figures 3A to 3D , Figures 4A to 4C and Figures 5A to 5B The electronic device 101 is similar to or corresponds to a head-mounted display. For example... Figure 6 As shown, in some examples, at 602, when the first electronic device is in a first power state, the first electronic device detects a first set of information via a first subset of the first input devices from one or more first input devices. In some examples, the first subset of input devices includes position sensors, orientation sensors, microphones, and other low-power sensors, such as... Figure 3A As described in [the document / document]. In some examples, the first set of information includes information to inform electronic device 101 of the environmental context.

[0076] In some examples, at 604, the first electronic device (e.g., electronic device 101) receives from the second electronic device a second set of information that differs from the first set of information, detected using one or more second input devices of the second electronic device. In some examples, electronic device 101 receives sensor data from input devices of one or more electronic devices (such as electronic device 303 and / or electronic device 305) communicating with electronic device 101, such as... Figure 3A and Figure 4B As described in more detail below. In some examples, electronic device 101 may receive motion data from a smartwatch, location data from a phone, or other data as described above to inform the environmental context.

[0077] In some examples, at 606, based on determining that one or more first criteria are met, including criteria satisfied based on a first set of information and a second set of information, a first electronic device (e.g., electronic device 101) transitions from a first power state to a second power state, wherein the second power state is associated with a power state higher than the first power state. In some examples, electronic device 101 transitions from the first power state to the second power state if the environmental context based on the first set of information and the second set of information meets one or more criteria based on historical data, current sensor data, and / or user preferences. Figures 3A to 3D As described, if prior historical data indicates a user's pattern, and if the environmental context corresponds to one or more patterns in the user's pattern, the electronic device 101 transitions to a second power state to perform one or more actions. In some examples, and as... Figures 4A to 4C As described, if the confidence level based on the environmental context (e.g., one or more sensor data) exceeds a confidence threshold, electronic device 101 transitions to a second power state to display relevant information. In some examples, at 608, based on the determination that one or more first criteria are not met, the first electronic device (e.g., electronic device 101) abandons the transition from the first power state to the second power state.

[0078] It should be understood that process 600 is an example, and more, fewer, or different operations may be performed in the same or different order. Additionally, the operations in process 600 described above may optionally be implemented by running one or more functional modules in an information processing device or dedicated chip, such as a general-purpose processor (e.g., as described with respect to Figure 2), and / or by other components of Figure 2. Additionally, in some examples, one or more operations described in process 600 may optionally be performed at any of the electronic devices in the computer system (e.g., electronic devices 101, 303, and / or 305).

[0079] Therefore, according to the foregoing, some examples of this disclosure relate to a method comprising: at a first electronic device having one or more first displays and one or more first input devices, the first electronic device communicating with a second electronic device having one or more second input devices: when the first electronic device is in a first power state, detecting a first set of information via a first subset of the first input devices; receiving from the second electronic device a second set of information detected by the one or more second input devices that differs from the first set of information; transitioning the first electronic device from the first power state to a second power state, wherein the second power state is associated with a power state higher than the first power state, based on determining that one or more first criteria are met, the one or more first criteria including criteria based on the first set of information and the second set of information; and abandoning the transition of the first electronic device from the first power state to the second power state based on determining that the one or more first criteria are not met. As a supplement or alternative to one or more examples of the examples disclosed above, in some examples, the first power state includes operating the first electronic device without operating the one or more first displays, and the second power state includes operating the first electronic device, including operating the one or more first displays. As a supplement to or alternative to one or more of the examples disclosed above, in some examples, operating the one or more first displays while in the second power state includes displaying a first user interface on the one or more displays of the first electronic device, wherein the first user interface is based on a first set of information and a second set of information. As a supplement to or alternative to one or more of the examples disclosed above, in some examples, transitioning from the first power state to the second power state includes activating a first application and a second subset of the one or more first input devices, the second subset being different from the first subset of the one or more first input devices. As a supplement to or alternative to one or more of the examples disclosed above, in some examples, the first electronic device deactivates the one or more first displays while in both the first power state and the second power state. As a supplement or alternative to one or more of the examples disclosed above, in some examples, transitioning the first electronic device from the first power state to the second power state further includes: activating a second subset of the one or more first input devices that are inactive in the first power state; and after activating the second subset of the one or more first input devices, detecting a third set of information via the second subset of the one or more first input devices, wherein the third set of information is used to notify the first electronic device and the second electronic device of one or more functions.As a supplement or alternative to one or more of the examples disclosed above, in some examples, the one or more first criteria include a second criterion satisfied based on one or more historical patterns of the user of the first electronic device. As a supplement or alternative to one or more of the examples disclosed above, in some examples, transitioning the first electronic device from the first power state to the second power state further includes: based on determining that the first information and the second information correspond to a first context, displaying one or more first user interface elements corresponding to the corresponding context information from the first information and the second information via the one or more first displays when in the second power state; and based on determining that the first information and the second information correspond to a second context, displaying one or more second user interface elements corresponding to the corresponding context information from the first information and the second information via the one or more first displays when in the second power state. As a supplement or alternative to one or more of the examples disclosed above, in some examples, transitioning the first electronic device from the first power state to the second power state further includes: displaying, when in the second power state, one or more first user interface elements corresponding to corresponding non-contextual information from the first information and the second information via the one or more first displays, based on determining that the first information and the second information correspond to a first context; and displaying, when in the second power state, one or more second user interface elements corresponding to corresponding contextual information from the first information and the second information via the one or more first displays, based on determining that the first information and the second information correspond to a second context, one or more second user interface elements corresponding to corresponding contextual information from the first information and the second information via the one or more first displays. As a supplement or alternative to one or more of the examples disclosed above, in some examples, the first subset of the first input devices in the one or more first input devices includes an image sensor, a hand tracking sensor, and / or a head tracking sensor.

[0080] Some examples of this disclosure relate to an electronic device comprising: one or more processors; a memory; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described above.

[0081] Some examples of this disclosure relate to a non-transitory computer-readable storage medium that stores one or more programs, the one or more programs including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform any of the methods described above.

[0082] Some examples of this disclosure relate to an electronic device that includes one or more processors, a memory, and components for performing any of the methods described above.

[0083] Some examples of this disclosure relate to an information processing apparatus used in an electronic device, the information processing apparatus including components for performing any of the methods described above.

[0084] This disclosure anticipates that, in some cases, the data used may include personal information that uniquely identifies or can be used to contact or locate specific individuals. Such personal information may include demographic data, content consumption activity, location-based data, telephone numbers, email addresses, Twitter IDs, home addresses, data or records relating to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information. Specifically, as described herein, one aspect of this disclosure is the use of a microphone to track a user's location and / or voice.

[0085] This disclosure recognizes that the use of such personal information data in the present invention can benefit users. For example, personal information data can be used to display suggested text that changes based on changes in a user's biometric data. For example, suggested text can be updated based on changes in a user's age, height, weight, and / or medical history.

[0086] This disclosure anticipates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with robust privacy policies and / or privacy measures. Specifically, such entities should implement and adhere to privacy policies and measures that are recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable entity purposes and should not be shared or sold outside of these legitimate purposes. Furthermore, such collection / sharing should be conducted only after receiving informed consent from users. Additionally, such entities should consider taking any necessary steps to protect and safeguard the right to access such personal information data and ensure that other entities with access to personal information data comply with the privacy policies and procedures of other entities. Furthermore, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and privacy measures. Moreover, policies and measures should be adapted to the specific types of personal information data collected and / or accessed, and to applicable laws and standards, including considerations of specific jurisdictions. For example, in the United States, the collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while in other countries, health data may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy measures should be advocated for different types of personal data in each country.

[0087] Regardless of the foregoing, this disclosure also anticipates examples of users selectively blocking the use or access to personal information data. That is, this disclosure anticipates providing hardware and / or software components to prevent or block access to such personal information data. For example, the inventive technology can be configured to allow a user to opt-in or opt-out during or at any time after registering for the service. In another example, a user can choose not to enable the recording of personal information data in a specific application (e.g., a first application and / or a second application). In addition to providing "opt-in" and "opt-out" options, this disclosure also anticipates providing notifications related to access to or use of personal information. For example, a user can be notified when collection is initiated that their personal information data will be accessed, and then reminded again just before the device accesses the personal information data.

[0088] Furthermore, the intent of this disclosure is that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by restricting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods.

[0089] For purposes of explanation, the foregoing description has been presented with reference to specific examples. However, the illustrative arguments above are not intended to be exhaustive or to limit this disclosure to the precise form disclosed. Many modifications and variations are possible in accordance with the foregoing teachings. The examples were chosen and described to best elucidate the principles of this disclosure and its practical application, thereby enabling others skilled in the art to best utilize this disclosure with various modifications suitable for the particular intended use, as well as the various described examples.

Claims

1. A method, the method comprising: Obtain a first set of information detected by a first subset of one or more first input devices of the first electronic device when the first electronic device is in a first power state; Obtain a second set of information that is different from the first set of information detected by a first subset of one or more second input devices of a second electronic device, wherein the second electronic device communicates with the first electronic device; Based on determining that one or more first criteria are met, the one or more first criteria including criteria based on a first set of information and a second set of information, the first electronic device transitions from a first power state to a second power state, wherein the second power state is associated with a power state higher than the first power state. as well as If it is determined that one or more of the first criteria are not met, the transition of the first electronic device from the first power state to the second power state is abandoned.

2. The method of claim 1, wherein the first power state includes operating the first electronic device without operating one or more first displays of the first electronic device, and the second power state includes operating the first electronic device, including operating the one or more first displays.

3. The method of claim 1, wherein operating the one or more first displays in the second power state includes displaying a first user interface on the one or more displays of the first electronic device, wherein the first user interface is based on the first set of information and the second set of information.

4. The method of claim 1, wherein the transition from the first power state to the second power state includes activating a first application and a second subset of the one or more first input devices, the second subset being different from the first subset of the one or more first input devices.

5. The method of claim 1, wherein when in the first power state and the second power state, the first electronic device deactivates one or more first displays of the first electronic device.

6. The method of claim 1, wherein transitioning the first electronic device from the first power state to the second power state further comprises: Activate a second subset of the one or more first input devices that are inactive in the first power state; as well as After activating the second subset of the one or more first input devices, a third set of information is detected via the second subset of the one or more first input devices, wherein the third set of information is used to notify the first electronic device and the second electronic device of one or more functions.

7. The method of claim 1, wherein the one or more first criteria include a second criterion satisfied based on one or more historical patterns of the user of the first electronic device.

8. The method of claim 1, wherein transitioning the first electronic device from the first power state to the second power state further comprises: Based on the first set of information and the second set of information corresponding to a first context, such that when in the second power state, one or more first user interface elements corresponding to the corresponding context information from the first set of information and the second set of information are displayed via one or more first displays of the first electronic device; as well as Based on the first set of information and the second set of information corresponding to a second context, such that when in the second power state, one or more second user interface elements corresponding to the corresponding context information from the first set of information and the second set of information are displayed via the one or more first displays.

9. The method of claim 1, wherein transitioning the first electronic device from the first power state to the second power state further comprises: Based on the first set of information and the second set of information corresponding to a first context, such that when in the second power state, one or more first user interface elements corresponding to the corresponding non-contextual information from the first set of information and the second set of information are displayed via one or more first displays; as well as Based on the first set of information and the second set of information corresponding to a second context, such that when in the second power state, one or more second user interface elements corresponding to the corresponding context information from the first set of information and the second set of information are displayed via the one or more first displays.

10. The method of claim 1, wherein the first subset of the first input devices in the one or more first input devices includes an image sensor, a hand tracking sensor, and / or a head tracking sensor.

11. An electronic device, the electronic device comprising: One or more processors; Memory; and One or more programs, the programs being stored in the memory and configured to be executed by the one or more processors, the programs including instructions for performing the method according to any one of claims 1 to 10.

12. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 10.

13. A computer system, the computer system comprising: One or more processors; Memory; and Components for performing any one of the methods according to claims 1 to 10.

14. A computer system that communicates with one or more displays and one or more input devices, the computer system comprising: One or more processors; Memory; A component for obtaining a first set of information detected by a first subset of one or more first input devices of the first electronic device when the first electronic device is in a first power state; A component for obtaining a second set of information that is different from the first set of information detected by a first subset of one or more second input devices of a second electronic device, wherein the second electronic device communicates with the first electronic device; The component is used to: cause the first electronic device to transition from a first power state to a second power state based on determining that one or more first criteria are met, the one or more first criteria including criteria based on a first set of information and a second set of information, wherein the second power state is associated with a power state higher than the first power state; and A component for abandoning the transition of the first electronic device from the first power state to the second power state based on the determination that one or more of the first criteria are not met.

15. An electronic device, the electronic device comprising: One or more input devices; One or more displays; One or more processors; Memory; and One or more programs, the programs being stored in the memory and configured to be executed by the one or more processors, the programs including instructions for performing the method according to any one of claims 1 to 10.

Images