Techniques for illuminating a physical space

Abstract

The present disclosure generally relates to simulating light in a physical space, such as simulating light passing through a physical opening, for simulating different amounts of light passing through a physical opening, and / or for adjusting one or more lighting conditions.

Description

Cross-references to related applications

[0001] This international application claims priority to U.S. Nonprovisional Patent Application Serial No. 19 / 056,198, filed February 18, 2025, entitled “TECHNIQUES FOR ILLUMINATING A PHYSICAL SPACE”, and U.S. Provisional Patent Application Serial No. 63 / 567,821, filed March 20, 2024, entitled “TECHNIQUES FOR ILLUMINATING A PHYSICAL SPACE”, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0002] This disclosure relates generally to computer user interfaces, and more specifically to techniques for illuminating physical spaces. Background Technology

[0003] Lights illuminate physical spaces. These lights can be controlled by the user to change how one or more physical spaces are illuminated. Summary of the Invention

[0004] However, some technologies used to illuminate physical spaces with electronic devices are often cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may include multiple buttons or keystrokes. These technologies require more time than necessary, resulting in wasted user time and device power. This latter consideration is particularly important in battery-powered devices.

[0005] Therefore, this technology provides electronic devices with faster and more efficient methods and interfaces for illuminating physical spaces. Such methods and interfaces can optionally complement or replace other methods for illuminating physical spaces. These methods and interfaces reduce the cognitive burden on users and result in more efficient human-machine interfaces. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charging cycles.

[0006] In some embodiments, a method is described that is executed at a computer system communicating with a light source and an input device. In some embodiments, the method includes: detecting a request to illuminate a physical space via the input device; and in response to detecting the request to illuminate the physical space: providing illumination in a first manner via the light source based on determining that a first portion of the physical space is set as a light path; and abandoning the provision of illumination in the first manner based on determining that the first portion of the physical space is not set as a light path.

[0007] In some embodiments, a non-transitory computer-readable storage medium is described, which stores one or more programs configured to be executed by one or more processors of a computer system communicating with a light source and an input device. In some embodiments, the one or more programs include instructions for: detecting a request to illuminate a physical space via an input device; and in response to detecting the request to illuminate the physical space: providing illumination in a first manner via a light source based on determining that a first portion of the physical space is set as a light path; and abandoning the provision of illumination in the first manner based on determining that the first portion of the physical space is not set as a light path.

[0008] In some embodiments, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system communicating with a light source and an input device. In some embodiments, the one or more programs include instructions for: detecting a request to illuminate a physical space via an input device; and in response to detecting the request to illuminate the physical space: providing illumination in a first manner via a light source based on determining that a first portion of the physical space is configured as a light path; and abandoning the provision of illumination in the first manner based on determining that the first portion of the physical space is not configured as a light path.

[0009] In some embodiments, a computer system configured to communicate with a light source and an input device is described. In some embodiments, the computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some embodiments, the one or more programs include instructions for: detecting a request to illuminate a physical space via an input device; and in response to detecting the request to illuminate the physical space: providing illumination in a first manner via a light source based on determining that a first portion of the physical space is set as a light path; and abandoning the provision of illumination in the first manner based on determining that the first portion of the physical space is not set as a light path.

[0010] In some embodiments, a computer system configured to communicate with a light source and an input device is described. In some embodiments, the computer system includes components for performing each of the following steps: detecting a request to illuminate a physical space via the input device; and in response to detecting the request to illuminate the physical space: providing illumination in a first manner via the light source based on determining that a first portion of the physical space is set as a light path; and abandoning the provision of illumination in the first manner based on determining that the first portion of the physical space is not set as a light path.

[0011] In some embodiments, a computer program product is described. In some embodiments, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a light source and an input device. In some embodiments, the one or more programs include instructions for: detecting a request to illuminate a physical space via an input device; and in response to detecting the request to illuminate the physical space: providing illumination in a first manner via a light source based on determining that a first portion of the physical space is set as a light path; and abandoning the provision of illumination in the first manner based on determining that the first portion of the physical space is not set as a light path.

[0012] In some embodiments, a method is described that is executed at a computer system communicating with a light source and an input device. In some embodiments, the method includes: detecting, via the input device, a request to change the size of a physical window in a physical space; and, in response to detecting the request to change the size of the physical window in the physical space: providing illumination via the light source in a first manner based on determining that the request corresponds to a first size; and providing illumination via the light source in a second manner based on determining that the request corresponds to a second size different from the first size.

[0013] In some embodiments, a non-transitory computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system communicating with a light source and an input device. In some embodiments, the one or more programs include instructions for: detecting a request via the input device to change the size of a physical window in physical space; and in response to detecting the request to change the size of the physical window in physical space: providing illumination via the light source in a first manner according to determining that the request corresponds to a first size; and providing illumination via the light source in a second manner according to determining that the request corresponds to a second size different from the first size.

[0014] In some embodiments, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system communicating with a light source and an input device. In some embodiments, the one or more programs include instructions for: detecting a request via the input device to change the size of a physical window in physical space; and in response to detecting the request to change the size of the physical window in physical space: providing illumination via the light source in a first manner according to determining that the request corresponds to a first size; and providing illumination via the light source in a second manner according to determining that the request corresponds to a second size different from the first size.

[0015] In some embodiments, a computer system configured to communicate with a light source and an input device is described. In some embodiments, the computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some embodiments, the one or more programs include instructions for: detecting a request via an input device to change the size of a physical window in physical space; and in response to detecting the request to change the size of the physical window in physical space: providing illumination via a light source in a first manner based on determining that the request corresponds to a first size; and providing illumination via a light source in a second manner based on determining that the request corresponds to a second size different from the first size.

[0016] In some embodiments, a computer system configured to communicate with a light source and an input device is described. In some embodiments, the computer system includes components for performing each of the following steps: detecting a request to change the size of a physical window in physical space via the input device; and in response to detecting the request to change the size of the physical window in physical space: providing illumination via the light source in a first manner according to determining that the request corresponds to a first size; and providing illumination via the light source in a second manner according to determining that the request corresponds to a second size different from the first size.

[0017] In some embodiments, a computer program product is described. In some embodiments, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a light source and an input device. In some embodiments, the one or more programs include instructions for: detecting a request via the input device to change the size of a physical window in physical space; and in response to detecting the request to change the size of the physical window in physical space: providing illumination via the light source in a first manner according to determining that the request corresponds to a first size; and providing illumination via the light source in a second manner according to determining that the request corresponds to a second size different from the first size.

[0018] In some embodiments, a method is described that is performed at a computer system communicating with an external light source and an input device. In some embodiments, the method includes: detecting light of a corresponding color in a physical environment via the input device; and in response to detecting light of a corresponding color in the physical environment: providing illumination in a first manner via the external light source based on determining that the corresponding color is a first color; and providing illumination in a second manner via the external light source based on determining that the corresponding color is a second color different from the first color.

[0019] In some embodiments, a non-transitory computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with an external light source and an input device. In some embodiments, the one or more programs include instructions for: detecting light of a corresponding color in a physical environment via an input device; and in response to detecting light of a corresponding color in the physical environment: providing illumination in a first manner via an external light source based on determining that the corresponding color is a first color; and providing illumination in a second manner via an external light source based on determining that the corresponding color is a second color different from the first color.

[0020] In some embodiments, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system communicating with an external light source and an input device. In some embodiments, the one or more programs include instructions for: detecting light of a corresponding color in a physical environment via an input device; and in response to detecting light of a corresponding color in the physical environment: providing illumination in a first manner via an external light source based on determining that the corresponding color is a first color; and providing illumination in a second manner via an external light source based on determining that the corresponding color is a second color different from the first color.

[0021] In some embodiments, a computer system configured to communicate with an external light source and an input device is described. In some embodiments, the computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some embodiments, the one or more programs include instructions for: detecting light of a corresponding color in a physical environment via an input device; and in response to detecting light of a corresponding color in the physical environment: providing illumination in a first manner via an external light source based on determining that the corresponding color is a first color; and providing illumination in a second manner via an external light source based on determining that the corresponding color is a second color different from the first color.

[0022] In some embodiments, a computer system configured to communicate with an external light source and an input device is described. In some embodiments, the computer system includes components for performing each of the following steps: detecting light of a corresponding color in the physical environment via an input device; and in response to detecting light of a corresponding color in the physical environment: providing illumination in a first manner via an external light source based on determining that the corresponding color is a first color; and providing illumination in a second manner via an external light source based on determining that the corresponding color is a second color different from the first color.

[0023] In some embodiments, a computer program product is described. In some embodiments, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with an external light source and an input device. In some embodiments, the one or more programs include instructions for: detecting light of a corresponding color in a physical environment via an input device; and in response to detecting light of a corresponding color in the physical environment: providing illumination in a first manner via an external light source based on determining that the corresponding color is a first color; and providing illumination in a second manner via an external light source based on determining that the corresponding color is a second color different from the first color.

[0024] Executable instructions for performing these functions may optionally be included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0025] Therefore, faster and more efficient methods and interfaces are provided for illuminating physical spaces, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace other methods used for illuminating physical spaces. Attached Figure Description

[0026] To better understand the various described embodiments, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein the same reference numerals in all the drawings indicate corresponding parts.

[0027] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some implementation schemes.

[0028] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0029] Figure 2 Examples of portable multi-functional devices with touchscreens according to some implementation schemes are shown.

[0030] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0031] Figure 4A An exemplary user interface for a menu applied to a portable multi-functional device, according to some implementation schemes, is illustrated.

[0032] Figure 4BAn exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is illustrated according to some embodiments.

[0033] Figure 5A Examples of personal electronic devices according to some implementation schemes are shown.

[0034] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.

[0035] Figures 6A to 6B An exemplary environment for simulating light passing through a physical opening is illustrated.

[0036] Figure 7 This is a flowchart illustrating a method for simulating light passing through a physical opening.

[0037] Figures 8A to 8B An exemplary environment is shown to simulate different amounts of light passing through a physical opening.

[0038] Figure 9 This is a flowchart illustrating a method for simulating different amounts of light passing through a physical opening.

[0039] Figures 10A to 10B An exemplary environment is illustrated for adjusting one or more lighting conditions.

[0040] Figure 11 This is a flowchart illustrating a method for adjusting one or more lighting conditions. Detailed Implementation

[0041] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0042] There is a need for electronic devices that provide efficient methods and interfaces for illuminating physical spaces. For example, one or more light sources within the physical space could be used to simulate light passing through a window. Such technologies can reduce the cognitive burden on users expecting to illuminate a physical space, thereby increasing productivity. Furthermore, such technologies can reduce processor and battery power that would otherwise be wasted on redundant user input.

[0043] In the following text, Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B and Figures 5A to 5B A description is provided of an exemplary device for performing techniques for illuminating a physical space.

[0044] Figures 6A to 6B An exemplary environment for simulating light passing through a physical opening is illustrated. Figure 7 This is a flowchart illustrating a method for simulating light passing through a physical opening. Figures 6A to 6B The environment described below is used to illustrate the process, including Figure 7 The process in. Figures 8A to 8B An exemplary environment is shown to simulate different amounts of light passing through a physical opening. Figure 9 This is a flowchart illustrating a method for simulating different amounts of light passing through a physical opening. Figures 8A to 8B The environment described below is used to illustrate the process, including Figure 9 The process in. Figures 10A to 10B An exemplary environment is illustrated for adjusting one or more lighting conditions. Figure 11 This is a flowchart illustrating a method for adjusting one or more lighting conditions. Figures 10A to 10B The user interface in the document is used to illustrate the processes described below, including Figure 11 The process in.

[0045] The processes described below enhance device operability and make the user-device interface more efficient through various technologies (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), including providing improved visual feedback to users, reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, performing operations when a set of conditions is met without requiring further user input, and / or additional technologies. These technologies also reduce power consumption and extend device battery life by enabling users to use the device faster and more efficiently.

[0046] Furthermore, in a method described herein where one or more steps depend on the satisfaction of one or more conditions, it should be understood that the method may be repeated in multiple repetitions such that, during the repetitions, all conditions determining the steps in the method are satisfied in different repetitions of the method. For example, if the method requires performing a first step (if the conditions are satisfied) and a second step (if the conditions are not satisfied), those skilled in the art will know that the stated steps are repeated until both conditions are satisfied and not satisfied (in no particular order). Thus, a method described as having one or more steps depending on the satisfaction of one or more conditions can be rewritten as a method that repeats until each condition described in the method is satisfied. However, this does not require the system or computer-readable medium to declare that the system or computer-readable medium contains instructions for performing discretionary operations based on the satisfaction of the corresponding one or more conditions, and thus to determine whether possible conditions have been satisfied without explicitly repeating the steps of the method until all conditions determining the steps in the method are satisfied. Those skilled in the art will also understand that, similar to a method having discretionary steps, a system or computer-readable storage medium may repeat the steps of the method multiple times as needed to ensure that all discretionary steps have been performed.

[0047] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. In some embodiments, these terms are used to distinguish one element from another. For example, a first touch may be referred to as a second touch, and similarly, a second touch may be referred to as a first touch, without departing from the scope of the various described embodiments. In some embodiments, a first touch and a second touch are two separate references to the same touch. In some embodiments, both a first touch and a second touch are touches, but they are not the same touch.

[0048] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and in the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0049] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "in," or "in response to determination" or "in response to detection." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "in response to determination..." or "in response to detection of [the stated condition or event]."

[0050] This document describes implementations of electronic devices, user interfaces for such devices, and associated processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functionalities such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, the iPhone from Apple Inc., Cupertino, California. ® Devices, iPod Touch ® Devices and iPads ® Device. Alternatively, other portable electronic devices may be used, such as laptops or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). In some embodiments, the electronic device is a computer system that communicates with a display generating component (e.g., via wireless or wired communication). The display generating component is configured to provide visual output, such as display via a CRT display, via an LED display, or via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. As used herein, “display” content includes displaying content (e.g., video data rendered or decoded by display controller 156) by sending data (e.g., image data or video data) to an integrated or external display generating component via a wired or wireless connection to visually generate content.

[0051] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device may optionally include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0052] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0053] Various applications running on this device may optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device may optionally be adapted and / or changed for different applications, and / or within the respective applications. In this way, the common physical architecture of the device (such as the touch-sensitive surface) may optionally support various applications using a user interface that is intuitive and clear to the user.

[0054] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1A This is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as or called a “touch-sensitive display system.” Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 may optionally include one or more haptic output generators 167 for generating haptic outputs on device 100 (e.g., generating haptic outputs on a haptic surface such as the haptic display system 112 of device 100 or the touchpad 355 of device 300). These components may optionally communicate via one or more communication buses or signal lines 103.

[0055] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute (alternative) for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values ​​that includes at least four different values ​​and more typically hundreds of different values ​​(e.g., at least 256). The intensity of contact may optionally be determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface may optionally be used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus may optionally be used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or change of the contact area detected on the touch-sensitive surface, the capacitance and / or change of the touch-sensitive surface adjacent to the contact, and / or the resistance and / or change of the touch-sensitive surface adjacent to the contact may be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of the contact as an attribute of user input allows users to access additional device functionality that would otherwise be inaccessible to the user on a space-constrained, scaled-down device used (e.g., on a touch-sensitive display) to display an indication and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0056] As used in this specification and claims, the term "tactile output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with the touch-sensitive surface, which has been physically pressed (e.g., displaced) by the user's movement, does not move. As another example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or sensed by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of the user (e.g., "release click", "press click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception described by a typical (or common) user.

[0057] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.

[0058] Memory 102 may optionally include high-speed random access memory, and may also optionally include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 may optionally control access to memory 102 by other components of device 100.

[0059] Peripheral interface 118 can be used to couple input and output peripherals of the device to CPU 120 and memory 102. The one or more processors 120 run or execute various software programs (such as computer programs (e.g., including instructions)) and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0060] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, subscriber identity module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks (such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular phone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs))) and other devices. RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as via short-range communication radio components. Wireless communication may optionally use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution, Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n and / or IEEE 802.11ac), Internet Protocol Voice (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)) and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.

[0061] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and sends the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and sends the audio data to peripheral interface 118 for processing. Audio data may optionally be retrieved by peripheral interface 118 from memory 102 and / or RF circuitry 108 and / or sent to that memory and / or RF circuitry. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both outputs (e.g., a mono-ear headset or a binaural headset) and inputs (e.g., a microphone).

[0062] I / O subsystem 106 couples input / output peripherals (such as touchscreen 112 and other input control devices 116) on device 100 to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, depth camera controller 169, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / transmit electrical signals to the other input control device 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, joysticks, click dials, etc. In some embodiments, input controller 160 may optionally be coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing device such as mouse. One or more buttons (e.g., ... Figure 2 Optionally, 208 may include increase / decrease buttons for volume control of speaker 111 and / or microphone 113. These one or more buttons may optionally include a push-button (e.g., Figure 2(Ref. 206 in the original text). In some embodiments, the electronic device is a computer system that communicates with one or more input devices (e.g., via wireless communication, via wired communication). In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a touchpad, as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), such as for tracking user gestures (e.g., hand gestures and / or air gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system. In some implementations, air gestures are gestures detected without the user touching an input element that is part of the device (or independently of an input element that is part of the device) and based on the detected movement of a part of the user's body through the air (including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to the user's other hand, and / or movement of the user's fingers relative to another of the user's fingers or a part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tapping gesture that includes the hand moving a predetermined amount and / or rate in a predetermined pose, or a shaking gesture that includes a predetermined rate or amount of rotation of a part of the user's body)).

[0063] A rapid press of the push button may optionally disengage the touchscreen 112 from its lock or optionally initiate a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent No. 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of the push button (e.g., 206) may optionally power on or off the device 100. The functionality of one or more of these buttons may optionally be user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0064] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output may optionally include graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output may optionally correspond to user interface objects.

[0065] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.

[0066] Touchscreen 112 may optionally employ LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies may be used in other embodiments. Touchscreen 112 and display controller 156 may optionally use any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption of contact. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology, such as that used in the iPhone from Apple Inc. (Cupertino, California), is used. ® and iPod Touch ® The technology used.

[0067] In some embodiments of the touchscreen 112, the touch-sensitive display may optionally resemble a multi-touch-sensitive touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.

[0068] The touch-sensitive display in some embodiments of the touchscreen 112 is described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, “Multipoint Touch Surface Controller”; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, “Multipoint Touchscreen”; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, “Mode-Based Graphical User Interfaces For Touch Sensitive Input”. (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, “Virtual Input Device Placement On A Touch Screen User Interface”; (7) U.S. Patent Application No. 11 / 228,700, filed September 16, 2005, “Operation Of A Computer With A Touch Screen Interface”; (8) U.S. Patent Application No. 11 / 228,737, filed September 16, 2005, “Activating Virtual Keys Of ATouch-Screen Virtual Keyboard”; and (9) U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, “Multi-Functional Hand-Held Device”. The full text of all these applications is incorporated herein by reference.

[0069] The touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users may optionally use any suitable object or accessory, such as a stylus, finger, etc., to interact with the touchscreen 112. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positioning or commands for performing the user-desired actions.

[0070] In some embodiments, in addition to the touchscreen, device 100 may optionally include a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. The touchpad may optionally be a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0071] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 may optionally include a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0072] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also referred to as a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, facing away from a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the positioning of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensors within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.

[0073] The device 100 may optionally also include one or more depth camera sensors 175. Figure 1A A depth camera sensor is shown coupled to a depth camera controller 169 in I / O subsystem 106. Depth camera sensor 175 receives data from the environment to create a 3D model of an object (e.g., a face) within the scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with imaging module 143 (also referred to as a camera module), depth camera sensor 175 may optionally be used to determine depth maps of different portions of an image captured by imaging module 143. In some embodiments, the depth camera sensor is located at the front of device 100, such that user images with depth information can be optionally acquired for video conferencing while a user views other video conferencing participants on a touchscreen display, and selfies with depth map data can be captured. In some embodiments, depth camera sensor 175 is located at the rear of the device, or both the rear and front of device 100. In some embodiments, the positioning of depth camera sensor 175 can be changed by the user (e.g., by rotating a lens and sensor within the device housing), such that depth camera sensor 175 is used in conjunction with a touchscreen display for both video conferencing and still image and / or video image acquisition.

[0074] In some implementations, the depth map (e.g., a depth map image) contains information (e.g., values) relating to the distance of objects in the scene from the viewpoint (e.g., a camera, optical sensor, depth camera sensor). In one implementation of the depth map, each depth pixel defines the location of its corresponding two-dimensional pixel on the Z-axis of the viewpoint. In some implementations, the depth map is composed of pixels, where each pixel is defined by a value (e.g., 0 to 255). For example, a "0" value represents the pixel furthest from the viewpoint (e.g., a camera, optical sensor, depth camera sensor) in the "3D" scene, and a "255" value represents the pixel closest to the viewpoint in the "3D" scene. In other implementations, the depth map represents the distance between objects in the scene and the plane of the viewpoint. In some implementations, the depth map includes information about the relative depth of various features of the object of interest within the field of view of the depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears of a user's face). In some implementations, the depth map includes information that enables the device to determine the contour of the object of interest along the z-direction.

[0075] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1A A contact strength sensor is shown coupled to a strength sensor controller 159 in I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0076] The device 100 may optionally also include one or more proximity sensors 166. Figure 1AA proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 may be coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may be configured as described in the following U.S. patent applications: 11 / 241,839, entitled "Proximity Detector In Handheld Device"; 11 / 240,788, entitled "Proximity Detector In Handheld Device"; 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", the entire contents of which are incorporated herein by reference. In some implementations, when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call), the proximity sensor is turned off and the touchscreen 112 is disabled.

[0077] The device 100 may optionally also include one or more haptic output generators 167. Figure 1A A haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., a haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.

[0078] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral device interface 118 is shown. Alternatively, the accelerometer 168 may be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may be configured as described in the following U.S. Patent Publications: 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and 20060017692, entitled "Methods and Apparatuses for Operating a Portable DeviceBased On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. Device 100 may optionally include, in addition to the accelerometer 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation (e.g., portrait or landscape) of device 100.

[0079] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the figure. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, indicating what applications, views or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from various sensors and input control devices 116 of the device; and position information relating to the device's position and / or orientation.

[0080] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0081] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, Wireless LAN, etc.). In some embodiments, the external port is for use with an iPod. ® (Trademark of Apple Inc.) The same or similar and / or compatible multi-pin (e.g., 30-pin) connectors used in Apple Inc. devices.

[0082] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations can optionally be applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0083] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by the user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in the set of intensity thresholds (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).

[0084] The touch / motion module 130 optionally detects gesture input performed by the user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures can be detected optionally by detecting specific contact patterns. For example, detecting a finger tap gesture includes: detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes: detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.

[0085] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0086] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from applications, etc., to specify the graphics to be displayed, and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data to output to the display controller 156.

[0087] The haptic feedback module 133 includes various software components for generating instructions which are used by the haptic output generator 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0088] Optionally, the text input module 134, which is a component of the graphics module 132, provides a soft keyboard for entering text in various applications, such as contacts 137, email 140, IM 141, browser 147, and any other application that requires text input.

[0089] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to telephone 138 for use in location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0090] Application 136 may optionally include the following modules (or instruction sets) or subsets or supersets thereof: • Contacts module 137 (sometimes called address book or contact list); • Telephone module 138; • Video conferencing module 139; • Email client module 140; • Instant Messaging (IM) sending and receiving module 141; • Fitness support module 142; • Camera module 143 for still images and / or video images; • Image management module 144; • Video player module; • Music player module; • Browser module 147; • Calendar module 148; • Widget module 149, which may optionally include one or more of the following: weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5 and other widgets obtained by the user, as well as user-created widget 149-6; • Widget creator module 150 for creating user-created widgets 149-6; • Search module 151; • Video and music player module 152, which combines a video player module and a music player module; • Memo module 153; • Map module 154; and / or • Online video module 155.

[0091] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0092] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing module 139, email 140, or IM 141; and so on.

[0093] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify input telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As noted above, wireless communication may optionally use any of a variety of communication standards, protocols, and technologies.

[0094] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0095] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0096] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the instant messaging module 141 includes executable instructions for performing the following operations: entering a character sequence corresponding to an instant message, modifying previously entered characters, sending a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing received instant messages. In some embodiments, the sent and / or received instant messages may optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant messaging" refers to both telephone-based messages (e.g., messages delivered using SMS or MMS) and internet-based messages (e.g., messages delivered using XMPP, SIMPLE, or IMPS).

[0097] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for performing the following operations: creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (exercise equipment); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0098] In conjunction with the touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0099] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for performing operations such as arranging, modifying (e.g., editing) or otherwise manipulating, marking, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0100] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for performing the following operations: browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as linking to attachments and other files on web pages.

[0101] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0102] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 may be a micro-application (e.g., weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a user-created micro-application (e.g., user-created widget 149-6). In some embodiments, the widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the widget includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! widgets).

[0103] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget creator module 150 can optionally be used by the user to create widgets (e.g., turning a user-specified section of a webpage into a widget).

[0104] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for performing the following operations: searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0105] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play back recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0106] Combining the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the memo module 153 includes executable instructions for creating and managing memos, to-do items, etc., according to user instructions.

[0107] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 may optionally be used to receive, display, modify, and store maps and map-related data (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.

[0108] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions to: allow users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Additional descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are incorporated herein by reference in their entirety.

[0109] Each of the modules and applications described above corresponds to an executable set of instructions for performing one or more functions described above and the methods described in this patent application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A (e.g., video and music player module 152). In some embodiments, memory 102 may optionally store a subset of the aforementioned modules and data structures. Additionally, memory 102 may optionally store additional modules and data structures not described above.

[0110] In some implementations, device 100 is a device whose operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (such as push-buttons and dial pads) on device 100 can be optionally reduced.

[0111] A predefined set of functions unique to touchscreens and / or touchpads may optionally include navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, main desktop menu, or root menu. In such embodiments, a "menu button" is implemented using a touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0112] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any of the aforementioned applications 137 to 151, 155, 380 to 390).

[0113] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0114] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0115] Event monitor 171 receives event information from peripheral interface 118. The event information includes information about sub-events, such as a user touch on touch-sensitive display 112 as part of a multi-touch gesture. Peripheral interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, accelerometer 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.

[0116] In some implementations, the event monitor 171 sends requests to the peripheral device interface 118 at predetermined intervals. In response, the peripheral device interface 118 sends event information. In other implementations, the peripheral device interface 118 sends event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0117] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0118] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0119] Another aspect of the user interface associated with an application is a collection of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected may optionally be called the hit view, and the set of events identified as correct input may optionally be determined at least in part based on the hit view of the initial touch that initiates the touch-based gesture.

[0120] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0121] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0122] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0123] In some implementations, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another implementation, event classifier 170 is a separate module or part of another module (such as contact / motion module 130) stored in memory 102.

[0124] In some implementations, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other implementations, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some implementations, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 may optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0125] The corresponding event identifier 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event identifier 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which may optionally include sub-event delivery instructions).

[0126] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information may optionally also include the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device orientation).

[0127] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in events (e.g., 187-1 and / or 187-2) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on the displayed object, movement of the touch on the touch-sensitive display 112, and lifting the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0128] In some implementations, event definition 186 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0129] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0130] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0131] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0132] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from delivering (and deferred delivering) the sub-events to the corresponding hit view. In some implementations, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag retrieves the flag and performs a predefined process.

[0133] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined process.

[0134] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137, or video files stored in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the positioning of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and transmits that display information to graphics module 132 for display on a touch-sensitive display.

[0135] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0136] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate the multifunction device 100, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0137] Figure 2 A portable multi-functional device 100 with a touchscreen 112 is illustrated according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon may optionally not select the corresponding application when the gesture corresponding to selection is a tap.

[0138] Device 100 may optionally also include one or more physical buttons, such as a "main desktop" or menu button 204. As previously described, menu button 204 may optionally be used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0139] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a subscriber identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to: power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts speech input via microphone 113 for activating or deactivating certain functions. Device 100 may also optionally include one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic output for the user of device 100.

[0140] Figure 3 This is a block diagram of an exemplary multi-functional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 may also optionally include a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the reference above). Figure 1A The described tactile output generator 167), sensor 359 (e.g., optical sensor, accelerometer, proximity sensor, touch sensor and / or contact intensity sensor (similar to the one described above)) Figure 1AThe described contact strength sensor 165). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores information related to portable multifunction device 100. Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 may optionally store additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 may optionally store a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 ( Figure 1A The memory 102 may optionally not store these modules.

[0141] Figure 3 Each of the elements described above may optionally be stored in one or more memory devices of the previously mentioned memory devices. Each of the modules described above corresponds to a set of instructions for performing the functions described above. The modules or computer programs described above (e.g., instruction sets or including instructions) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules, and therefore various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 may optionally store a subset of the modules and data structures described above. In addition, memory 370 may optionally store additional modules and data structures not described above.

[0142] Now let’s turn our attention to the implementation of the user interface, which may be optionally implemented on, for example, a portable multifunction device 100.

[0143] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is illustrated. A similar user interface may optionally be implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof: • Signal strength indicator 402 for wireless communications (such as cellular signals and Wi-Fi signals); • Time 404; • Bluetooth indicator 405; • Battery status indicator 406; • Tray 408 with icons for frequently used applications, such as: ○ The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 for the number of missed calls or voicemails; ○ An icon 418 labeled "Mail" in the email client module 140, which optionally includes an indicator 410 for the number of unread emails; ○ The icon 420 of browser module 147, labeled "Browser"; and ○ The video and music player module 152 (also known as the iPod (a trademark of Apple Inc.) module 152) is marked with an icon 422 labeled "iPod"; and • Icons of other applications, such as: ○ Icon 424 of IM module 141 marked as "Message"; ○ The icon 426 labeled "Calendar" in calendar module 148; ○ The icon 428 of the image management module 144 that is labeled "Photo"; ○ The icon 430 of camera module 143, which is labeled "camera"; ○ Icon 432 of the online video module 155, labeled "Online Video"; ○ The icon 434 labeled "Stock Market" in the Stock Market widget 149-2; ○ Map module 154's icon 436, labeled "Map"; ○ The weather widget 149-1 with icon 438 labeled "weather"; ○ The alarm clock widget 149-4 has an icon 440 labeled "clock"; ○ Icon 442 of the fitness support module 142, which is labeled "fitness support"; ○ The icon 444 labeled "Memo" in the Memo module 153; and ○ A settings icon 446 labeled "Settings" is used to set the settings of the device 100 and its various applications 136.

[0144] It should be pointed out that, Figure 4AThe illustrated icon labels are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0145] Figure 4B An example is illustrated having a touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)). Figure 3 Devices such as tablets or touchpads (e.g., 355) Figure 3 An exemplary user interface on the device 300. The device 300 may also optionally include one or more contact intensity sensors (e.g., one or more sensors in sensor 359) for detecting the intensity of contact on the tactile surface 451 and / or one or more tactile output generators 357 for generating tactile output for the user of the device 300.

[0146] While some examples of inputs on a reference touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations, the device detects inputs on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450) corresponding to the main axis on the display (e.g., Figure 4B The main shaft of 453 in the middle (e.g., Figure 4B (452 in the example). According to these embodiments, the device detects the position corresponding to a specific location on the display (e.g., in...). Figure 4B In the diagram, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, when the touch-sensitive surface (e.g., ...) Figure 4B 451 in the middle) and the display of a multi-functional device (e.g., Figure 4B When the touch-sensitive surface (450) is separated, user input detected by the device on the touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods can be optionally used for other user interfaces described herein.

[0147] Additionally, while the examples below are given primarily with reference to finger input (e.g., finger touch, finger tap, finger swipe), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the touch). As another example, a tap gesture may optionally be replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0148] Figure 5A An exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include devices 100 and 300 (e.g., Figures 1A to 4B The device 500 may include some or all of the features described herein. In some embodiments, the device 500 has a touch-sensitive display 504, referred to below as a touchscreen 504. Alternatively, or in addition to the touchscreen 504, the device 500 may also have a display and a touch-sensitive surface. Similar to the cases of devices 100 and 300, in some embodiments, the touchscreen 504 (or touch-sensitive surface) may optionally include one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). The one or more intensity sensors of the touchscreen 504 (or touch-sensitive surface) may provide output data representing the intensity of the touch. The user interface of the device 500 may respond to touches based on the intensity of the touch, meaning that touches of different intensities may invoke different user interface operations on the device 500.

[0149] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying UserInterface Objects Corresponding to an Application,” published as WIPO Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between TouchInput to Display Output Relationships,” published as WIPO Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.

[0150] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508, if included, can be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0151] Figure 5B An exemplary personal electronic device 500 is depicted. In some embodiments, device 500 may include information about... Figure 1A , Figure 1B and Figure 3Some or all of the components described. Device 500 has a bus 512 that operatively couples I / O portion 514 to one or more computer processors 516 and memory 518. I / O portion 514 may be connected to display 504, which may have touch-sensitive component 522 and optionally have intensity sensor 524 (e.g., contact intensity sensor). Furthermore, I / O portion 514 may be connected to communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 may optionally be a rotatable input device. In some embodiments, input mechanism 508 may optionally be a button.

[0152] In some implementations, the input mechanism 508 may optionally be a microphone. The personal electronic device 500 may optionally include various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which may be operatively connected to the I / O section 514.

[0153] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, may cause those computer processors to perform, for example, the techniques described below, including processes 700, 900, and 1100. Figure 7 , Figure 9 and Figure 11 A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments, the storage medium is a transient computer-readable storage medium. In some embodiments, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices 500 are not limited to... Figure 5B It can be the components and configurations, or it can include other components or additional components with multiple configurations.

[0154] As used herein, the term "power indication" refers to the ability to indicate power in devices 100, 300, and / or 500 (…). Figure 1A , Figure 3 and Figures 5A to 5BA user-interactive graphical user interface object displayed on a screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) may each constitute a functional representation.

[0155] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of direct interaction with user interface elements on a touchscreen display) that enable direct interaction with user interface elements on the touchscreen display. Figure 1A The touch-sensitive display system 112 or Figure 4A In some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of ​​the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that conveys the user's expected interaction with the user interface (e.g., by indicating to the device the elements of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0156] As used in the specification and claims, the term "characteristic strength" of a contact refers to a characteristic of the contact based on one or more strengths of the contact. In some embodiments, the characteristic strength is based on multiple strength samples. The characteristic strength may optionally be based on a predefined number of strength samples or a set of strength samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted, before or after contact begins to move, before contact ends, before or after contact strength is detected to increase, and / or before or after contact strength is detected to decrease). The characteristic strength of the contact may optionally be based on one or more of the following: the maximum value of the contact strength, the mean value of the contact strength, the average value of the contact strength, the value at the top 10% of the contact strength, the half maximum value of the contact strength, or the 90% maximum value of the contact strength, etc. In some embodiments, the duration of the contact is used when determining the characteristic strength (e.g., when the characteristic strength is the average value of the contact strength over time). In some implementations, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether a user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a characteristic intensity not exceeding the first threshold results in a first action, contact with a characteristic intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a characteristic intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the characteristic intensity and one or more thresholds is used to determine whether one or more actions should be performed (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.

[0157] As used herein, "installed application" refers to a software application that has been downloaded to an electronic device (e.g., device 100, 300, and / or 500) and is ready to be launched on the device (e.g., become open). In some implementations, the downloaded application becomes an installed application using an installer that extracts program portions from the downloaded software package and integrates the extracted portions with the computer system's operating system.

[0158] As used herein, the terms "open application" or "running application" refer to a software application that maintains state information (e.g., as part of device / global internal state 157 and / or application internal state 192). An open or running application can optionally be any of the following types of applications: • The active application, which is currently displayed on the screen of the device that is using the application; • Background applications (or background processes) that are not currently displayed but whose one or more processes are being handled by one or more processors; and • Suspended or hibernating applications that are not running but have state information stored in memory (volatile and non-volatile, respectively) that can be used to resume application execution.

[0159] As used herein, the term "closed application" refers to a software application that does not retain state information (e.g., the state information of a closed application is not stored in the device's memory). Therefore, closing an application includes: stopping and / or removing the application's process and removing the application's state information from the device's memory. Generally, opening a second application while the first application is running does not close the first application. When the second application is displayed and the first application stops displaying, the first application becomes a background application.

[0160] Now let’s turn our attention to the implementation of user interfaces (“UIs”) on electronic devices, such as portable multifunction devices 100, 300 or 500, and the associated processes.

[0161] Figures 6A to 6B Exemplary environments for simulating light passing through physical openings, according to some embodiments, are illustrated. The user interface in these figures is used to illustrate the processes described below, including... Figure 7 The process in.

[0162] Figure 6A An example of physical space 600 is given, which is a room in a house. Physical space 600 includes several physical features, such as ceiling 602, left wall 604 (e.g., Figure 6A The left wall), the middle wall 606 (e.g., Figure 6A (Central wall), right wall 608 (e.g., Figure 6A The right-hand wall) and floor 610. In some embodiments, physical space 600 is a different physical space, such as physical space for offices, buildings, vehicles and / or barns, and / or includes spaces related to... Figure 6A The physical characteristics represented in the text are different or the same as those physical characteristics.

[0163] like Figure 6A As illustrated, window 606a and clock 606b are located on the intermediate wall 606. Window 606a is a window through which light passes from the physical environment outside physical space 600 into physical space 600. Figure 6A At this location, Sun 614 is part of the physical environment outside physical space 600, where light from Sun 614 is currently passing through window 606a and affecting the physical environment inside physical space 600. Additionally, as... Figure 6AAs illustrated, desk 618 is located on top of floor 610. Desk 618 represents an object within physical space 600 in the physical environment. In some embodiments, desk 618 can cause one or more shadows to appear in the physical environment within physical space 600 when a light source and / or the physical environment outside physical space 600 illuminates physical space 600.

[0164] like Figure 6A As illustrated, ceiling 602 includes a light source 612 physically connected to ceiling 602. Light source 612 includes light sources 612a-612d for illuminating physical space 600. In some embodiments, light source 612 communicates with one or more other devices (e.g., computer systems and / or processing devices). For example, light source 612 may communicate directly with one or more sensor devices, and / or indirectly with one or more sensor devices via one or more computing devices that communicate with one or more sensor devices. In some embodiments, one or more sensor devices detect changes and / or levels of one or more properties of the physical space, such as the amount of ambient light and / or movement in the environment. As another example, light source 612 may communicate with one or more processing devices that process sensor data, determine illumination levels, and / or process inputs that assist and / or command light source 612 to output illumination, as described in the examples described herein. For ease of explanation, various operations (e.g., outputting illumination, detecting inputs, and / or determining properties) are described below as being performed by light source 612. However, it should be recognized that one or more of the operations described below are performed by devices other than light source 612, such as one or more personal computing devices (e.g., telephones, tablets, laptops, desktop computers, head-mounted display (HMD) devices and / or wearable devices) and / or public devices (e.g., smart speakers, televisions, routers and / or hubs). In some embodiments, one or more computing devices and / or public devices include one or more features and / or components of devices 100, 300 and / or 500 as described above. Unless otherwise expressly stated, this description should not be construed as limiting the scope of such operations performed by a single device (e.g., light source 612) or a specific combination of devices.

[0165] Figure 6A An example is illustrated by a simulated window 604a located on the left wall 604. The simulated window 604a serves as a visual aid indicating the location of simulated light paths (e.g., windows, skylights, openings, and / or doors) within the physical space 600. (Regarding...) Figure 6AIn the described embodiments, light source 612 works in concert to create the visual effect of physical space 600, including a light path at the location of simulated window 604a. In some embodiments, the simulated light path is a designated area on the surface of physical space 600, through which light source 612 emits light as if passing (e.g., entering and / or leaving physical space 600). In some embodiments, the surface is a wall, roof, ceiling, the surface of an object (e.g., desk 618 and / or another object), and / or the floor.

[0166] exist Figure 6A At the location, light sources 612a and 612b (e.g., Figure 6A The leftmost light is illuminating physical space 600, so that the light is simulated to originate from simulation window 604a. Figure 6A At this point, light sources 612a and 612b are energized, indicated by the presence of shadow lines on them, while light sources 612c and 612d are de-energized, indicated by the absence of shadow lines on them. Figure 6A In this process, light sources 612a and 612b illuminate the physical space 600 by emitting light from the physical environment outside the physical space 600, which passes through the left wall 604. For example, Figure 6A Light sources 612a and 612b emit light that extends from the left wall 604 toward the center of the physical space 600, as if the physical window were located at... Figure 6A The simulated window 604a is located at the same position as the window on the left wall 604. Light will extend from the left wall 604 because the weather in the physical environment outside the physical space 600 is sunny and the time is 3:00 PM. It is worth noting that the light simulated to pass through the simulated window 604a will also change based on the movement of one or more objects outside the physical space 600, such as trees blocking sunlight and / or cars moving outside the left wall 604.

[0167] In some implementations, the user can control the location of the simulated window 604a. For example, in response to the detection of an air gesture pointing towards the left wall 604 (e.g., via a camera and / or multiple cameras), the light source 612 emits light in a manner that visually indicates the simulated window 604a on the left wall 604 at a specific location. In some implementations, in response to the detection of movement of the air gesture, the light source 612 emits light in one or more different ways that visually indicate the simulated window 604a on the left wall 604 at a changed location on the left wall 604 and / or on another surface, and / or visually indicate the movement of the simulated window 604a. For example, in response to the detection of gaze input pointing towards the left wall 604 (e.g., via a camera and / or multiple cameras), the light source 612 positions and / or moves the simulated window 604a to a specific location on the left wall 604 (and / or another surface of the physical space 600). As another example, in response to the detection of a voice command pointing towards the right wall 608 (e.g., “form a window on this wall”), the light source 612 generates a simulated window on the right wall 608 (e.g., as shown in the image). Figure 6B (Illustrated by simulated window 608a). As another example, in response to the detection of touch input pointing towards the right wall 608 (e.g., a user touching the wall with a single finger, hand, and / or drawing a shape on the surface) (e.g., via a camera and / or multiple cameras), light source 612 places simulated window 604a on the right wall 608. In some embodiments, the user can use different types of gestures (such as mouse clicks, voice commands, and / or gaze input) to control the location of the simulated window.

[0168] In some implementations, the user can control how the simulated window is shaped. For example, in response to detecting an air gesture, light source 612 can emit light that alters the shape of the simulated window, such as changing its shape to a circle, square, triangle, rectangle, and / or ellipse. In some implementations, the shape of the air gesture causes light source 612 to emit light according to the shape of the air gesture. For example, if the user's hand is shaped into a circle, light source 612 will emit light, causing the simulated window to change into a circular shape. In some implementations, the user can use different types of gestures (such as mouse clicks, voice commands, and / or gaze input) to control how the simulated window is shaped. In some implementations, the user can use one or more techniques similar to those discussed above regarding controlling the shape of simulated window 604a to control one or more other properties of the simulated window, such as size and / or simulated light intensity. Figure 6AThe input is detected at the location. In some embodiments, the input is detected as pointing towards the right wall 608. In some embodiments, the input is detected by one or more light sources in light source 612 and / or one or more computing devices and / or common devices communicating with light source 612. In some embodiments, the input is an air gesture, gaze input, mouse click, voice command, and / or touch input.

[0169] exist Figure 6B At that location, in response to the detection of input, the light sources 612 change the way they illuminate to create (and / or generate) an analog window 608a located on the right wall 608. Figure 6B At that point, in response to the detected input, the analog window 604a is no longer present on the left wall 604, and / or the light source 612 changes the way it illuminates to remove the analog window 604a. Figure 6B At this location, light sources 612c and 612d are currently illuminating physical space 600 (e.g., indicated by the shadow lines on light sources 612c and 612d), while light sources 612a and 612b are not currently illuminating physical space 600 (e.g., indicated by the absence of shadow lines on 612a and 612b). Therefore, in Figure 6B At this location, light source 612 adjusts the illumination of light sources 612a-612d to change the properties of the simulated window and / or generate another simulated window. In some embodiments, in response to the detection of input, light sources 612 change the way they illuminate to create (and / or generate) simulated window 608a, while also keeping simulated window 608a visible. In some embodiments, multiple simulated windows are created on multiple surfaces. For example, one simulated window exists on the ceiling, and another simulated window exists on the wall. In some embodiments, multiple simulated windows exist on the same surface. For example, two simulated windows are created on the same wall.

[0170] In some implementations, one or more properties of the analog window 604a are changed and / or maintained in response to the detection of input. In some implementations, in response to the detection of input, the light sources 612 change the way they illuminate to move the analog window 604a from the left wall 604 to the right wall 608, wherein the analog window 608a is the analog window 604a in the changed position.

[0171] It should be understood that in some of these implementations, the properties of the simulation window 604a are modified to generate the simulation window 608a to indicate that different amounts of light are passing through the simulation window 608a, because the position of the simulation window 608a is... Figure 6B The physical environment outside of physical space 600 is in the same state as simulation window 604a. Figure 6A The different positions they occupy. For example, in Figure 6BAt this location, simulated window 608a is located on the right wall 608. Because simulated window 608a is located in a different position than simulated window 604a, light source 612 illuminates physical space 600 in a different way. For example, due to the change in position (e.g., simulated window 604a is positioned to the left, and simulated window 608a is positioned to the right (e.g., due to the time of day (e.g., 3:00 PM), light sources 612a and 612b emit less light into physical space 600 through simulated window 604a than light sources 612c and 612d emit light through simulated window 608a because the sun is closer to the right)), light sources 612c and 612d illuminate physical space 600 in a way that is different from light sources 612a and 612b). Figure 6A Different methods are used to illuminate the physical space 600. To illustrate the differences, in Figure 6B A denser shadow line has been applied to light sources 612c and 612d than the shadow lines applied to light sources 612a and 612b, to indicate that light sources 612c and 612d are illuminating physical space 600 with more light, less light, different colors of light, and / or different characteristics (e.g., brightness, hue, coolness, softness, and / or shape).

[0172] exist Figure 6A and Figure 6B In the middle, the sun 614, visible through window 606a, is closer to the west (e.g., due to the current time of day (e.g., 3:00 PM)). Because the sun 614 is positioned closer to the right wall 608, therefore... Figure 6B The simulated light in the simulation window 608a needs to be more than that for the target Figure 6A The simulated light at window 604a is brighter. Therefore, in Figure 6B In order to generate simulation window 608a, light sources 612c and 612d use more light to illuminate physical space 600 compared to the illumination provided by light sources 612a and 612b for generating simulation window 604a.

[0173] In some implementations, the location of the simulated window is determined automatically by the light source 612 (e.g., without intervention input, such as input from the user). For example, the light source 612 may generate the simulated window based on the user's determined historical preferences and / or habits (such as the user's preference for softer and / or brighter light at a specific time of day and / or the user's preference for a specific type of light when performing a specific activity, such as talking to a friend and / or sleeping). In some implementations, the simulated window is generated based on light outside of different physical spaces (such as houses and / or buildings in a country different from physical space 600).

[0174] In some implementations, light source 612 generates a simulated window based on the position of a celestial body (e.g., the sun, moon, and / or another planet). For example, in response to (e.g., via a camera and / or multiple cameras) detecting that the sun is rising directly above the current position of physical space 600, light source 612 generates a simulated window on the ceiling of physical space 600. In some implementations, the celestial body is simulated within the simulated window. As another example, if the simulated window is created based on the sun's position, light source 612 emits much brighter light to simulate being able to see the sun through the window.

[0175] In some embodiments, light source 612 generates a simulated window covering the entire surface of physical space 600 (e.g., left wall 604, right wall 608, and / or ceiling 602). In some embodiments, light source 612 generates multiple simulated windows concurrently. For example, light source 612 generates simulated windows concurrently on left wall 604 and ceiling 602. As another example, light source 612 generates simulated windows on every available surface in physical space 600 (e.g., ceiling 602, left wall 604, middle wall 606, right wall 608, and / or floor 610). In some embodiments, light source 612 does not generate simulated windows within physical space 600.

[0176] In some embodiments, as part of generating simulation window 608a, light source 612 emits light to simulate weather effects (e.g., clouds, rain, and / or wind). In some embodiments, the simulated weather effects can mimic real-world weather conditions present at physical space 600. For example, based on (e.g., via cameras (e.g., multiple cameras) and / or through communication with a server and / or application (e.g., a weather application)) determining that the weather at the location of physical space 600 is a thunderstorm, light source 612 outputs dim lighting and emits light in a manner that makes rain shadows and shadows indicating occasional flashes of lightning visible in physical space 600. In some embodiments, light source 612 emits light to simulate weather effects according to user preferences. For example, in response to the detection of input (e.g., air gestures, voice commands, gaze input, and / or touch input), light source 612 outputs lighting that mimics sunny conditions to physical space 600, even though rain is detected at physical space 600.

[0177] In some implementations, as part of generating simulation windows 604a and / or 608a, light source 612 simulates shadows (e.g., shadows). For example, in Figure 6AIf a tree exists outside physical space 600 on the east side of physical space 600, then light source 612 will simulate the tree's shadow as extending from the left wall 604 towards the center of physical space 600. Because light source 612 simulates window 604a as being located on the leftmost wall of physical space 600, the simulated shadow will extend from the left wall 604 towards the center of physical space 600. Conversely, if... Figure 6B As illustrated, simulation window 608a is located on the right wall 608. Figure 6B Because light source 612 simulates light as extending from right wall 608 toward the center of physical space 600, the simulated shadow will extend from right wall 608 toward the center of physical space 600. For example, in Figure 6B If a tree located outside physical space 600 exists on the west side of physical space 600, light source 612 will simulate the tree's shadow as extending from right wall 608 toward the center of physical space 600. In some embodiments, light source 612 simulates shadows (e.g., multiple shadows) when outputting light based on current weather conditions. For example, if simulated windows 604a and / or 608a are located on ceiling 602, and it is determined, for example, via a camera (e.g., multiple cameras) and / or through communication with a server and / or application (e.g., a weather application), that clouds are passing over physical space 600, light source 612 will simulate the shadow of the clouds into physical space 600 (e.g., light source 612 simulates the shape, size, and / or movement of the cloud shadows).

[0178] In some implementations, when light source 612 generates simulation window 608a, light source 612 changes the way it illuminates physical space 600 based on the passage of time. For example, in Figure 6B As time passes (e.g., daytime transitions from afternoon to night), light source 612 gradually reduces the amount and / or brightness of light emitted from it to mimic changes in the environment outside the physical space. As an additional example, when light source 612 emits light to simulate a simulated window on the left wall 604 of physical space 600, in the morning, light source 612 outputs bright light (e.g., because the sun is rising and / or located in the east, the light will be brighter). As day progresses (e.g., and the Earth rotates), light source 612 reduces the brightness of the light (e.g., as physical space 600 and the spatial orientation of the sun change) and moves away from the left wall 604, light source 612 emits less bright light.

[0179] In some implementations, light source 612 illuminates physical space 600 based on its current location. For example, if light source 612 is located in Cupertino, California, it may illuminate physical space 600 based on the current weather and / or lighting conditions in the physical environment of Cupertino (e.g., if the light in Cupertino is bright and sunny, light source 612 will output bright and sunny light into physical space 600). In some implementations, light source 612 illuminates physical space 600 based on a location different from its current location. For example, if light source 612 is located in Cupertino, California, it may illuminate physical space 600 based on the current weather and / or lighting conditions in the physical environment of Juneau, Alaska (e.g., if the light in Juneau is dim, light source 612 will still output dim light into physical space 600 even though the light in Cupertino is bright and sunny). In some implementations, light source 612 illuminates physical space 600 based on a corresponding time at a location (e.g., different or the same location where light source 612 is located), which is different from the current time. In some implementations, a user can select the corresponding time via one or more inputs, such as past times and / or expected future times (e.g., based on one or more predictions of future weather predicted to occur at a location).

[0180] In some implementations, light source 612 may allow light to pass through a physical space rather than actively emitting light to illuminate the physical space 600. For example, in a scenario where light source 612 is a skylight, light source 612 may be modified (e.g., repositioning of a movable cover for one or more light sources 612 and / or addition and / or reduction of color filters on one or more light sources 612) to allow more or less light from the physical environment outside the physical space 600 into the physical space 600. In some implementations, other types of light sources besides those described herein are recognized and can be implemented using one or more techniques.

[0181] Figure 7 This is a flowchart illustrating a method (e.g., method 700) for simulating light passing through a physical opening according to some embodiments. Some operations in method 700 may be combined, the order of some operations may be changed, and some operations may be omitted.

[0182] As described below, Method 700 provides an intuitive way to simulate light passing through a physical opening. Method 700 reduces the cognitive load on the user, thus creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to simulate light passing through a physical opening faster and more efficiently saves power and increases the time between battery charging cycles.

[0183] In some implementations, method 700 involves a light source (e.g., 612) (e.g., a lighting device, a point light source, a spotlight, and / or one or more light sources) and an input device (e.g., as described above regarding...). Figures 6A to 6B Computer systems that communicate (e.g., as described above regarding) (e.g., cameras, depth sensors, microphones, hardware input mechanisms, rotatable input mechanisms, heart monitors, temperature sensors, and / or touch-sensitive surfaces) Figures 6A to 6B The computer system is executed at the location described. In some embodiments, the computer system is a telephone, watch, tablet, fitness tracker, wearable device, accessory, speaker, light, head-mounted display (HMD), and / or personal computing device. In some embodiments, the light source is not physically connected to and / or coupled to the computer system.

[0184] The computer system detects (702) the physical space (e.g., 600) (e.g., physical environment, room, office, and / or building) (e.g., as described above regarding) via an input device. Figures 6A to 6B The request to illuminate (as described) is included. In some embodiments, detecting the request includes detecting input corresponding to the request (e.g., tap gesture, long press gesture, verbal request and / or command, physical button press, pointing input and / or air gesture, and / or rotation of a physical input mechanism). In some embodiments, detecting the request includes receiving a message from another computer system indicating that the request was received by that other computer system.

[0185] In response to (704) the detection of a physical space (e.g., 600) (e.g., as mentioned above regarding Figures 6A to 6B The computer system provides (706) illumination via a light source (e.g., 612a-612d) in a first manner (e.g., the location, color, intensity, and / or size of the illumination) based on a request for illumination and the determination of a first part (e.g., 602, 604, 604a, 606, and / or 608) of a physical space (e.g., a region, section, and / or part) (e.g., a portion of a ceiling and / or wall) as a light path (e.g., a window, opening, and / or a location where light is set for emission, passage, and / or otherwise provided thereto). This illumination is based on the first part of the physical space being set as a light path. Figures 6A to 6B(As described). In some embodiments, providing illumination in a first manner includes activating a light source. In some embodiments, providing illumination in a first manner includes changing the light output by the light source. In some embodiments, providing illumination in a first manner includes transmitting a request to the light source to modify the light output by the light source. In some embodiments, illumination is provided in a first manner until a request to change or stop providing illumination in the first manner is received.

[0186] In response to (704) detecting a request to illuminate the physical space and determining that a first portion (e.g., 602, 604, 606, 606a and / or 608) of the physical space (e.g., 600) is not set as a light path, the computer system abandons (708) (e.g., via a light source) providing illumination in a first manner (e.g., in some embodiments, when illumination is not provided in another manner, or in some embodiments, when illumination is provided in a different manner) (e.g., as described above regarding Figures 6A to 6B (as described) (e.g., 604a and / or 608a). Depending on whether the first part is configured as a light path to selectively provide illumination in a first manner, the computer system can customize how the physical space is illuminated, thereby reducing the amount of input required to perform an operation and / or performing an operation without further user input when a set of conditions has already been met.

[0187] In some embodiments, a first portion (e.g., 602, 604, 606, 606a, and / or 608) of a physical space (e.g., 600) corresponds to (e.g., is, includes, or is formed therefrom) a first set of one or more physical structures (e.g., solid structures, e.g., roofs) (e.g., 602, 604, 606, 606a, and / or 608) that covers (e.g., covers the top of the physical space and not the sides of the physical space) at least a portion of the physical space (e.g., covers less than the entire physical space and / or covers the entire physical space). In some embodiments, the first set of one or more physical structures does not form at least a portion of the perimeter of the physical space. Making the first portion correspond to the first set of one or more physical structures covering at least a portion of the physical space allows a computer system to simulate a certain type of physical opening (e.g., skylight and / or window) even when there is no actual physical opening in the physical space, thereby reducing the amount of input required to perform an operation and / or performing an operation without further user input when a set of conditions has already been met.

[0188] In some embodiments, a second set of one or more physical structures (e.g., solid structures) (e.g., walls and / or partitions) (e.g., 602, 604, 606, 606a and / or 608) forming at least a portion of the perimeter of a physical space (e.g., 608) is not configured as an optical path. In some embodiments, the second set of one or more physical structures surrounds the perimeter of the physical space. In some embodiments, the second set of one or more physical structures surrounds a portion of the perimeter of the physical space. In some embodiments, the second set of one or more physical structures is configured as an optical path. In some embodiments, both the first set of one or more physical structures and the second set of one or more physical structures are configured as optical paths. The fact that the second set of one or more physical structures forming at least a portion of the perimeter of the physical space is not configured as an optical path allows the computer system to create an appearance of the physical space, thereby reducing the amount of input required to perform operations and / or performing operations when a set of conditions has already been met without requiring further user input.

[0189] In some embodiments, the first portion (e.g., 602, 604, 606, 606a, and / or 608) corresponds to (e.g., is, includes) a third set (e.g., 602, 604, 606, 606a, and / or 608) of one or more structures forming at least a portion of the perimeter (e.g., walls, room dividers, and / or partitions) of a physical space (e.g., 600) (e.g., walls and / or windows). In some embodiments, the third set of one or more physical structures surrounds the perimeter of the physical space. In some embodiments, the third set of one or more physical structures surrounds a portion of the perimeter of the physical space. Having the first portion correspond to a third set of one or more structures forming at least a portion of the perimeter of the physical space allows the computer system to simulate windows on a wall that are not actually physical windows on a wall or specific portions of a wall in the physical space, thereby reducing the amount of input required to perform an operation and / or performing an operation without further user input when a set of conditions has already been met.

[0190] In some implementations, providing lighting in a first manner includes providing weather events (e.g., as described above) via a light source (e.g., 612). Figures 6A to 6BThe simulation (e.g., imitation, copying, duplication, and / or reproduction) of weather events described (e.g., clouds, rain, wind, tornadoes, hail, hurricanes, and / or storms) (e.g., weather effects and / or weather elements). In some embodiments, the simulation of a weather event corresponds to past, present, and / or future weather at a location corresponding to a physical space (e.g., weather outside the physical space but at the same location). In some embodiments, the simulation of a weather event corresponds to past, present, and / or future weather at another physical space outside the physical space. In some embodiments, the simulation of a weather event corresponds to past, present, and / or future weather at another location different from the location corresponding to the physical space. In some embodiments, in response to detecting a request to illuminate the physical space, the computer system, via a set of one or more playback devices, causes audio mimicking the weather event to be output when the simulation of the weather event is provided. Selectively providing the simulation of weather events allows the computer system to inform the user of the weather event and / or provide a specific atmosphere, thereby reducing the amount of input required to perform an operation, providing improved feedback, and / or performing an operation without further user input when a set of conditions has already been met.

[0191] In some implementations, before (e.g., and / or simultaneously) a request to illuminate a physical space (e.g., 600) is detected, the computer system detects, via an input device, input corresponding to a selection of a first part of the physical space (e.g., one or more walls, a portion of a wall, a window, a door, a roof, and / or a floor) (e.g., inputs such as swipe input, tap input, voice commands, gaze, and / or air gestures). Figures 6A to 6B (As described above). In some implementations, in response to detecting input corresponding to the selection of a first portion of the physical space, the computer system sets (e.g., allocates, designates, establishes, limits, and / or assigns) the first portion of the physical space as an optical path (e.g., as described above regarding...). Figures 6A to 6B (As described). In some embodiments, a first portion of the physical space is set as a light path for a predetermined time period (e.g., 1 second to 60 seconds). In some embodiments, a first portion of the physical space is set as a light path for an indeterminate time period (e.g., until an input corresponding to a selection of another portion is detected and / or the first portion of the physical space is set as a light path). In some embodiments, the first portion of the physical space corresponds to two or more distinct discrete regions of the physical space. Setting the first portion of the physical space as a light path in response to detecting an input corresponding to a selection of the first portion of the physical space allows the user to change how the physical space is illuminated via the input, thereby providing an additional input mechanism without cluttering the user interface, and performing an operation when a set of conditions has been met without requiring further user input.

[0192] In some implementations, the input corresponding to the selection of a first portion of the physical space (e.g., 600) includes (and / or) air gestures (e.g., air taps, air swipes, tapping the index finger and thumb and / or hand rotations) (e.g., as described above regarding...). Figures 6A to 6B (As described). The input corresponding to the selection of the first part includes air gestures, allowing the user to make gestures in a certain direction to indicate where the light path is formed, instead of requiring verbal input and / or other types of input to define the area, thereby providing the user with improved feedback, reducing the amount of input required to perform the operation, and / or performing the operation without further user input when a set of conditions has already been met.

[0193] In some implementations, the first part (e.g., 602, 604, 606, 606a and / or 608) is set as an optical path (e.g., automatically and / or without detecting user input) in the absence of user input. Figures 6A to 6B (As described). Setting the first part as a light path without detecting input from the user allows the computer system to select a location in the physical space that is meaningful for the current illumination target, rather than requiring the user to provide one or more inputs to explicitly indicate where to add the light path, thereby providing improved feedback to the user, reducing the amount of input required to perform an operation, and / or performing an operation without further user input when a set of conditions has already been met.

[0194] In some implementations, this is combined with (e.g., after and / or in response to) the detection of a request to illuminate the physical space (e.g., as described above regarding...). Figures 6A to 6BAs described, and based on determining that a celestial body (e.g., 614) (e.g., the sun, moon, and / or one or more stars) is located at a first position (e.g., relative to physical space (e.g., and / or the computer system) (e.g., the celestial body is to the north, east, west, and / or south of physical space) (e.g., the celestial body is at 10 degrees, 90 degrees, and / or 120 degrees relative to physical space)), the computer system sets up a second portion of the physical space as a light path (e.g., 602, 604, 606, 606a, and / or 608). In some embodiments, in conjunction with detecting a request to illuminate the physical space and based on determining that the celestial body is located at the first position, the computer system provides illumination via a light source in a second manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., based on the second portion of the physical space being set up as a light path) (e.g., 602, 604, 606, 606a, and / or 608). In some embodiments, in conjunction with detecting a request to illuminate the physical space and determining that the celestial body is located at a first location, the computer system provides illumination via a light source in a second manner without setting a second portion of the physical space as a light path. In some embodiments, in conjunction with detecting a request to illuminate the physical space and determining that the celestial body (e.g., 614) is located at a second location different from the first location (e.g., 602, 604, 606, 606a and / or 608) (e.g., relative to the physical space (e.g., and / or the computer system) (e.g., the celestial body is to the north, east, west and / or south of the physical space)), the computer system sets a third portion of the physical space as a light path, wherein the third portion is different from the second portion (e.g., 602, 604, 606, 606a and / or 608). In some embodiments, in conjunction with detecting a request to illuminate the physical space and determining that the celestial body is located at a second location different from the first location, the computer system provides illumination via a light source in a third manner different from the second manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., based on a third portion of the physical space being configured as a light path) (e.g., 602, 604, 606, 606a, and / or 608). In some embodiments, in conjunction with detecting a request to illuminate the physical space and determining that the celestial body is located at a second location, the computer system provides illumination via a light source in a third manner without configuring the third portion of the physical space as a light path. In some embodiments, after providing illumination in a second manner when the celestial body is located at the first location, the computer system provides illumination via a light source in a third manner when the celestial body is located at the second location. In some embodiments, after providing illumination in a second manner when the celestial body is located at the first location, the computer system configures the third portion of the physical space as a light path when the celestial body is located at the second location. In some embodiments, the third region overlaps with the second region.In some implementations, the third region does not overlap with the second region. In some implementations, the third region and the second region are located on opposite sides of the physical space. In some implementations, the third region and the second region are located on the same side of the physical space. Providing illumination in different ways according to the position of celestial bodies allows the computer system to adjust the illumination as the celestial bodies move relative to a certain position in the sky, thereby providing improved feedback to the user, reducing the amount of input required to perform an operation, and / or performing an operation without further user input when a set of conditions has already been met.

[0195] In some embodiments, providing illumination via a light source in a first manner includes: simulating (e.g., mimicking, imitating, and / or simulating) light as passing through a first portion (e.g., 602, 604, 606, 606a, and / or 608) of a physical space (e.g., 600) (e.g., light is simulated to enter and / or leave the physical space through the first portion) (e.g., 602, 604, 606, 606a, and / or 608); and abandoning the simulation of light passing through a fourth portion (e.g., 602, 604, 606, 606a, and / or 608) of the physical space (e.g., 600). In some embodiments, the fourth portion differs from the first portion (e.g., light is not simulated to enter and / or leave the physical space through a third corresponding portion) (e.g., 602, 604, 606, 606a, and / or 608). In some implementations, providing illumination via a light source in a first manner includes simulating light passing through a first part and a fourth part of the physical space. Simulating light passing through the first part rather than the fourth part allows the computer system to create different lighting effects depending on where the light path is positioned, thereby performing operations when a set of conditions is already met without requiring further user input.

[0196] In some embodiments, providing illumination via a light source in a first manner includes simulating (e.g., imitating, mimicking, emulating, and / or mimicking) a beam of light (e.g., a beam, a strip, and / or a flash) as if passing through (e.g., originating from, emitting from, or diffusing from) a first portion of physical space (e.g., light is simulated as entering and / or leaving physical space via a first portion of physical space) (e.g., 602, 604, 606, 606a, and / or 608) (e.g., as described above regarding...). Figures 6A to 6B(As described). In some embodiments, the beam pulses. In some embodiments, the beam tracks the movement of objects not located in physical space (e.g., celestial bodies). In some embodiments, the beam comprises two or more colors. In some embodiments, the beam is of one color. Simulating the beam as traversing a first portion of physical space allows a computer system to simulate light passing through an optical path even when the actual light does not pass through that first portion, thereby performing operations when a set of conditions is already met without requiring further user input.

[0197] In some implementations, in response to a detected request to illuminate a physical space (e.g., 600) and based on the determination that a fifth portion of the physical space (e.g., 602, 604, 606, 606a, and / or 608) is configured as a light path (e.g., light is simulated to enter and / or leave the physical space via a third portion) (e.g., and the first portion of the physical space is not configured as a light path), the computer system provides illumination via a light source (e.g., 612) in a fourth manner different from the first manner (e.g., the position, color, intensity, and / or size of the illumination), wherein the fifth portion differs from the first portion (e.g., 602, 604, 606, 606a, and / or 608) (e.g., as described above regarding...). Figures 6A to 6B (As described). Providing illumination in different ways when different parts of the physical space are set up as light paths allows the computer system to adapt the illumination based on where the light paths are set up, thereby providing improved feedback to the user, reducing the amount of input required to perform an operation, and / or performing an operation without further user input when a set of conditions has already been met.

[0198] In some implementations, in response to a detected request to illuminate a physical space (e.g., 600) and based on the determination that a sixth portion (e.g., 602, 604, 606, 606a, and / or 608) and a seventh portion of the physical space are configured as light paths (e.g., light is simulated to enter and / or leave the physical space via the sixth and seventh portions), the computer system provides illumination via a light source in a fifth manner different from the first manner (e.g., based on multiple light paths, including the sixth and seventh portions), wherein the seventh portion is separate from the sixth portion (e.g., the physical space includes another area between the seventh and sixth portions) (e.g., as described above regarding...). Figures 6A to 6B(As described). In some implementations, the sixth portion of the physical space does not overlap with or adjoin the seventh portion of the physical space. Providing illumination in different ways when multiple portions of the physical space are each configured as light paths allows the computer system to generate complex lighting with multiple virtual sources without the need for actual physical windows and / or other physical mechanisms to allow light to pass through, thereby providing improved feedback to the user, reducing the amount of input required to perform an operation, and / or performing an operation without further user input when a set of conditions has already been met.

[0199] In some implementations, in response to detecting a request to illuminate a physical space (e.g., 600) and based on determination (e.g., a light source and / or a computer system) that one or more light paths are not being used for illumination (e.g., the physical space may include a portion of the physical space that is set as part of a light path), the computer system illuminates the physical space via a light source in a sixth manner, different from the first manner (e.g., as described above regarding...). Figures 6A to 6B Illumination is provided in a sixth manner (e.g., the location, color, intensity, and / or size of the light source). In some embodiments, providing illumination in a sixth manner includes providing illumination in the physical space generally and / or not in a particular direction via a light source. In some embodiments, providing illumination in a sixth manner includes providing illumination in a particular direction (e.g., downward and / or in a direction parallel to the light source) via a light source. In some embodiments, providing illumination in a first manner is based on one or more objects within the physical space (e.g., the illumination takes one or more objects into account, such that simulated shadows are created when objects are present). In some embodiments, providing illumination in a sixth manner is not based on objects within the physical space. Providing illumination in different ways when the current setup is not using a light path allows users to establish different scenarios when different types of lighting are to be used, thereby providing improved feedback to the user and / or performing actions when a set of conditions has already been met without further user input.

[0200] In some implementations, in response to detecting a request to illuminate a physical space (e.g., 600) and based on determining that the physical space is not part of a light path (e.g., 602, 604, 606, 606a, and / or 608), the computer system provides illumination via a light source in a seventh manner different from the first manner (e.g., the position, color, intensity, and / or size of the illumination) (and / or abandons providing illumination in the first manner) (e.g., as described above regarding...). Figures 6A to 6B(As described). In some embodiments, in response to detecting a request to illuminate the physical space and based on determining that the physical space is not part of the light path, the computer system stops and / or abandons providing illumination via a light source. In some embodiments, providing illumination in a seventh manner includes providing illumination in the physical space generally and / or not in a particular direction via a light source. In some embodiments, providing illumination in a seventh manner includes providing illumination in a particular direction (e.g., downward and / or in a direction parallel to the light source) via a light source. In some embodiments, providing illumination in a seventh manner is not based on objects within the physical space. Providing illumination differently when the physical space is not part of the light path allows the user to establish different scenarios when different types of lighting are to be used, thereby providing improved feedback to the user and / or performing operations when a set of conditions has been met without further user input.

[0201] In some embodiments, after illumination is provided in a first manner via a light source (e.g., 612) (e.g., and / or simultaneously), the computer system detects, via an input device, an input corresponding to a request to set an eighth portion (e.g., 602, 604, 606, 606a, and / or 608) of a physical space (e.g., walls, roof, ceiling, and / or floor of the physical space) as a light path (e.g., light is simulated to enter and / or leave the physical space through the path). In some embodiments, in response to detecting an input corresponding to a request to set an eighth portion of the physical space as a light path, the computer system provides illumination via the light source in an eighth manner different from the first manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., and stops providing illumination in the first manner) (e.g., based on the first and / or eighth portions of the physical space being set as light paths) (e.g., 602, 604, 606, 606a, and / or 608). In response to the detection of input corresponding to the request to set the eighth part of the physical space as a light path, the system provides illumination in a different manner, allows users to add new light paths to change the illumination, thereby providing improved feedback to the user and / or performing operations when a set of conditions has been met without requiring further user input.

[0202] In some embodiments, providing illumination via a light source in an eighth manner includes simulating (e.g., mimicking, copying, replicating, and / or reproducing) light as it traverses (e.g., originates from, emits from, and / or diffuses from) a first portion (e.g., 602, 604, 606, 606a, and / or 608) of physical space and an eighth portion of physical space (e.g., 600). In some embodiments, equal amounts of illumination are simulated as traversing the first and eighth portions of physical space. In some embodiments, more or less illumination is simulated as traversing the first portion of physical space compared to the eighth portion. In some embodiments, illumination of the same color is simulated as traversing the first and eighth portions of physical space. In some embodiments, illumination of different colors is simulated as traversing the first and eighth portions of physical space. Simulating light as traversing previous light paths in addition to new light paths allows computer systems to create illumination based on the concept of virtual light paths, thereby providing improved feedback to the user, reducing the amount of input required to perform an operation, and / or performing an operation without further user input when a set of conditions has already been met.

[0203] In some implementations, providing illumination via a light source in an eighth manner includes simulating (e.g., imitating, copying, replicating, and / or reproducing) light as an eighth part (e.g., 602, 604, 606, 606a, and / or 608) of physical space rather than a first part of physical space (e.g., 600).

[0204] In some implementations, after illumination is provided in a first manner via a light source (e.g., 612) (e.g., and / or simultaneously), the computer system detects that a timeout (e.g., 0.1 minutes to 360 minutes) has expired (e.g., as described above regarding...). Figure 6B (As described above). In some implementations, in response to the detection of time expiration (e.g., and in the absence of detected user input), the computer system provides illumination via a light source (e.g., 612) in a ninth manner different from the first manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., as described above regarding...). Figure 6B(As described) (e.g., light has a different set of characteristics (e.g., brightness, hue, chroma, and / or pulsation frequency when the light source provides illumination in a ninth manner compared to when it provides illumination in a first manner)). In some embodiments, the light source continues to provide illumination in a first manner after a detected timeout. In some embodiments, the light source outputs illumination in a ninth manner in response to a detected input. Providing illumination in different manners over time allows the illumination to be dynamic and / or change over time without requiring different user inputs to change the illumination, thereby providing improved feedback to the user, reducing the amount of input required to perform an operation, and / or performing an operation when a set of conditions has been met without requiring further user input.

[0205] In some implementations, providing illumination in a first manner, based on determining that a first portion (e.g., 602, 604, 606, 606a, and / or 608) of a physical space (e.g., located at a first position within the physical space), includes forming a first shadow (e.g., as described above regarding...). Figure 6B (As described above). In some embodiments, the first shadow is based on one or more objects located in another physical space and / or outside the physical space. In some embodiments, providing illumination in a first manner, based on determining that a first portion of the physical space is located at a second location within the physical space (e.g., situated within the physical space), includes forming a second shadow different from the shadow (e.g., in the absence of a first shadow) (e.g., as described above regarding...). Figure 6B (As described). In some embodiments, the second shadow is based on one or more objects located in another physical space and / or outside the physical space. In some embodiments, the second location is different from (and / or separate from) the first location (e.g., as described above regarding...). Figure 6B (As described). In some implementations, the first shadow and / or the second shadow are moved based on the movement of one or more objects in the physical space and / or another physical space outside the physical space. Simulating different shadows based on the location of the first part allows lighting to take into account the physical space and one or more objects in the physical space, thereby providing improved feedback to the user and / or performing actions when a set of conditions has been met without requiring further user input.

[0206] In some implementations, lighting is provided in a first manner based on the current (e.g., present) location of the computer system (e.g., different current locations result in different ways of providing lighting) (e.g., as described above regarding...). Figure 6B(As described). In some embodiments, lighting is provided in a first manner based on the previous and / or future location of the computer system. In some embodiments, the current location of the computer system is within a physical space. In some embodiments, the current location of the computer system is outside a physical space. Providing lighting based on the current location of the computer system allows the computer system to reflect the current situation (e.g., even when the physical space may not physically include certain light paths to reflect such a current situation), thereby providing improved feedback to the user and / or performing actions when a set of conditions has been met without requiring further user input.

[0207] In some implementations, lighting is provided in a first manner based on another location different from the current location (e.g., present) of the computer system (e.g., as described above). Figure 6B (As described). In some implementations, the other location is geographically different from the physical space. Providing lighting based on another location different from the computer system's current location allows the computer system to use lighting to artificially create the appearance of the physical space being in a different geographic location, thereby providing improved feedback to the user and / or performing actions without further user input when a set of conditions has already been met.

[0208] In some embodiments, providing illumination via a light source in a first manner includes: outputting illumination (e.g., as described above) via a first light source (e.g., 612a-612d) (e.g., an external light source, a lighting device, a point light source, a spotlight, and / or one or more light sources). Figures 6A to 6B As described above), and illumination output via a second light source (e.g., 612a-612d) different from (and / or separate from) the first light source (e.g., external light source, lighting device, point light source, spotlight and / or one or more light sources) (e.g., as described above regarding) Figures 6A to 6B (As described). In some embodiments, the first light source and the second light source are the same type of light source. In some embodiments, the first light source and the second light source are different types of light sources. In some embodiments, the first light source and the second light source output the same amount and / or type of illumination. In some embodiments, the first light source and the second light source output different amounts and / or types of illumination. In some embodiments, the first light source and the second light source output different colors of light. In some embodiments, the first light source and the second light source output the same color of light.

[0209] It should be noted that the above text regarding method 700 (for example, Figure 7The details of the process described herein also apply in a similar manner to the other methods described herein. For example, method 700 may optionally include one or more characteristics of the various methods described herein. For example, one or more techniques described above with respect to method 700 may be used to illuminate a physical space, wherein one or more techniques described herein with respect to method 900 may be used to select the manner of illuminating the physical space based on the selected window size. For the sake of brevity, these details will not be repeated below.

[0210] Figures 8A to 8B Exemplary environments for simulating varying amounts of light passing through physical openings, according to some embodiments, are illustrated. The user interface in these figures is used to illustrate the processes described below, including... Figure 9 The process in.

[0211] Figure 8A Example of physical space 600, which includes as described above. Figures 6A to 6B One or more features and / or components described include ceiling 602, intermediate wall 606, and desk 618. (Example) Figure 8A As illustrated, the central wall 606 includes a window 606a and a clock 606b. Furthermore, the clock 606b displays the current time as 3:00 PM, and the sun 614 is visible through the window 606a. The window 606a is a window through which light enters the physical space 600 from the external physical environment. Figure 8A At this location, the sun 614 is part of the physical environment outside physical space 600, where light from the sun 614 is currently passing through window 606a and affecting the physical environment inside physical space 600. In some embodiments, physical space 600 is the physical space of an area within another type of building, such as a hotel, office, and / or enterprise.

[0212] like Figures 8A to 8BAs illustrated, ceiling 602 includes a light source 612 physically connected to ceiling 602. Light source 612 includes light sources 612a-612d for illuminating physical space 600. In some embodiments, light source 612 communicates with one or more other devices (e.g., computer systems and / or processing devices). For example, light source 612 may communicate directly with one or more sensor devices, and / or indirectly with one or more sensor devices via one or more computing devices that communicate with one or more sensor devices. In some embodiments, one or more sensor devices detect changes and / or levels of one or more properties of the physical space, such as the amount of ambient light and / or movement in the environment. As another example, light source 612 may communicate with one or more processing devices that process sensor data, determine illumination levels, and / or process inputs that assist and / or command light source 612 to emit light, as described herein. For ease of explanation, various operations (e.g., outputting illumination, detecting inputs, and / or determining properties) are described below as being performed by light source 612. However, it should be recognized that one or more of the operations described below are performed by devices other than light source 612, such as one or more personal computing devices (e.g., telephones, tablets, laptops, desktop computers, head-mounted display (HMD) devices and / or wearable devices) and / or public devices (e.g., smart speakers, televisions, routers and / or hubs). In some embodiments, one or more computing devices and / or public devices include one or more features and / or components of devices 100, 300 and / or 500 as described above. Unless otherwise expressly stated, this description should not be construed as limiting the scope of such operations performed by a single device (e.g., light source 612) or a specific combination of devices.

[0213] To illustrate that the light source is energized (e.g., powered on, woken up, emitting a certain amount of light, and / or in an active state), the light source 612 includes diagonal shaded lines. Figure 8A At this location, light source 612 detects input that defines the size of window 606a. Examples include verbal input (e.g., “Double the size of my window”) and / or air gestures (e.g., the user tracing a shape in the air with their finger that defines the size of window 606a). In some embodiments, another input defines the size of window 606a (such as gaze input, mouse click, and / or touch input).

[0214] like Figure 8B As illustrated, in response to Figure 8A Input is detected at the location, and the light source 612 generates an analog window 800 corresponding to the input (e.g., and / or a captured image). Figure 8B In the image, light source 612 has a horizontally shaded appearance to indicate that it is in contrast to... Figure 8A The light source 612 emits light in a different way. Therefore, in Figure 8B At that location, in response to the detection of input, the light sources 612 have changed the way they emit light. Figure 8B At a location where light source 612 emits light in a manner that makes the simulated window 800 appear within physical space 600, the simulated window 800 is positioned around window 606a, and the generation of the simulated window 800 makes it appear as if more light is passing through window 606a from the physical environment outside physical space 600. In some embodiments, with... Figure 8A Compared to the light emitted by light source 612, light source 612 in Figure 8B This will emit more light into physical space 600. In some implementations, in Figure 8B At that location, light source 612 emits light to mimic the effect of light emitted from window 606a compared to window 606a at that location. Figure 8B In cases where the amount of light passing through window 606a is large, the light source 612 may allow light to pass through the physical space rather than actively emitting light to illuminate the physical space 600. For example, in a scenario where the light source 612 is a skylight, the light source 612 may be modified (e.g., by changing the positioning of a movable cover for one or more of the light sources 612 and / or by increasing and / or decreasing the color filter on one or more of the light sources 612) to allow more or less light from the physical environment outside the physical space 600 into the physical space 600. In some embodiments, other types of light sources besides those described herein are recognized and can be implemented using one or more techniques.

[0215] In some embodiments, a window is detected in response to capturing an image of the physical space. In some embodiments, the computer system uses a camera communicating with the computer system to capture an image of the physical space including the window. In some embodiments, the computer system is a personal computer system, smartphone, smartwatch, wearable device, tablet, laptop, fitness tracker, head-mounted display (HMD) device, and / or desktop computer. In some embodiments, the image is captured automatically without a request from the user. In some embodiments, the image is captured manually (e.g., via input) and / or with a request from the user. In some embodiments, the level of simulated light simulating the window is based on the captured image. For example, if the computer system captures an image of a window providing bright light, the computer system determines that the light from the window is bright. In response to determining the brightness level of the light from the window, light source 612 simulates bright light to match the bright light from the window. In some embodiments, windows of the same size are illuminated with different amounts of simulated light based on external factors such as location, weather, and / or time of day. For example, light source 612 simulates bright light on the window in the morning and dim light on the same window at night. In another example, the light source simulates illumination with a low brightness level on a small window (e.g., to replicate the expected amount of light that might come from a smaller window) and illumination with a high brightness level on a large window (e.g., to replicate the expected amount of light that might come from a larger window).

[0216] In this disclosure Figure 8B In the context of the simulation window, refer to the area indicated by the dashed boundary. The dashed boundary is a visual aid used to indicate the simulated window. The simulated window is a designated area on a window that the light source 612 defines to alter (e.g., modify and / or change) the physical window (e.g., increase its size, increase its brightness, decrease its size, decrease its brightness, and / or change its shape). When the simulated window has been generated, the light source 612 emits light into the physical space 600 as if the simulated window were a real window. For example, as will be seen below, in a scene where the simulated window is larger than the real window (increasing the window size), the light source 612 emits more light into the physical space 600 to simulate the existence of a larger window in the physical space 600.

[0217] In some embodiments, light source 612 can modify existing simulated windows. For example, in response to detected input, light source 612 can enlarge, shrink, increase brightness, decrease brightness, change shape, and / or change the position of the simulated window. In some embodiments, one simulated window is modified at a time. For example, light source 612 increases the size of one simulated window on a physical window in response to detected input. In some embodiments, multiple simulated windows are modified at once. For example, light source 612 increases the size of multiple simulated windows in physical space. In some embodiments, simulated windows are modified multiple times as needed. In some embodiments, simulated windows can only be modified a specific number of times. In some embodiments, one or more modifications are made only on the simulated window to which the input is directed. For example, in response to detecting an air gesture pointing to a simulated window and / or another physical window, light source 612 applies changes only to the simulated window and / or the simulated window representing the relationship between the physical window (e.g., the relationship between window 606a and simulated window 800).

[0218] In some implementations, the illumination level depends on the size of the simulated window. For example, when it is determined that a simulated window should be generated (e.g., as guided by the user) to make the physical window appear larger, light source 612 emits light brighter than the light provided by the physical window. In another example, when it is determined that a simulated window should be generated (e.g., as guided by the user) to make the physical window appear smaller, light source 612 emits light dimmer than the light provided by the physical window. In some implementations, in response to the detection of an input corresponding to a request to increase the amount of light passing through the physical window, light source 612 concurrently illuminates the light at a higher intensity level. In some implementations, the intensity is increased when an input is detected and / or when movement of the input is detected. In some implementations, light source 612 stops emitting light when it is determined that the physical window should allow light to pass through as if the physical window were below a threshold size. In some implementations, the user can delete and / or reconfigure the simulated window (and / or simulated effects) caused by the light emitted by light source 612.

[0219] In some embodiments, light source 612 generates the simulation window based on the current location of the physical space. For example, when it is determined that the physical environment outside physical space 600 is sunny, light source 612 emits light to simulate a greater amount of light passing through window 606a compared to the amount of light emitted to simulate the amount of light passing through window 606a when it is determined that the physical environment outside physical space 600 is not sunny (such as during nighttime). In some embodiments, light source 612 emits light to simulate the amount of light passing through window 606a based on the physical environment of another location. For example, even when physical space 600 is located in sunny California, light source 612 may emit dim light based on the dark environment of Alaska. In some embodiments, the amount of light emitted via light source 612 causes the simulation window 800 to change based on the passage of time and / or weather changes relative to the physical environment and / or location. For example, in the morning, light source 612 may emit light to simulate brighter light to match sunrise, and in the evening, emit light to simulate dimmer light to match sunset. As another example, if a thunderstorm is detected in the external weather, light source 612 emits light to simulate dim light including one or more rain shadows and / or flashes of lightning. In some embodiments, light source 612 emits light to simulate shadows in the physical environment outside physical space 600. For example, when it is determined that a shadow has already been generated in the physical environment outside physical space 600 (e.g., if clouds pass in front of the sun 614 and / or a car passes by window 606a), light source 612 may emit light to simulate shadows and / or less light penetrating the environment when shadows are present in the physical environment.

[0220] Figure 9 This is a flowchart illustrating a method (e.g., method 900) for simulating different amounts of light passing through a physical opening, according to some embodiments. Some operations in method 900 may be combined, the order of some operations may be changed, and some operations may be omitted.

[0221] As described below, Method 900 provides an intuitive way to simulate varying amounts of light passing through a physical opening. Method 900 reduces the cognitive load on the user, thus creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to simulate varying amounts of light passing through a physical opening faster and more efficiently saves power and increases the time between battery charging cycles.

[0222] In some embodiments, method 900 is performed at a computer system communicating with a light source (e.g., 612a-612d and / or 612) (e.g., lighting devices, point light sources, spotlights, and / or one or more light sources) (and / or a collection of one or more light sources) and an input device (e.g., a camera, depth sensor, microphone, hardware input mechanism, rotatable input mechanism, heart monitor, temperature sensor, and / or touch-sensitive surface). In some embodiments, the computer system is a telephone, watch, tablet, fitness tracker, wearable device, accessory, speaker, light, head-mounted display (HMD), and / or personal computing device. In some embodiments, the light source is not physically connected to and / or coupled to the computer system.

[0223] The computer system detects (902) a request via an input device to change the size of a physical window (e.g., 606a) (e.g., a physical opening, a physical object through which light can pass, and / or a physical object through which light can pass from the external environment to the internal environment) in a physical space (e.g., 600) (e.g., a physical environment, a room, an office, and / or a building). Figures 8A to 8B (As described). In some embodiments, detecting a request includes detecting input corresponding to the request (e.g., tap gesture, long press gesture, verbal request and / or command, physical button press, pointing input and / or air gesture, and / or rotation of a physical input mechanism). In some embodiments, detecting a request includes receiving a message from another computer system indicating that the request was received by that other computer system.

[0224] In response to (904) detecting a request to change the size of a physical window (e.g., 606a) in the physical space (e.g., 600) (e.g., as described above regarding...) Figures 8A to 8B (as described) and based on determining that the request corresponds to a first size (e.g., a request to change a physical window to a first size and / or to allow a certain amount of light to pass through as if the physical window were a first size), the computer system provides (906) illumination via a light source (e.g., 612a-612d and / or 612) in a first manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., as described above regarding Figures 8A to 8B (As described). In some embodiments, providing illumination in a first manner includes activating a light source and / or transmitting an instruction to the light source to cause the light source to provide illumination (e.g., more illumination and / or less illumination). In some embodiments, providing illumination in a first manner includes changing the light output by the light source. In some embodiments, providing illumination in a first manner includes transmitting a request to the light source to modify the light output by the light source. In some embodiments, illumination is provided in a first manner until a request to change or stop providing illumination in the first manner is received.

[0225] In response to (904) detecting a request to change the size of a physical window in the physical space and based on determining that the request corresponds to a second size different from the first size (e.g., changing the physical window to the second size and / or allowing a certain amount of light to pass through as if the physical window were the second size), the computer system provides (908) illumination via a light source (e.g., 612a-612d and / or 612) in a second manner different from the first manner (e.g., as described above regarding...). Figures 8A to 8B (As described). In some embodiments, providing illumination in a second manner includes activating a light source. In some embodiments, providing illumination in a second manner includes altering the light output by the light source. In some embodiments, providing illumination in a second manner includes sending a request to the light source to modify the light output by the light source. In some embodiments, illumination is provided in a second manner until a request to change or stop providing illumination in a second manner is received. In some embodiments, providing illumination in a different manner allows the computer system to make it appear as if the amount of light from the window is more or less than the actual light from the window (e.g., more when the request corresponds to increasing the size of the window, and / or less when the request corresponds to decreasing the size of the window). In some embodiments, providing illumination in a corresponding manner includes altering the color, intensity, brightness, hue, warmth, coolness, and / or saturation of the light source to mimic the physical environment visible when looking out from the window. In some embodiments, providing illumination in a corresponding manner includes altering the color, intensity, brightness, hue, warmth, coolness, and / or saturation of the light source to mimic a virtual environment projected as existing outside the window. In some implementations, lighting is provided in a manner appropriate to the type of environment (e.g., physical environment) outside the window and / or including the structure (e.g., house, apartment, and / or building) that contains the window. The selective provision of lighting in a manner appropriate to the satisfaction of a predetermined set of conditions automatically allows the computer system to simulate the lighting conditions of the physical space when the physical window has different sizes, thereby performing operations when the set of conditions has been met without requiring further user input.

[0226] In some embodiments, the computer system communicates with a first set of one or more cameras (e.g., wireless and / or wired communication). In some embodiments, the computer system captures (e.g., before, during, and / or simultaneously with, detecting a request to change the size of a physical window in physical space) a first image (e.g., a still photograph and / or video of the physical window) of a physical window (e.g., 606a) via the set of one or more cameras (e.g., as described above). Figures 8A to 8B (As described above). In some implementations, in response to a request to change the size of a physical window in physical space (e.g., as described above regarding...), Figures 8A to 8B(As described above) and based on determining that the request corresponds to a third size (e.g., different from or the same as the first and / or second size) and using the first image to determine that the physical window is detected as a fourth size, the computer system provides illumination via a light source (e.g., 612a-612d and / or 612) in a third manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the third manner is different from or the same as the first and / or second manner) (e.g., as described above). Figures 8A to 8B (As described above). In some implementations, in response to detecting a request to change the size of a physical window in physical space and based on determining that the request corresponds to a third size (e.g., different from or the same as the first and / or second size) (e.g., the third size) and using a first image to determine that the physical window is detected as a fifth size (e.g., larger or smaller) different from the fourth size, the computer system provides illumination via a light source in a fourth manner different from the third manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the fourth manner is different from or the same as the first and / or second manner) (e.g., as described above). Figures 8A to 8B (As described). When a set of specified conditions is met (e.g., the physical window is detected to be of a corresponding size), lighting is selectively provided in a corresponding manner. This automatically allows the computer system to simulate lighting conditions in the physical space for physical windows of different sizes, thereby performing operations when the set of conditions has been met without requiring further user input.

[0227] In some embodiments, the computer system communicates with a second set of one or more cameras (e.g., wireless and / or wired communication). In some embodiments, the computer system captures a second image of a physical window (e.g., 606a) via the second set of one or more cameras (e.g., before, during, and / or simultaneously with detecting a request to change the size of the physical window in physical space) (e.g., still photographs and / or videos of the physical window) (e.g., as described above regarding...). Figures 8A to 8B (As described above). In some implementations, in response to a request to change the size of a physical window in physical space (e.g., as described above regarding...), Figures 8A to 8B(As described) and based on determining that the request corresponds to a sixth size and using a second image (and / or a set of one or more images) (and / or in the second image (and / or a set of one or more images)) determining that a first amount of illumination is passing through (e.g., visible and / or detected as visible) a physical window (e.g., the first amount of illumination enters the physical space via the physical window and / or the first amount of illumination leaves the physical space via the physical window), the computer system provides illumination via a light source (e.g., 612a-612d and / or 612) in a fifth manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the fifth manner is different from or the same as the first and / or second manner) (e.g., as described above regarding) Figures 8A to 8B (As described above). In some embodiments, in response to detecting a request to change the size of a physical window in the physical space and determining, based on determining that the request corresponds to a sixth size and using a second image, that a second amount of lighting, different from the first amount of lighting, is passing through the physical window (e.g., the second amount of lighting enters the physical space via the physical window and / or the second amount of lighting leaves the physical space via the physical window), the computer system provides lighting via a light source in a sixth manner different from the fifth manner (e.g., the position, color, intensity, and / or size of the lighting) (e.g., the sixth manner is different from or the same as the first and / or second manner). Figures 8A to 8B (As described). In some embodiments, when the illumination passing through the window is a corresponding amount and a request to reduce the window size is detected, the computer system reduces the amount of illumination provided via the light source. In some embodiments, when the illumination passing through the window is a corresponding amount and a request to increase the window size is detected, the computer system increases the amount of illumination provided via the light source. Selectively providing illumination in a corresponding manner when a predetermined set of conditions is met (e.g., when a corresponding amount of illumination is detected passing through a physical window) automatically allows the computer system to simulate the amount of light entering the physical space via the physical space, thereby performing operations when the set of conditions has been met without requiring further user input.

[0228] In some implementations, detecting a request to change the size of a physical window in physical space includes: detecting input via an input device that indicates the physical window (e.g., 606a) should be changed to a corresponding size (e.g., before, after, and / or simultaneously with detecting the request to change the size of the physical window in physical space) (e.g., tap input, swipe input, voice command, and / or air hand gesture). Figures 8A to 8B (As described above). In some implementations, in response to detecting input indicating that the physical window should be changed to a corresponding size, the computer system specifies (e.g., assigns, sets, and / or determines) the size of the physical window to the corresponding size (e.g., as described above regarding...). Figures 8A to 8B(As described). In some embodiments, the size of the physical window corresponds to a virtual size of the physical window (and in some embodiments, not the exact size of the physical window). In some embodiments, the size of the physical window does not correspond to the physical window's dimensions. In some embodiments, the window size corresponds to the amount of light passing through the physical window. In some embodiments, the size of the physical window depends on the time of day. In some embodiments, the size of the physical window can be changed. In some embodiments, the size of the physical window cannot be changed. Specifying the size of the physical window in response to detecting input indicating that the physical window should be changed to an appropriate size allows the computer system to adjust the lighting conditions within the physical space based on user preferences (e.g., based on a user-defined size of the physical window), thereby providing improved visual feedback and / or reducing the amount of input required to perform an operation.

[0229] In some implementations, detecting a request to change the size of a physical window (e.g., 606a) in physical space includes detecting (e.g., before and / or simultaneously with detecting a request to change the size of a physical window in physical space) input corresponding to a selection of the physical window's position (e.g., position relative to an object and / or structure within or outside physical space). In some implementations, in response to detecting input corresponding to a selection of the physical window's position (e.g., as described above regarding...), the input... Figures 8A to 8B (As described above) and based on determining that the input corresponds to a seventh size and that the selected position of the physical window is a first position in physical space, the computer system provides illumination via a light source (e.g., 612a-612d and / or 612) in a seventh manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the seventh manner is different from or the same as the first and / or second manner). Figures 8A to 8B (As described above). In some embodiments, in response to detecting an input corresponding to the selection of a physical window's position and based on determining that the input corresponds to a seventh size and that the selected position of the physical window is a second position in physical space different from the first position, the computer system provides illumination via a light source in an eighth manner different from the seventh manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the eighth manner is different from or the same as the first and / or second manner). Figures 8A to 8B(As described). In some embodiments, the second position overlaps with the first position. In some embodiments, the first and second positions do not overlap. In some embodiments, lighting is based on the window's position in physical space and the time of day. In some embodiments, the computer system provides different lighting via a light source when the selected position is different, because the selected position may have different lighting and / or different natural lighting (e.g., lighting without changing the light source) through the window at that position. The automatic provision of lighting in a selective manner when a predetermined set of conditions is met (e.g., the selected position of the window corresponds to a corresponding position) allows the computer system to simulate the physical window's positioning at different locations in physical space, thereby performing operations when the set of conditions has been met without requiring further user input.

[0230] In some implementations, the physical window is the first physical window (e.g., 606a) (e.g., as described above regarding...). Figures 8A to 8B (As described above). In some implementations, the physical space includes a second physical window that is different from the first physical window (and / or physical window) (e.g., as described above regarding...). Figures 8A to 8B (As described). In some embodiments, providing illumination in a first manner via a light source (e.g., 612a-612d and / or 612) includes simulating (e.g., altering, mimicking, emulating, and / or reproducing) a change in one or more characteristics of light corresponding to a first physical window (e.g., a certain amount of light entering and / or leaving the physical space via the first physical window) (e.g., an increase in light intensity, a decrease in light intensity, an increase in light brightness, and / or a decrease in light brightness), without simulating a change (and / or alteration) in one or more characteristics of light corresponding to a second physical window (e.g., a certain amount of light entering and / or leaving the physical space via the second physical window and / or being simulated as a certain amount of light entering and / or leaving the physical space via the second physical window) (e.g., altering the illumination around and / or near the first physical window without altering the illumination around and / or near the second physical window) (e.g., as described above regarding Figures 8A to 8B(As described). In some embodiments, providing illumination via a light source in a second manner includes simulating a change in one or more characteristics of light corresponding to a first physical window, but not simulating a change in one or more characteristics of light corresponding to a second physical window. In some embodiments, the first and second physical windows are on a common side and / or the same side of the physical space (and in some embodiments, along the same wall of the physical space). In some embodiments, the first and second physical windows are on opposite sides of the physical space (and / or in some embodiments, on different walls of the physical space). In some embodiments, the first and second physical windows are identical (e.g., the same size, shape, and / or transparency). In some embodiments, the first and second physical windows are not identical. Simulating a change in one or more characteristics of light corresponding to the first physical window but not simulating a change in one or more characteristics of light corresponding to the second physical window when a predetermined set of conditions is met automatically allows a computer system to selectively change the lighting conditions in portions of the physical space (e.g., less than the entire physical space) without changing the lighting conditions in other portions of the physical space, thereby performing an operation when the set of conditions has been met without requiring further user input.

[0231] In some implementations, the physical window is a third physical window (e.g., 606a) (e.g., as described above regarding...). Figures 8A to 8B (As described above). In some implementations, the physical space includes a fourth physical window (e.g., as described above regarding...). Figures 8A to 8B (As described). In some embodiments, providing illumination in a first manner via a light source (e.g., 612a-612d and / or 612) includes simulating changes in one or more characteristics of light corresponding to a third physical window (e.g., an increase in light intensity, a decrease in light intensity, an increase in light brightness, and / or a decrease in light brightness), and simulating changes in one or more characteristics of light corresponding to a fourth physical window (e.g., as described above regarding...). Figures 8A to 8B(As described) (e.g., a certain amount of light entering and / or leaving the physical space via a third physical window and a fourth physical space, and / or simulated as a certain amount of light entering and / or leaving the physical space via a third physical window and a fourth physical window). In some embodiments, providing illumination via a light source in a second manner includes simulating a change in one or more characteristics of light corresponding to the third physical window, and simulating a change in one or more characteristics of light corresponding to the fourth physical window. In some embodiments, simulating a change in one or more characteristics of light corresponding to the third physical window includes changing one or more light sources in a manner different from the manner used to change one or more light sources to simulate a change in one or more characteristics of light corresponding to the fourth physical window. In some embodiments, simulating a change in one or more characteristics of light corresponding to the third physical window includes changing one or more light sources in the same manner as the manner used to change one or more light sources to simulate a change in one or more characteristics of light corresponding to the fourth physical window. In some embodiments, the third physical window and the fourth physical window are on a common side and / or the same side of the physical space (and in some embodiments, along the same wall of the physical space). In some embodiments, the third physical window and the fourth physical window are on opposite sides of the physical space (and / or in some embodiments, on different walls of the physical space). In some implementations, the third and fourth physical windows are identical (e.g., the same size, shape, and / or transparency). In other implementations, the third and fourth physical windows are not identical. Simulating changes in one or more properties of light corresponding to the third and fourth physical windows when a specified set of conditions is met automatically allows the computer system to simultaneously change the lighting conditions in two or more discrete parts of the physical space, thereby performing operations without further user input once the set of conditions has been met.

[0232] In some implementations, illumination is provided via a light source (e.g., 612a-612d and / or 612) in a first and second manner based on (e.g., according to, about, originating from, depending on) the current location of the computer system in the physical space (e.g., as stated above regarding) Figures 8A to 8B(As described). In some implementations, the closer the computer system is to the physical window, the more light the light source provides. In some implementations, the farther the computer system is from the physical window, the less light the light source provides. In some implementations, providing illumination in the first and second modes is not based on the current position of the computer system within the physical space. In some implementations, one or more characteristics of the illumination change as the computer system moves throughout the physical space. Providing illumination based on the current position of the computer system within the physical space when a specified set of conditions is met automatically allows the computer system to dynamically change the illumination of the physical space as it moves throughout the physical space, thereby performing operations when the set of conditions has been met without requiring further user input.

[0233] In some embodiments, illumination provided via a light source (e.g., 612a-612d and / or 612) in a first manner (and in some embodiments, in a second manner) is based on (e.g., according to, about, originating from, depending on) a location exceeding a threshold distance (e.g., 10 miles to 100,000 miles) from the current location of the computer system (e.g., the location of an external computer system with which the computer system communicates (and in some embodiments, the owner and / or user of the external computer system is different from the owner / user of the computer system) (e.g., communicating via real-time communication sessions (such as telephone calls, video calls, and / or messaging sessions)), the previous location of the computer system, the current location of the computer system, and / or the future location of the computer system) (e.g., as stated above regarding Figures 8A to 8B (As described). When a set of specified conditions is met, lighting is automatically provided based on the location of the computer system. This allows the computer system to provide indications of current lighting conditions in various environments around the world, thereby performing operations when the set of conditions has been met without requiring further user input.

[0234] In some embodiments, providing illumination in a first manner includes outputting a first amount of light (e.g., 0.5 lux, 1 lux, 2 lux, 5 lux, or 10 lux) via a light source (e.g., 612a-612d and / or 612). In some embodiments, providing illumination in a second manner includes outputting a second amount of light (e.g., 0.5 lux, 1 lux, 2 lux, 5 lux, or 10 lux) via a light source that differs from the first amount of light (e.g., 0.5 lux, 1 lux, 2 lux, 5 lux, or 10 lux) (e.g., 0.5 times, 1.5 times, 2 times, 5 times, or 10 times the first amount of light) (e.g., as described above regarding...). Figures 8A to 8B(As described). In some embodiments, providing illumination in a first manner and in a second manner includes outputting the same amount of light. In some embodiments, the first amount of light is a factor and / or a multiple of the second amount of light. In some embodiments, providing illumination in a first manner includes having a first number of light sources provide light and / or changing the amount of light provided by the first number of light sources, and providing illumination in a second manner includes having a second number of light sources, different from the first number, provide light and / or changing the amount of light provided by the second number of light sources. Selectively providing different amounts of illumination when a predetermined set of conditions is met automatically allows the computer system to change the amount of light in the physical space based on the selected size of the physical window, thereby performing operations when the set of conditions has been met without requiring further user input.

[0235] In some implementations, providing illumination in a first manner (and / or a second manner) includes simulating (e.g., mimicking, imitating, and / or emulating) light as if it were traveling from a region outside (and / or outside the physical space and / or not inside the physical space) (e.g., outside the physical space and / or on the side of the window opposite to the physical space) through (e.g., emitted from, diffused from) a physical window (e.g., 606a) (e.g., light is simulated to enter and / or leave the physical space via a first physical space) to a region inside (and / or inside the physical space and / or not outside the physical space) (and / or on the same side of the window) (e.g., as described above regarding) a region inside (and / or inside the physical space and / or not outside the physical space). Figures 8A to 8B (As described). In some implementations, providing illumination in a first and second manner includes simulating light as traveling from an area within the physical space through a physical window to an area outside the physical space. Simulating light as traveling from an area outside the physical space through a physical window when a predetermined set of conditions is met automatically allows the computer system to simulate lighting conditions directly outside the physical space, thereby performing operations when the set of conditions has already been met without requiring further user input.

[0236] In some embodiments, a request to change the size of a physical window (e.g., 606a) in a physical space (e.g., 600) is a request to reduce the size of the physical window (e.g., to 1 / 2, 1 / 3, 1 / 4, 1 / 5, or 1 / 10). In some embodiments, providing illumination via a light source (e.g., 612a-612d and / or 612) in a first manner (and / or a second manner) includes reducing the illumination in the physical space (and in some embodiments, simulating (e.g., mimicking, imitating, and / or emulating) a reduction in the amount of light entering and / or leaving the physical space via the physical window) corresponding to the physical window (e.g., light simulated as entering and / or leaving the physical space via the physical window)) (e.g., as described above regarding Figures 8A to 8B(As described). In some embodiments, the size of the physical window does not change in response to receiving a request to reduce the size of the physical window. In some embodiments, the light source simulates the same reduction in the amount of light corresponding to the physical window when providing illumination in a first mode and a second mode. In some embodiments, the light source simulates different amounts of reduction in the amount of light corresponding to the physical window when providing illumination in a first mode, compared to when the light source provides illumination in a second mode. Reducing the illumination in the physical space in response to detecting a request to reduce the size of the physical window allows the computer system to simulate the lighting conditions in the physical space when the physical window is smaller, thereby performing operations without further user input when the set of conditions has been met.

[0237] In some embodiments, a request to change the size of a physical window (e.g., 606a) in a physical space (e.g., 600) is a request to increase the size of the physical window (e.g., reduce it to 2x, 3x, 4x, 5x, or 10x). In some embodiments, providing illumination via a light source (e.g., 612a-612d and / or 612) in a first manner (and / or a second manner) includes increasing the illumination in the physical space (and / or simulating an increase in the amount of light corresponding to the physical window (e.g., simulated as light entering and / or leaving the physical space via the physical window)) (e.g., as described above regarding...). Figures 8A to 8B (As described). In some embodiments, the size of the physical window does not change in response to receiving a request to increase the size of the physical window. In some embodiments, the light source simulates the same increase in the amount of light corresponding to the physical window when providing illumination in a first mode and a second mode. In some embodiments, the light source simulates different amounts of increase in the amount of light corresponding to the physical window when providing illumination in the first mode, compared to when the light source provides illumination in the second mode. Increasing the illumination in the physical space in response to detecting a request to increase the size of the physical window allows the computer system to simulate the lighting conditions in the physical space even when the physical window is larger, thereby performing operations without further user input when the set of conditions has already been met.

[0238] In some implementations, in response to detecting a request to change the size of a physical window (e.g., 606a) in a physical space (e.g., 600) and based on determining that the request corresponds to an eighth size less than a threshold size (e.g., greater than 1 square foot to 10 square feet or 1 square inch to 120 square inches), the computer system abandons providing illumination via light sources (e.g., 612a-612d and / or 612) (and / or reduces the provided illumination to an amount of zero and / or no greater than a threshold amount (e.g., 0.1 lux to 1 lux), and / or does not increase the provided illumination to an amount greater than zero and / or greater than a threshold amount) (e.g., as described above regarding...). Figures 8A to 8B(As described). In some embodiments, the physical space is not illuminated when the light source does not provide illumination and / or changes the illumination. In some embodiments, the physical space is illuminated when the light source does not provide illumination and / or changes the illumination. In some embodiments, the size threshold is determined by the user. In some embodiments, the size threshold is determined by the computer system. In some embodiments, the size threshold is determined based on the size of the physical space (e.g., the threshold is larger when the physical space is larger). In some embodiments, the size threshold is determined based on the actual size of the first physical window. In some embodiments, the size threshold is determined based on the shape of the physical space and / or one or more objects in the physical space (e.g., one or more objects that are close to the physical window, in front of the physical window, adjacent to the physical window, and / or near the physical window). Rejecting illumination in the physical space when a predetermined set of conditions is met (e.g., a request to change the size of the physical window corresponds to a size smaller than the threshold size) automatically allows the computer system to simulate lighting conditions within the physical space where light cannot pass through the physical window, thereby performing operations when the set of conditions has been met without further user input.

[0239] In some implementations, the threshold size corresponds to (e.g., is and / or is set to) a size greater than zero (e.g., as described above regarding...). Figures 8A to 8B (as described).

[0240] In some implementations, the threshold size corresponds to (e.g., is and / or is set to) a size of zero (e.g., as described above regarding...). Figures 8A to 8B (as described).

[0241] In some embodiments, providing illumination in a first manner via a light source (e.g., 612a-612d and / or 612) includes: during a first time period, while providing illumination in the first manner, changing the illumination via the light source to a first illumination amount (e.g., as described above regarding...). Figures 8A to 8B (as described above); and during a second time period different from the first time period, when illumination is provided in the first manner, a second illumination amount is changed via a light source to a different amount than the first illumination amount (e.g., as described above regarding...). Figures 8A to 8B(As described). In some embodiments, when illumination is provided via a light source in a first manner, the computer system detects that a time period (e.g., 1 second to 600 seconds) has expired. In some embodiments, in response to detecting the expiration of the time period and in the absence of detected input (e.g., automatically without intervention input), the computer system: stops providing illumination in the first manner; and provides illumination via the light source in a fifth manner different from the first manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the light source outputs more or less illumination when outputting illumination in the fifth manner than when the light source outputs illumination in the first manner). In some embodiments, the light source stops providing illumination in the fifth manner and provides illumination in the first manner in response to detecting the expiration of the corresponding time period. Changing the illumination by a first amount during the first time period and by a second amount during the second time period automatically allows the computer system to change the amount of illumination in the physical space to simulate how the characteristics of light change over time, thereby performing operations when a set of conditions has been met without requiring further user input.

[0242] In some implementations, while illumination is provided in a first manner via a light source (e.g., 612a-612d and / or 612), the computer system detects input (e.g., swipe input, tap input, long press, voice command, gaze, and / or hand gestures in the air) corresponding to an additional request to change the size of a physical window via an input device (e.g., as described above regarding...). Figures 8A to 8B (As described above). In some implementations, in response to detecting input corresponding to an additional request to change the size of a physical window, the computer system stops providing illumination in the first manner, and the computer system provides illumination via a light source in a fifth manner different from the first manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the light source outputs more or less illumination when outputting illumination in the fifth manner than when the light source outputs illumination in the first manner). Figures 8A to 8B (As described). In some embodiments, the computer system, in response to detecting the same or different corresponding input, stops providing illumination in a fifth manner and provides illumination in a first manner via a light source. In some embodiments, the fifth manner is based on an additional request rather than the original request, and the first manner is not based on the additional request (e.g., indicated by the additional request and / or the size of the physical window corresponding to the additional request). Stopping the first manner of illumination and providing illumination in a fifth manner in response to detecting input corresponding to an additional request to change the size of the window allows the computer system to provide an indication of the state of the computer system (e.g., the computer system has detected input corresponding to an additional request to change the size of the window), thereby providing improved feedback.

[0243] In some embodiments, providing illumination in a first manner via a light source (e.g., 612a-612d and / or 612) includes causing a first silhouette (e.g., a shape and / or outline (e.g., a shadow) formed by the absence of light) to appear in physical space. In some embodiments, providing illumination in a second manner via a light source includes causing a second silhouette (e.g., a shape and / or outline (e.g., a shadow) formed by the absence of light) to appear in physical space, different from (e.g., different in size, shape, hue, color, warmth, coolness, and / or brightness) the first silhouette (e.g., different from the first silhouette in terms of size, shape, hue, color, warmth, coolness, and / or brightness). Figures 8A to 8B (As described). When a set of specified conditions is met, the first silhouette appears in the physical space. This automatically allows the computer system to simulate the appearance of one or more objects (e.g., one or more objects in the physical space or one or more objects in the space outside the physical space), thereby performing an operation when the set of conditions has been met without requiring further user input.

[0244] In some embodiments, the light source (e.g., 612a-612d and / or 612) includes a first external lamp and a second external lamp different from the first external lamp. In some embodiments, providing illumination via the light source in a first manner includes causing the first external lamp (e.g., 612a-612d) (e.g., a lighting device, a point light source, a spotlight, and / or one or more light sources) to provide first illumination (e.g., as described above regarding...). Figures 8A to 8B As described above); and to cause a second external lamp (e.g., 612a-612d) (e.g., a lighting device, point light source, spotlight, and / or one or more light sources) to provide a second illumination (e.g., separate from and / or different from the first illumination) (e.g., as described above regarding Figures 8A to 8B (As described). In some embodiments, the first illumination has a different amount, shape, color, hue, and / or brightness than the second illumination. In some embodiments, the first and second external lights are the same type of light. In some embodiments, the first and second external lights are different types of lights (e.g., ceiling lights, floor lamps, light bulbs, LED lights, halogen lamps, lights fixed and / or attached to the physical space, and / or lights not attached and / or fixed to the physical space). In some embodiments, the first illumination has the same amount, shape, color, hue, and / or brightness as the second illumination. Enabling the first external light to provide first illumination and the second external light to provide illumination when a predetermined set of conditions is met allows the computer system to use multiple lights to illuminate the physical space, thereby performing an operation without further user input once the set of conditions has been met.

[0245] It should be noted that the above text regarding method 900 (for example, Figure 9The details of the process described herein also apply in a similar manner to the other methods described herein. For example, method 700 may optionally include one or more features of the various methods described herein. For example, one or more techniques described herein with respect to method 900 may be used to illuminate a physical space, wherein the illumination of the physical space is initiated based on the detection of light of a corresponding color within the physical space as described herein with respect to method 1100. For the sake of brevity, these details will not be repeated below.

[0246] Figures 10A to 10B The accompanying drawings illustrate techniques for adjusting one or more lighting conditions according to some embodiments. Such drawings are used to illustrate the processes described below, including... Figure 11 The process in.

[0247] Figures 10A to 10B The example illustrates physical space 600. It should be recognized that the above text, in its discussion of... Figures 6A to 6B and Figures 8A to 8B The discussion concerning physical space 600 applies to one or more aspects of the discussion. Figures 10A to 10B The physical space 600 includes a ceiling 602, a light source 612, a central wall 606, a window 606a, a clock 606b, and a desk 618. Further details are provided below. Figures 10A to 10BThe discussion describes a controller device (e.g., a smartphone, watch, tablet, fitness tracker, wearable device, television, multimedia device, accessory, head-mounted device (HMD), and / or personal computing device) and / or computer system detecting the color of light within physical space 600 and causing light source 612 to perform an operation. Light source 612 includes light sources 612a-612d for illuminating physical space 600. However, it should be appreciated that one or more computer systems and / or controller devices can detect sensor data, communicate sensor data, determine the operation to be performed in response to the sensor data, and / or cause the operation to be performed. For example, in some embodiments, light source 612a detects the color of light in physical space 600, determines that the color of the light satisfies a set of one or more criteria, and, in response, changes the color of the light emitted by light source 612a and / or light source 612b. As another example, a user's personal device (e.g., smartphone, watch, tablet, fitness tracker, wearable device, television, multimedia device, accessory, head-mounted device (HMD), and / or personal computing device) can receive an image of physical space 600 from a camera external to the personal device, detect the color of light in physical space 600 via the image, determine that the color of the light satisfies a set of one or more criteria, and, in response, change the color of the light emitted by light source 612a and / or light source 612b. Note that the examples of light sources 612a and 612b are merely examples, and light sources 612c and 612d are also capable of performing the above operations.

[0248] like Figure 10A As illustrated, clock 606b displays the current time as 3:00 PM. Additionally, window 606a is filled with diagonal shading to indicate the color (e.g., blue, red, yellow, orange, and / or green) of the light entering physical space 600 via window 606a. In some embodiments, the color of the light can be a solid color, a pattern of light, and / or a gradient. In some embodiments, the light passing through window 606a affects the ambient lighting within physical space 600.

[0249] In some implementations, other light sources can influence the ambient lighting within physical space 600. For example, objects within physical space 600 (e.g., lights, flashlights, displays, computer systems, and / or candles) can emit light that alters the color of the ambient lighting within physical space 600. In some implementations, Figure 10A The color of light entering physical space 600 and passing through window 606a shall not cause the color of light within physical space 600 to satisfy a set of one or more criteria for changing the color of light within physical space 600.

[0250] like Figure 10AAs illustrated, light source 612 emits light into physical space 600 (e.g., as represented by the diagonal shading lines filling each light source within light source 612). Figure 10A Previously, the computer system had determined (e.g., selected and / or chosen) that light source 612 should emit white light into physical space 600. In some embodiments, the color of such light is defined by the user and / or the computer system when the light output is initiated. For example, the user can request light source 612 to emit blue light or light that matches the amount and / or color of light at another different time of day (e.g., to change the lighting throughout the day).

[0251] like Figure 10B As illustrated, clock 606b displays the current time as 3:30 PM. Therefore, in Figure 10A and Figure 10B Thirty minutes have passed. Figure 10B As illustrated, window 606a is filled with a horizontal shaded line to indicate that the light outside physical space 600 is orange (e.g., the color of the light in the environment outside window 606a is orange). Figure 10A and Figure 10B (and / or with) Figure 10A (Different colors in the text). Figure 10B Orange light enters physical space 600 through window 606a, and the ambient lighting within physical space 600 has an orange hue.

[0252] exist Figure 10B At a location (e.g., by a controller device, light source 612, and / or by a computer system external to light source 612), it is determined that orange light is entering physical environment 600 (e.g., white light emitted by light source 612 is overwhelmed by orange incident light, causing the ambient lighting of physical space 600 to turn orange) (e.g., the controller device detects the color of the incident light when light source 612 is illuminating physical space 600).

[0253] In some implementations, the controller device detects incident light when the light source 612 is not illuminating the physical space 600. For example, in scenarios where the light source 612 is not emitting light (e.g., in an inactive mode and / or sleep mode), the light source 612 cancels the incident light because it is determined that the color of the incident light from window 606a satisfies a set of one or more criteria. In some implementations, in scenarios where the light source 612 is not emitting light (e.g., in an inactive mode and / or sleep mode), the light source 612 does not cancel the incident light in response to detecting that the color of the incident light from window 606a satisfies a set of one or more criteria. In some implementations, the light source 612 cancels the incident light only when a user is detected in the physical space 600.

[0254] In some embodiments, the color of light entering physical space 600 is determined to satisfy a set of one or more criteria for changing the color of light within physical space 600. In some embodiments, the determination is made using an image of physical space 600 (e.g., an image of a general or specific area, such as an image of window 606a or wall 606). In other embodiments, the determination is made using instructions received from a server (e.g., a weather service). In other embodiments, the determination is based on input (e.g., verbal input indicating that the color of light in a room has an orange hue). In some embodiments, the intensity of the orange hue within physical space 600 is determined (e.g., by light source 612 and / or by a computer system external to light source 612) to satisfy a set of one or more criteria.

[0255] exist Figure 10B At that location, based on the determination that orange light is entering physical space 600, light source 612 begins to emit blue light to counteract the orange light. That is, based on the determination that light of the corresponding color is entering physical space 600 or that light of the corresponding color is currently within physical space 600, light source 612 emits light of a certain color that neutralizes the corresponding color light within physical space 600. Figure 10B Because light source 612 is emitting blue light, light source 612 is filled with horizontal shadow lines (e.g., representing blue light) (e.g., compared to...). Figure 10A The horizontal shadow line filled by light source 612 creates contrast. Figure 10B In response to determining that blue light is needed, light source 612 emits blue light into physical space 600 (as demonstrated by the contrast between vertical shadows on light source 612 and horizontal shadows on window 606a). In some embodiments, the controller device determines that blue light is needed to counteract orange light. In some embodiments, the controller device determines that a color other than blue is needed to counteract orange light.

[0256] In some embodiments, light source 612 does not emit light of a certain color that cancels out light of a corresponding color detected within physical space 600. For example, light source 612 emits colorless light and / or does not emit light. In some embodiments, light source 612 cancels out and / or does not cancel out light of a color detected within physical space 600 based on user preference (e.g., user preference for yellow light), the current state of light source 612, and / or the active settings of light source 612.

[0257] In some embodiments, the light source 612 increases or decreases the amount of illumination within the physical space 600 based on the detection of light of a corresponding color within the physical space 600. For example, based on determining that the physical space 600 contains light of a dark hue (e.g., black and / or gray) (e.g., a color that reduces the overall brightness of the physical space 600), the light source 612 will increase the amount of emitted illumination. In some embodiments, conversely, based on determining that the physical space 600 contains light of a bright hue (e.g., yellow and / or pink) (e.g., a color that increases the overall brightness of the physical space 600), the light source 612 will decrease the amount of emitted illumination.

[0258] In some implementations, the light source 612 emits different colors of light based on determining that the color of light within the physical space 600 is changing and / or has changed. For example, based on determining that the light within the physical space 600 changes from orange (e.g., as described above) to green, the light source 612 changes from emitting blue light (e.g., as described above) to emitting red light.

[0259] In some embodiments, the light source 612 changes one or more characteristics of the illumination over a period of time. In some embodiments, in Figure 10A and Figure 10B During the time interval (e.g., thirty minutes have passed), Figure 10A At this location, light source 612 emits one type of light (e.g., light source 612 does not cancel any incident light entering physical space 600). Figure 10B At a certain point, thirty minutes have passed (e.g., the sun has changed position), and light source 612 (and / or a computer system external to light source 612) detects orange light in physical space 600. In response, light source 612 determines that the color of the light satisfies a set of one or more criteria, and based on this determination, changes the color of the light emitted by light source 612 to cancel the incident orange light. As another example, at dusk, light source 612 (and / or a computer system external to light source 612) detects light in physical space 600 as pale blue. In response to detecting the color of the incident light, light source 612 emits a type of light (e.g., light source 612 does not cancel any incident light entering physical space 600). Therefore, when one hour has passed from dusk to night, light source 612 (and / or a computer system external to light source 612) detects light in physical space 600 as deep blue. In response, light source 612 determines that the deep blue light satisfies a set of one or more criteria, and based on this determination, changes the color and brightness level of the light emitted by light source 612 to cancel the incident deep blue light.

[0260] Figure 11This is a flowchart illustrating a method (e.g., method 1100) for adjusting one or more lighting conditions according to some embodiments. Some operations in method 1100 may be combined, the order of some operations may be changed, and some operations may be omitted.

[0261] As described below, Method 1100 provides an intuitive way to adjust one or more lighting conditions. Method 1100 reduces the cognitive load on the user, thereby creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to interact with such devices faster and more efficiently, saving power and increasing the time between battery charging.

[0262] In some embodiments, method 1100 is performed at a computer system that communicates with external light sources (e.g., 612a-612d and / or 612) (e.g., lighting devices, point light sources, spotlights, and / or one or more light sources) and input devices (e.g., cameras, depth sensors, microphones, hardware input mechanisms, rotatable input mechanisms, heart monitors, temperature sensors, and / or touch-sensitive surfaces). In some embodiments, the computer system is a telephone, watch, tablet, fitness tracker, wearable device, accessory, speaker, light, head-mounted display (HMD), and / or personal computing device. In some embodiments, the light source is not physically connected to and / or coupled to the computer system.

[0263] The computer system detects (1102) the physical environment (and / or the physical space as described above with respect to method 900) via an input device, including light of the corresponding color (e.g., as described above with respect to...). Figures 10A to 10B (As described). In some embodiments, detecting the physical environment, including light of a corresponding color, includes capturing an image of the physical environment via a camera. In some embodiments, detecting the physical environment, including light of a corresponding color, includes receiving a message from another computer system that includes an indication of the corresponding color.

[0264] In response to (1104) detecting that the physical environment includes light of a corresponding color and determining that the corresponding color is a first color, the computer system provides (1106) illumination via an external light source (e.g., 612a-612d and / or 612) in a first manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, providing illumination in a first manner includes activating a light source. In some embodiments, providing illumination in a first manner includes changing the light output by the light source. In some embodiments, providing illumination in a first manner includes transmitting a request to the light source to modify the light output by the light source. In some embodiments, illumination is provided in a first manner until a request to change or stop providing illumination in the first manner is received.

[0265] In response to (1104) the detection of light of a corresponding color in the physical environment and based on the determination that the corresponding color is a second color different from the first color, the computer system provides (1108) illumination via an external light source (e.g., 612a-612d and / or 612) in a second manner different from the first manner (e.g., in some embodiments, in the absence of illumination provided in the first manner). Figures 10A to 10B (As described). In some embodiments, the difference between providing lighting in a first manner and a second manner does not include simply changing the color of the lighting from one color to a different color. In some embodiments, the difference between providing lighting in a first manner and a second manner does not include changing the color of the lighting. Selectively providing lighting based on the corresponding color of light in the physical environment automatically allows the computer system to mitigate any effects that the corresponding color of light in the physical environment may have on an individual's perception, thereby performing operations without requiring further user input when a set of conditions has already been met.

[0266] In some embodiments, providing illumination in a first manner includes increasing the amount of illumination (e.g., light and / or brightness) within the physical environment (e.g., 600) via an external light source (e.g., 612a-612d and / or 612) (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, providing illumination in a second manner includes increasing the amount of illumination (e.g., and / or brightness) within the physical environment via an external light source. In some embodiments, the illumination increases by a larger or smaller amount when providing illumination in a first manner compared to providing illumination in a second manner. In some embodiments, the illumination increases by the same amount when providing illumination in a first manner compared to providing illumination in a second manner. In some embodiments, the illumination increases at a faster or slower rate when providing illumination in a first manner compared to providing illumination in a second manner. In some embodiments, the illumination increases at the same rate when providing illumination in a first manner compared to providing illumination in a second manner. Increasing the first amount of illumination within the physical environment when a predetermined set of conditions is met automatically allows the computer to offset any reduction in the amount of illumination in the physical environment caused by the color of the light within the physical environment, thereby performing the operation when the set of conditions has been met without requiring further user input.

[0267] In some embodiments, providing illumination in a first manner includes reducing a second amount of illumination (e.g., and / or brightness) within the physical environment (e.g., 600) via an external light source (e.g., 612a-612d and / or 612) (e.g., as described above regarding...). Figures 10A to 10B(As described). In some embodiments, providing illumination in a second manner includes reducing a certain amount of illumination (e.g., and / or brightness) of the physical environment via an external light source. In some embodiments, the illumination is reduced by a larger or smaller amount when illumination is provided in the first manner compared to when illumination is provided in the second manner. In some embodiments, the illumination is reduced by the same amount when illumination is provided in the first or second manner. In some embodiments, the illumination is reduced at a faster or slower rate when illumination is provided in the first manner compared to when illumination is provided in the second manner. In some embodiments, the illumination is reduced at the same rate when illumination is provided in the first or second manner. Reducing the second amount of illumination within the physical environment when a predetermined set of conditions is met automatically allows the computer to counteract any increase in the amount of illumination in the physical environment caused by the color of light within the physical environment, thereby performing the operation when the set of conditions has been met without requiring further user input.

[0268] In some embodiments, providing illumination in a first manner includes outputting a third color (orange, blue, red, green, and / or yellow) via an external light source (e.g., 612a-612d). In some embodiments, providing illumination in a second manner includes outputting a fourth color different from the third color via an external light source (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, the third color is different from the first color. In some embodiments, the fourth color is different from the second color. In some embodiments, the third and / or fourth color is a combination of two or more colors. In some embodiments, the third and / or fourth color neutralizes the corresponding color (e.g., neutralizes to white, neutralizes to black, neutralizes to a less warm color, and / or neutralizes to a less cool color). In some embodiments, the third and / or fourth color is opposite to and / or opposite to each other (e.g., opposite to the corresponding color on the color wheel). Outputting the corresponding color when a specified set of conditions is met automatically allows the computer system to selectively output a color that is complementary to or contrasts with the corresponding color of the light detected in the physical environment, such that the effect of the light of the corresponding color is enhanced or mitigated, thereby performing an operation when the set of conditions has been met without further user input.

[0269] In some implementations, providing illumination in a first manner includes outputting a color via an external light source that neutralizes (e.g., cancels, balances, counteracts, and / or invalidates) the corresponding color (e.g., as described above regarding...). Figures 10A to 10B(As described). In some embodiments, providing illumination in a second manner includes outputting a color that does not neutralize the corresponding color via an external light source. In some embodiments, providing illumination in a second manner includes outputting a color that neutralizes the corresponding color via an external light source. In some embodiments, the corresponding color is invisible when the external light source outputs a color that neutralizes the corresponding color. In some embodiments, both the corresponding color and the color that neutralizes the corresponding color are invisible (and / or appear invisible) when the external light source outputs a color that neutralizes the corresponding color. In some embodiments, the color that neutralizes the corresponding color is invisible when the corresponding color is detected in the environment. In some embodiments, when the computer system outputs a color that neutralizes the corresponding color, another color different from the corresponding color and the color that neutralizes the corresponding color is visible. Outputting a color that neutralizes the corresponding color when a predetermined set of conditions is met automatically allows the computer system to mitigate any effect that the corresponding color may have on the perception of individuals in the physical environment, thereby performing an operation without further user input when the set of conditions has been met.

[0270] In some implementations, upon detecting that the physical environment (e.g., 600) includes light of the corresponding color (e.g., and / or before), the computer system provides illumination via an external light source (e.g., 612a-612d) in a third manner (e.g., the location, color, intensity, and / or size of the illumination) (e.g., the third manner differs from the first and / or second manner) (e.g., as described above regarding...). Figures 10A to 10B (As described above). In some implementations, in response to detecting that the physical environment includes light of the corresponding color and based on determining that the corresponding color is a fifth color (e.g., red, blue, yellow, orange, and / or black) different from the first and second colors, the computer system continues to provide illumination in a third manner via an external light source (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, the fifth color is a darker shade of the first and / or second color. In some embodiments, in response to detecting that the physical environment includes light of the corresponding color and based on determining that the corresponding color is the fifth color, the computer system provides illumination in a first and / or second manner. Continuing to provide illumination in a third manner when a predetermined set of conditions is met automatically allows the computer system to avoid disrupting the atmosphere created in the physical environment due to the inclusion of the fifth color, thereby performing operations without further user input once the set of conditions has been met.

[0271] In some embodiments, the corresponding color of light is a first corresponding color of light (e.g., red, yellow, blue, orange, and / or green) (e.g., before detecting a change in the corresponding color of light from the first corresponding color of light to another corresponding color of light). In some embodiments, when illumination is provided in a first manner (and / or a second manner) via an external light source (e.g., 612a-612d), the computer system detects via an input device a change in the corresponding color of light from the first corresponding color of light to the second corresponding color of light (e.g., the first corresponding color of light changes based on changes in the environment and / or based on changes in the operating state of the artificial light) (e.g., as described above regarding...). Figures 10A to 10B (As described above). In some embodiments, in response to detecting a change in light of a corresponding color from a first corresponding color to a second corresponding color (e.g., when illumination is provided in a first manner via an external light source), the computer system provides illumination via the external light source in a fourth manner different from (e.g., different color, different color intensity, different color hue, different color tone) the first manner (e.g., and / or the second manner) (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, in response to detecting a change in the color of light from a first corresponding color to a second corresponding color, the computer system continues to provide illumination in a first manner. In some embodiments, when the external light source outputs illumination in a fourth manner, it outputs light of a different color compared to when the external light source outputs illumination in the first manner. In some embodiments, when the external light source outputs illumination in a fourth manner, it outputs light of the same color compared to when the external light source outputs illumination in the first manner. Providing illumination in a fourth manner, different from the first manner, in response to detecting a change in the color of light from a first corresponding color to a second corresponding color allows the computer system to continuously update the illumination in the physical space based on detected changes in the color of light in the physical environment, thereby providing improved feedback to the user and / or reducing the amount of input required to perform operations.

[0272] In some embodiments, providing illumination via an external light source (e.g., 612a-612d) in a first manner includes outputting light of a color different from the corresponding color of light (e.g., orange, yellow, green, blue, and / or red) (e.g., as described above regarding...). Figures 10A to 10B(As described). In some embodiments, providing illumination in a first manner includes outputting a darker color of light as the corresponding color. In some embodiments, providing illumination in a second manner via an external light source includes outputting another color different from the corresponding color of light. Outputting light of a color different from the corresponding color of light when a predetermined set of conditions is met automatically allows the computer system to output a color that is complementary to or contrasts with the corresponding color of light detected in the physical environment, such that the effect of that color of light is enhanced or mitigated, thereby performing an operation when the set of conditions has been met without further user input.

[0273] In some embodiments, the external light source (e.g., 612a-612d) includes a first external lamp (e.g., 612a-612d) and a second external lamp (e.g., 612a-612d) different from the first external lamp. In some embodiments, providing illumination via the light source in a first manner includes: causing the first external lamp (e.g., a lighting device, a point light source, a spotlight, and / or one or more light sources) to provide first illumination (e.g., as described above regarding...). Figures 10A to 10B As described above); and causing a second external light (e.g., a lighting device, point light source, spotlight, and / or one or more light sources) to provide a second illumination (e.g., separate from and / or different from the first illumination) (e.g., as described above regarding Figures 10A to 10B (As described). In some embodiments, the first illumination has a different amount, shape, color, hue, and / or brightness than the second illumination. In some embodiments, the first and second external lights are the same type of light. In some embodiments, the first and second external lights are different types of lights (e.g., ceiling lights, floor lamps, light bulbs, LED lights, halogen lamps, lights fixed and / or attached to the physical space, and / or lights not attached and / or fixed to the physical space). In some embodiments, the first illumination has the same amount, shape, color, hue, and / or brightness as the second illumination. Automatically enabling the computer system to simultaneously cancel out the effects of corresponding colors in different parts of the physical environment when a predetermined set of conditions is met allows the computer system to perform the operation without further user input once the set of conditions has been met.

[0274] In some embodiments, providing illumination via an external light source in a first manner includes: during a first time period, while providing illumination in the first manner, changing the illumination by a first illumination amount via the light source (e.g., 612a-612d) (e.g., as described above regarding...). Figures 10A to 10BAs described above); and during a second time period different from the first time period, when illumination is provided in the first manner, a second illumination amount (e.g., as described above) is used to change the illumination via a light source (e.g., 612a-612d) to a different amount than the first illumination amount (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, when illumination is provided via a light source in a first manner, the computer system detects that a time period (e.g., 1 second to 600 seconds) has expired. In some embodiments, in response to detecting the expiration of the time period and in the absence of detected input (e.g., without intervention input and / or automatically), the computer system: stops providing illumination in the first manner and provides illumination via the light source in a fifth manner different from the first manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., the light source outputs more or less illumination when outputting illumination in the fifth manner than when the light source outputs illumination in the fifth manner). In some embodiments, the light source stops providing illumination in the fifth manner and provides illumination in the first manner in response to detecting the expiration of the corresponding time period. Changing the illumination to a first illumination amount during a first time period and changing the illumination to a second illumination amount during a second time period when a predetermined set of conditions is met automatically allows the computer system to change the atmosphere and / or lighting in the physical space within different time periods to take into account any changes to one or more conditions of the physical space, thereby performing operations when the set of conditions has been met without requiring further user input.

[0275] In some implementations, providing illumination via an external light source in a first manner includes: based on determining that a detected user preference (e.g., settings, preferences determined by one or more historical preferences and / or actions of the user, and / or learned preferences over time) is a first user preference, causing an external light source (e.g., 612a-612d) to illuminate light having a first characteristic (e.g., amount of color, light, brightness, and / or shape) rather than a second characteristic (e.g., as described above regarding...). Figures 10A to 10B As described above); and based on determining that the detected user preference (e.g., settings, preferences determined by one or more historical preferences and / or actions of the user, and / or learned preferences over time) is a second user preference different from the first user preference, causing an external light source (e.g., 612a-612d) to illuminate light having the second characteristic instead of the first characteristic (e.g., as described above regarding...). Figures 10A to 10B (As described). When a set of specified conditions is met, the computer system can selectively illuminate the space with light of corresponding characteristics, automatically allowing the computer system to customize the lighting of the physical space according to the user's preferences, thereby performing operations without further user input once the set of conditions has been met.

[0276] In some implementations, when the physical environment (e.g., 600) is detected via an input device to include light of a corresponding color, the computer system provides illumination via an external light source (e.g., 612a-612d) in a sixth manner (e.g., the position, color, intensity, and / or size of the illumination) (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, the sixth mode may be the same as or different from the first and / or second modes. In some embodiments, detecting that the physical environment includes light of the corresponding color causes the computer system to provide illumination in the sixth mode via an external light source. In some embodiments, the computer system stops providing illumination in the sixth mode via an external light source when the physical environment no longer includes light of the corresponding color. Providing illumination in the sixth mode when the physical environment includes light of the corresponding color allows the computer system to provide illumination to the physical space such that the physical space is not only illuminated by light of the corresponding color, thereby performing an operation (e.g., providing illumination) when the set of conditions is already met (when the computer system detects that the physical environment includes light of the corresponding color) without requiring further user input.

[0277] In some implementations, light of the corresponding color included in the physical environment is detected even when no input is detected (e.g., input from a user and / or subject). In some implementations, light of the corresponding color is detected when the physical environment (e.g., 600) (e.g., currently and / or at present) is not illuminated by an external light source (e.g., 612a-612d) (e.g., as described above). Figures 10A to 10B (As described). In some embodiments, the physical environment is illuminated via a corresponding light source when the color of light included in the physical environment is detected. In some embodiments, the physical environment is not illuminated via any corresponding light source when the color of light included in the physical environment is detected. In some embodiments, light of a corresponding color included in the physical environment is detected in response to the detection of input. Detecting light of a corresponding color in the absence of input allows the computer system (e.g., without intervention from user input) to continuously provide illumination in a corresponding manner based on changes in the corresponding color of light in the physical environment without user prompting, thereby providing additional control options without cluttering the user interface with additional displayed controls.

[0278] In some embodiments, providing illumination in a first manner includes outputting a sixth color (e.g., red, orange, green, yellow, and / or purple) via an external light source (e.g., 612a-612d). In some embodiments, detecting the physical environment (e.g., 600) includes light of the corresponding color, which includes detecting input via an input device corresponding to a request to illuminate the physical environment with a seventh color different from the sixth color (e.g., tap input, swipe input, air input, pressing a hardware button, voice command, and / or gaze) (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, the input corresponds to a request to illuminate the physical environment with a color different from the color corresponding to the second method. In some embodiments, the input corresponding to a request to illuminate the physical environment with a seventh color includes a selection of a user interface object displayed via a monitor of a computer system. In some embodiments, providing illumination in the second method includes outputting a seventh color via an external light source.

[0279] In some embodiments, providing illumination in a first manner includes outputting an eighth color (e.g., red, orange, green, yellow, and / or purple) via an external light source (e.g., 612a-612d). In some embodiments, detecting the physical environment (e.g., 600) including light of the corresponding color includes detecting input (e.g., tap input, swipe input, air input, pressing a hardware button, voice command, and / or gaze) corresponding to a request to illuminate the physical environment with the eighth color via an input device (e.g., tap input, swipe input, air input, pressing a hardware button, voice command, and / or gaze) (e.g., as described above regarding...). Figures 10A to 10B (As described). Responding to the detection of input corresponding to a request to illuminate the physical environment with an eighth color, the output of the eighth color allows the computer system to provide lighting to the physical environment based on the user's explicit preferences, thereby providing improved feedback.

[0280] In some embodiments, the computer system communicates with a collection of one or more cameras (e.g., wireless and / or wired communication). In some embodiments, detecting the physical environment, including light of corresponding colors, involves capturing images (e.g., still images and / or videos) of the physical environment via a collection of one or more cameras (e.g., 612a-612d) (e.g., images including corresponding colors) (e.g., as described above regarding...). Figures 10A to 10B (As described). In some embodiments, detecting the physical environment includes using an image of the physical environment to detect light of the corresponding color. In some embodiments, an image of the physical environment is captured before detecting the corresponding light in the physical environment. In some embodiments, an array of one or more cameras captures an image of the physical environment in response to a request to detect light of the corresponding color within the physical environment. In some embodiments, capturing an image of the physical environment is done automatically (e.g., without user intervention and / or intervention input).

[0281] It should be noted that the above text regarding method 1100 (for example, Figure 11 The details of the process described herein also apply in a similar manner to the methods described herein. For example, method 700 may optionally include one or more characteristics of the various methods described herein. For example, one or more techniques described herein with respect to method 700 may be used to illuminate a physical space, wherein the illumination of the physical space is initiated using one or more techniques described with respect to method 700, and the illumination is based on the detection of light of a corresponding color within the physical space as described with respect to method 1100. For the sake of brevity, these details will not be repeated below.

[0282] For purposes of explanation, the foregoing description has been given by reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible based on the teachings above. These embodiments were chosen and described in order to best explain the principles of these techniques and their practical application. Others skilled in the art will thus be able to best utilize these techniques and the various embodiments with various modifications suitable for the particular intended use.

[0283] While this disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. It should be understood that such changes and modifications are considered to be included within the scope of this disclosure and examples as defined by the claims.

[0284] As described above, one aspect of the present invention is to collect and use data from various sources to improve lighting in physical spaces. This disclosure contemplates that, in some instances, such collected data may include personal information that uniquely identifies or can be used to contact or locate specific individuals. Such personal information may include demographic data, location-based data, telephone numbers, email addresses, social media identifiers, home addresses, data or records relating to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying information or personal information.

[0285] This disclosure recognizes that the use of such personal information data in the techniques of this invention can benefit a user. For example, personal information data can be used to illuminate a physical space. Thus, the use of such personal information data enables a user to enable a computer system to perform operations for illuminating a physical space. Furthermore, this disclosure also contemplates other uses of personal information data that benefit the user. For example, health and fitness data can be used to provide insights into a user's overall health status or can be used as positive feedback for individuals using the technology to pursue health goals.

[0286] This disclosure anticipates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will comply with robust privacy policies and / or privacy measures. Specifically, such entities should implement and adhere to privacy policies and measures that are recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy and security of personal information data. These policies should be readily accessible to users and should be updated as data collection and / or use change. Personal information from users should be collected for legitimate and reasonable entity purposes and should not be shared or sold outside of these legitimate purposes. Furthermore, such collection / sharing should be conducted only after receiving informed consent from users. Additionally, such entities should consider taking any necessary steps to protect and safeguard the right to access such personal information data and ensure that other entities with access to personal information data comply with the privacy policies and procedures of those other entities. Moreover, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and privacy measures. Furthermore, policies and measures should be appropriate for the specific types of personal information data collected and / or accessed, and should be compatible with applicable laws and standards, including considerations of specific jurisdictions. For example, in the United States, the collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while in other countries, health data may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy measures should be advocated for different types of personal data in each country.

[0287] Regardless of the foregoing, this disclosure also contemplates implementation schemes for users to selectively block the use or access to personal information data. That is, this disclosure contemplates providing hardware and / or software components to prevent or block access to such personal information data. For example, with respect to certain services, the inventive technology can be configured to allow users to opt-in or opt-out at any time during or after service registration to participate in the collection of personal information data. In another example, users may choose not to provide certain data for certain services. In yet another example, users may choose to limit the length of time data is maintained or completely prohibit the development of user profiles. In addition to providing opt-in and opt-out options, this disclosure also contemplates providing notifications related to access to or use of personal information. For example, users may be notified when downloading an application that their personal information data will be accessed, and then reminded again just before the application accesses the personal information data.

[0288] Furthermore, the intent of this disclosure is that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Once data is no longer needed, this risk can be minimized by restricting data collection and deleting data. Additionally, and where applicable, including in certain health-related applications, data deidentification can be used to protect user privacy. Deidentification can be facilitated, where appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the address level), controlling how data is stored (e.g., aggregating data among users), and / or other methods.

[0289] Therefore, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, it is also contemplated that various embodiments can be implemented without access to such personal information data. That is, various embodiments of the present invention will not become inoperable due to the absence of all or part of such personal information data. For example, preferences can be inferred based on non-personal information data or an absolute minimum amount of personal information, such as content requested by a device associated with a user, other non-personal information available for certain services, or publicly available information, thereby selecting content and delivering it to the user.

Claims

1. A method, the method comprising: At the computer system that communicates with the light source and input device: The input device detects requests for lighting in the physical space. as well as In response to the detection of the request to illuminate the physical space: According to the determination that a first portion of the physical space is set as a light path, illumination is provided via the light source in a first manner; and Based on the determination that the first portion of the physical space is not configured as a light path, illumination provided in the first manner is abandoned.

2. The method of claim 1, wherein the first portion of the physical space corresponds to a first set of one or more physical structures covering at least a portion of the physical space.

3. The method of claim 2, wherein a second set of one or more physical structures forming at least a portion of the perimeter of the physical space is not configured as an optical path.

4. The method according to any one of claims 1 to 3, wherein the first portion corresponds to a third set of one or more structures forming at least a portion of the perimeter of the physical space.

5. The method according to any one of claims 1 to 4, wherein providing the lighting in the first manner includes providing a simulation of a weather event via the light source.

6. The method according to any one of claims 1 to 5, further comprising: Before the request to illuminate the physical space is detected, input corresponding to the selection of the first portion of the physical space is detected via the input device; as well as In response to detecting the input corresponding to the selection of the first portion of the physical space, the first portion of the physical space is set as an optical path.

7. The method of claim 6, wherein the input corresponding to the selection of the first portion of the physical space includes an air gesture.

8. The method according to any one of claims 1 to 5, wherein the first portion is configured as an optical path when no input from the user is detected.

9. The method according to any one of claims 1 to 5 and 8, the method further comprising: Combined with the detected request to illuminate the physical space: Based on the determination that the celestial body is located at the first position: The second part of the physical space is configured as an optical path; and Illumination is provided in a second manner via the light source; and Based on the determination that the first celestial body is located at a second location different from the first location: The third part of the physical space is set as an optical path, wherein the third part is different from the second part; as well as Illumination is provided via the light source in a third manner, different from the second manner.

10. The method according to any one of claims 1 to 9, wherein providing illumination via the light source in the first manner comprises: Light is simulated as the first portion passing through the physical space; as well as The simulation of light as passing through a fourth portion of the physical space is abandoned, wherein the fourth portion is different from the first portion.

11. The method according to any one of claims 1 to 10, wherein providing illumination via the light source in the first manner comprises simulating the light beam as the first portion passing through the physical space.

12. The method according to any one of claims 1 to 11, further comprising: In response to the detection of the request to illuminate the physical space and based on the determination that a fifth portion of the physical space is set as a light path, illumination is provided via the light source in a fourth manner different from the first manner, wherein the fifth portion is different from the first portion.

13. The method according to any one of claims 1 to 12, further comprising: In response to the detection of the request to illuminate the physical space and based on the determination that a sixth portion and a seventh portion of the physical space are configured as light paths, illumination is provided via the light source in a fifth manner different from the first manner, wherein the seventh portion is separate from the sixth portion.

14. The method according to any one of claims 1 to 13, further comprising: In response to the detection of the request to illuminate the physical space and based on the determination that one or more light paths are not being used for illumination in the current setup, illumination is provided via the light source in a sixth manner, different from the first manner.

15. The method according to any one of claims 1 to 14, the method further comprising: In response to the detection of the request to illuminate the physical space and based on the determination that the physical space does not have a portion of the physical space that is set as a light path, illumination is provided via the light source in a seventh manner different from the first manner.

16. The method according to any one of claims 1 to 15, the method further comprising: After illumination is provided via the light source in the first manner, an input corresponding to a request to set up an eighth part of the physical space as a light path is detected via the input device; as well as In response to the detection of the input corresponding to the request to set the eighth portion of the physical space as a light path, illumination is provided via the light source in an eighth manner, different from the first manner.

17. The method of claim 16, wherein providing illumination via the light source in the eighth manner comprises simulating light passing through the first portion of the physical space and the eighth portion of the physical space.

18. The method of claim 16, wherein providing illumination via the light source in the eighth manner comprises simulating light as passing through the eighth portion of the physical space rather than the first portion of the physical space.

19. The method according to any one of claims 1 to 18, the method further comprising: After illumination is provided via the light source in the first manner, the detection time expires; as well as In response to the detection that the time has expired, illumination is provided via the light source in a ninth manner, different from the first manner.

20. The method according to any one of claims 1 to 19, wherein: Providing illumination in the first manner, based on determining that the first portion of the physical space is located at a first position within the physical space, includes forming a first shadow; Based on determining that the first portion of the physical space is located at a second position within the physical space, providing illumination in the first manner includes forming a second shadow different from the shadow; and The second position is different from the first position.

21. The method according to any one of claims 1 to 20, wherein the lighting provided in the first manner is based on the current position of the computer system.

22. The method of any one of claims 1 to 20, wherein the illumination provided in the first manner is based on another location different from the current location of the computer system.

23. The method according to any one of claims 1 to 21, wherein providing illumination via the light source in the first manner comprises: Illumination is output via the first light source; as well as Illumination is output via a second light source that is different from the first light source.

24. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source and an input device, the one or more programs including instructions for performing the method according to any one of claims 1 to 23.

25. A computer system configured to communicate with a light source and an input device, the computer system comprising: One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 1 to 23.

26. A computer system configured to communicate with a light source and an input device, the computer system comprising: Components for performing the method according to any one of claims 1 to 23.

27. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source and an input device, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 23.

28. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source and an input device, the one or more programs including instructions for: The input device detects a request to illuminate the physical space; and In response to the detection of the request to illuminate the physical space: Based on the determination that a first portion of the physical space is configured as a light path, illumination is provided via the light source in a first manner; and Based on the determination that the first portion of the physical space is not configured as a light path, illumination provided in the first manner is abandoned.

29. A computer system configured to communicate with a light source and an input device, the computer system comprising: One or more processors; and The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: The input device detects a request to illuminate the physical space; and In response to the detection of the request to illuminate the physical space: According to the determination that a first portion of the physical space is set as a light path, illumination is provided via the light source in a first manner; and Based on the determination that the first portion of the physical space is not configured as a light path, illumination provided in the first manner is abandoned.

30. A computer system configured to communicate with a light source and an input device, the computer system comprising: A component for detecting a request to illuminate a physical space via the input device; and In response to the detection of the request to illuminate the physical space: Components for providing illumination in a first manner via the light source, based on the determination that a first portion of the physical space is configured as a light path; and The component is used to abandon providing illumination in the first manner based on the determination that the first portion of the physical space is not set as a light path.

31. A computer program product comprising one or more programs configured to execute by one or more processors of a computer system in communication with a light source and an input device, the one or more programs comprising instructions for performing the following operations: The input device detects a request to illuminate the physical space; and In response to the detection of the request to illuminate the physical space: Based on the determination that a first portion of the physical space is configured as a light path, illumination is provided via the light source in a first manner; and Based on the determination that the first portion of the physical space is not configured as a light path, illumination provided in the first manner is abandoned.

32. A method, the method comprising: At the computer system that communicates with the light source and input device: The input device detects requests to change the size of a physical window in the physical space. as well as In response to the detection of the request to change the size of the physical window in the physical space: Illumination is provided via the light source in a first manner, based on the determination that the request corresponds to a first size; and Based on the determination that the request corresponds to a second size different from the first size, illumination is provided via the light source in a second manner different from the first manner.

33. The method of claim 32, wherein the computer system communicates with a first set of one or more cameras, the method further comprising: A first image of the physical window is captured via the first set of one or more cameras; as well as In response to the detection of the request to change the size of the physical window in the physical space: Based on determining that the request corresponds to a third size and using the first image to determine that the physical window is detected as a fourth size, illumination is provided via the light source in a third manner; and Based on determining that the request corresponds to the third size and using the first image to determine that the physical window is detected as a fifth size different from the fourth size, illumination is provided via the light source in a fourth manner different from the third type.

34. The method of any one of claims 32 to 33, wherein the computer system communicates with a second set of one or more cameras, the method further comprising: A second image of the physical window is captured via a second set of one or more cameras; as well as In response to the detection of the request to change the size of the physical window in the physical space: Based on determining that the request corresponds to a sixth size and using the second image to determine that a first amount of illumination is passing through the physical window, illumination is provided in a fifth manner via the light source; and Based on the determination that the request corresponds to the sixth size and using the second image to determine that a second amount of illumination, different from the first amount of illumination, is passing through the physical window, providing illumination via the light source in a sixth manner different from the fifth manner.

35. The method of any one of claims 32 to 34, wherein detecting the request to change the size of the physical window in the physical space comprises: The input device detects that the physical window should be changed to an input of the appropriate size; as well as In response to detecting the input indicating that the physical window should be changed to the corresponding size, the size of the physical window is specified as the corresponding size.

36. The method of any one of claims 32 to 35, wherein detecting the request to change the size of the physical window in the physical space includes detecting input corresponding to the selection of the position of the physical window, the method further comprising: In response to detecting the input corresponding to the selection of the position of the physical window: Illumination is provided via the light source in a seventh manner based on determining that the input corresponds to a seventh size and determining that the selected position of the physical window is a first position in the physical space; and Illumination is provided via the light source in an eighth manner, different from the seventh manner, based on the determination that the input corresponds to the seventh size and that the selected position of the physical window is a second position in the physical space different from the first position.

37. The method according to any one of claims 32 to 36, wherein: The physical window is the first physical window; The physical space includes a second physical window that is different from the first physical window; and Providing illumination via the light source in the first manner includes simulating a change in one or more characteristics of light corresponding to the first physical window, but not simulating a change in one or more characteristics of light corresponding to the second physical window.

38. The method according to any one of claims 32 to 37, wherein: The physical window is a third physical window; The physical space includes a fourth physical window; and Providing illumination via the light source in the first manner includes simulating a change in one or more characteristics of light corresponding to the third physical window, and simulating a change in one or more characteristics of light corresponding to the fourth physical window.

39. The method according to any one of claims 32 to 38, wherein illumination is provided via the light source in the first and second modes based on the current position of the computer system in the physical space.

40. The method of any one of claims 32 to 39, wherein the illumination provided via the light source in the first manner is based on a position exceeding a threshold distance from the current position of the computer system.

41. The method according to any one of claims 32 to 40, wherein providing illumination in the first manner includes outputting a first amount of light via the light source, and wherein providing illumination in the second manner includes outputting a second amount of light via the light source that is different from the first amount of light.

42. The method according to any one of claims 32 to 41, wherein providing illumination in the first manner comprises simulating light as passing through the physical window from an area outside the physical space to an area within the physical space.

43. The method according to any one of claims 32 to 42, wherein the request to change the size of the physical window in the physical space is a request to reduce the size of the physical window, and wherein providing illumination via the light source in the first manner includes reducing the illumination in the physical space.

44. The method according to any one of claims 32 to 42, wherein the request to change the size of the physical window in the physical space is a request to increase the size of the physical window, and wherein providing illumination via the light source in the first manner includes increasing the illumination in the physical space.

45. The method according to any one of claims 32 to 44, the method further comprising: In response to the detection of a request to change the size of the physical window in the physical space and based on the determination that the request corresponds to an eighth size less than a threshold size, illumination is abandoned via the light source.

46. ​​The method of claim 45, wherein the threshold size corresponds to a size greater than zero.

47. The method of claim 45, wherein the threshold size corresponds to the size of zero.

48. The method according to any one of claims 32 to 47, wherein providing illumination via the light source in the first manner comprises: During the first time period, when the illumination is provided in the first manner, the illumination is changed to a first illumination amount via the light source; as well as During a second time period different from the first time period, when the illumination is provided in the first manner, the illumination is changed to a second illumination amount different from the first illumination amount via the light source.

49. The method according to any one of claims 32 to 48, the method further comprising: When illumination is provided via the light source in the first manner, input corresponding to an additional request to change the size of the physical window is detected via the input device; as well as In response to the detection of the input corresponding to the additional request to change the size of the physical window: Stop providing lighting in the first manner; and Illumination is provided via the light source in a fifth manner, different from the first manner.

50. The method of any one of claims 32 to 49, wherein providing illumination via the light source in the first manner comprises causing a first silhouette to appear in the physical space, and wherein providing illumination via the light source in the second manner comprises causing a second silhouette, different from the first silhouette, to appear in the physical space.

51. The method according to any one of claims 32 to 50, wherein the light source comprises a first external lamp and a second external lamp different from the first external lamp, and wherein providing illumination via the light source in the first manner comprises: The first external light provides the first illumination; as well as The second external light provides a second illumination.

52. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source and an input device, the one or more programs including instructions for performing the method according to any one of claims 32 to 51.

53. A computer system configured to communicate with a light source and an input device, the computer system comprising: One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 32 to 51.

54. A computer system configured to communicate with a light source and an input device, the computer system comprising: Components for performing the method according to any one of claims 32 to 51.

55. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source and an input device, the one or more programs comprising instructions for performing the method according to any one of claims 32 to 51.

56. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source and an input device, the one or more programs including instructions for: The input device detects requests to change the size of a physical window in physical space; and In response to the detection of the request to change the size of the physical window in the physical space: Based on the determination that the request corresponds to a first size, illumination is provided via the light source in a first manner; and Based on the determination that the request corresponds to a second size different from the first size, illumination is provided via the light source in a second manner different from the first manner.

57. A computer system configured to communicate with a light source and an input device, the computer system comprising: One or more processors; and The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: The input device detects requests to change the size of a physical window in physical space; and In response to the detection of the request to change the size of the physical window in the physical space: Illumination is provided via the light source in a first manner, based on the determination that the request corresponds to a first size; and Based on the determination that the request corresponds to a second size different from the first size, illumination is provided via the light source in a second manner different from the first manner.

58. A computer system configured to communicate with a light source and an input device, the computer system comprising: A component for detecting a request to change the size of a physical window in physical space via the input device; as well as In response to the detection of the request to change the size of the physical window in the physical space: A component for providing illumination in a first manner via the light source based on determining that the request corresponds to a first size; and A component for providing illumination via the light source in a second manner, different from the first manner, based on the determination that the request corresponds to a second size different from the first size.

59. A computer program product comprising one or more programs configured to execute by one or more processors of a computer system in communication with a light source and an input device, the one or more programs comprising instructions for: The input device detects requests to change the size of a physical window in physical space; and In response to the detection of the request to change the size of the physical window in the physical space: Based on the determination that the request corresponds to a first size, illumination is provided via the light source in a first manner; and Based on the determination that the request corresponds to a second size different from the first size, illumination is provided via the light source in a second manner different from the first manner.

60. A method comprising: At the computer system that communicates with external light sources and input devices: The physical environment, including light of the corresponding color, is detected via the input device; as well as In response to the detection of light of the corresponding color in the physical environment: Based on the determination that the corresponding color is a first color, illumination is provided via the external light source in a first manner; and Illumination is provided via the external light source in a second manner, different from the first manner, based on the determination that the corresponding color is a second color different from the first color.

61. The method of claim 60, wherein providing illumination in the first manner includes increasing the illumination within the physical environment by a first amount via the external light source.

62. The method of claim 60, wherein providing illumination in the first manner includes reducing a second amount of illumination within the physical environment via the external light source.

63. The method according to any one of claims 60 to 62, wherein providing illumination in the first manner includes outputting a third color via the external light source, and wherein providing illumination in the second manner includes outputting a fourth color different from the third color via the external light source.

64. The method according to any one of claims 60 to 63, wherein providing illumination in the first manner includes outputting a color neutralized by the corresponding color via the external light source.

65. The method according to any one of claims 60 to 64, the method further comprising: When the physical environment is detected to include light of the corresponding color, illumination is provided via the external light source in a third manner. as well as In response to detecting that the physical environment includes light of the corresponding color and determining that the corresponding color is a fifth color different from the first color and the second color, illumination continues to be provided via the external light source in the third manner.

66. The method according to any one of claims 60 to 65, wherein the corresponding color light is a first corresponding color light, the method further comprising: When illumination is provided via the external light source in the first manner, the input device detects that the light of the corresponding color changes from the first corresponding color to the second corresponding color. as well as In response to detecting that the light of the corresponding color changes from the first corresponding color to the second corresponding color, illumination is provided via the external light source in a fourth manner different from the first manner.

67. The method according to any one of claims 60 to 66, wherein providing illumination in the first manner via the external light source comprises outputting light of a color different from the corresponding color of light.

68. The method according to any one of claims 60 to 67, wherein the external light source comprises a first external lamp and a second external lamp different from the first external lamp, and wherein providing illumination via the light source in the first manner comprises: The first external light provides the first illumination; as well as The second external light provides a second illumination.

69. The method according to any one of claims 60 to 68, wherein providing illumination in the first manner via the external light source comprises: During the first time period, when the illumination is provided in the first manner, the illumination is changed to a first illumination amount via the light source; as well as During a second time period different from the first time period, when the illumination is provided in the first manner, the illumination is changed to a second illumination amount different from the first illumination amount via the light source.

70. The method of any one of claims 60 to 69, wherein providing illumination in the first manner via the external light source comprises: Based on the determination that the detected user preference is a first user preference, the external light source is made to emit light with the first characteristic rather than the second characteristic; as well as Based on the determination that the detected user preference is a second user preference different from the first user preference, the external light source is made to illuminate light having the second characteristic instead of the first characteristic.

71. The method according to any one of claims 60 to 70, the method further comprising: When the physical environment is detected to include light of the corresponding color via the input device, illumination is provided via the external light source in a sixth manner.

72. The method according to any one of claims 60 to 71, wherein light of the corresponding color included in the physical environment is detected without input being detected, and wherein light of the corresponding color is detected when the physical environment is not illuminated by the external light source.

73. The method according to any one of claims 60 to 72, wherein providing illumination in the first manner includes outputting a sixth color via the external light source, and wherein detecting the physical environment includes light of the corresponding color includes detecting, via the input device, an input corresponding to a request to illuminate the physical environment with a seventh color different from the sixth color.

74. The method of any one of claims 60 to 73, wherein providing illumination in the first manner includes outputting an eighth color via the external light source, and wherein detecting the physical environment including light of the corresponding color includes detecting an input via the input device corresponding to a request to illuminate the physical environment with the eighth color.

75. The method of any one of claims 60 to 74, wherein the computer system communicates with a set of one or more cameras, and wherein detecting the physical environment including light of the corresponding color includes capturing an image of the physical environment via the set of one or more cameras.

76. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with an external light source and an input device, the one or more programs comprising instructions for performing the method according to any one of claims 60 to 75.

77. A computer system configured to communicate with an external light source and an input device, the computer system comprising: One or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 60 to 75.

78. A computer system configured to communicate with an external light source and an input device, the computer system comprising: Components for performing the method according to any one of claims 60 to 75.

79. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with an external light source and an input device, the one or more programs comprising instructions for performing the method according to any one of claims 60 to 75.

80. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with an external light source and an input device, the one or more programs including instructions for: The physical environment, including light of the corresponding color, is detected via the input device; and In response to the detection of light of the corresponding color in the physical environment: Based on the determination that the corresponding color is a first color, illumination is provided via the external light source in a first manner; and Illumination is provided via the external light source in a second manner, different from the first manner, based on the determination that the corresponding color is a second color different from the first color.

81. A computer system configured to communicate with an external light source and an input device, the computer system comprising: One or more processors; and The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: The physical environment, including light of the corresponding color, is detected via the input device; and In response to the detection of light of the corresponding color in the physical environment: Based on the determination that the corresponding color is a first color, illumination is provided via the external light source in a first manner; and Illumination is provided via the external light source in a second manner, different from the first manner, based on the determination that the corresponding color is a second color different from the first color.

82. A computer system configured to communicate with an external light source and an input device, the computer system comprising: Components for detecting the physical environment, including light of corresponding colors, via the input device; and In response to the detection of light of the corresponding color in the physical environment: A component for providing illumination in a first manner via the external light source based on determining that the corresponding color is a first color; and A component for providing illumination via an external light source in a second manner, different from the first manner, based on the determination that the corresponding color is a second color different from the first color.

83. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with an external light source and an input device, the one or more programs comprising instructions for performing the following operations: The physical environment, including light of the corresponding color, is detected via the input device; and In response to the detection of light of the corresponding color in the physical environment: Based on the determination that the corresponding color is a first color, illumination is provided via the external light source in a first manner; and Illumination is provided via the external light source in a second manner, different from the first manner, based on the determination that the corresponding color is a second color different from the first color.

Images