Lighting effects

Abstract

The present disclosure generally relates to providing lighting effects.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Nonprovisional Patent Application Serial No. 18 / 611,568 entitled "LIGHTING EFFECTS", filed March 20, 2024, and U.S. Provisional Patent Application Serial No. 63 / 453,721 entitled "LIGHTING EFFECTS", filed March 21, 2023, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0002] This disclosure relates generally to computer user interfaces, and more specifically to techniques for providing lighting effects. Background Technology

[0003] Light is typically used for various purposes. For example, light can be used to illuminate a room and / or area of ​​the physical environment. Summary of the Invention

[0004] However, some technologies used to provide lighting effects with electronic devices are often cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may include multiple buttons or keystrokes. These existing technologies require more time than necessary, resulting in wasted user time and device power. This latter consideration is particularly important in battery-powered devices.

[0005] Therefore, the present invention provides electronic devices with faster and more efficient methods and interfaces for providing lighting effects. Such methods and interfaces optionally complement or replace other methods for providing lighting effects. These methods and interfaces reduce the cognitive burden on the user and result in a more efficient human-machine interface. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charging cycles.

[0006] In some examples, a method is described that is executed at a computer system communicating with a light source. In some examples, the method includes: detecting a request for an area of ​​a physical space to be illuminated; in response to detecting the request for an area of ​​the physical space to be illuminated: providing a first type of illumination via a light source based on determining that the area of ​​the physical space has a first attribute; and abandoning the provision of the first type of illumination based on determining that the area of ​​the physical space has a second attribute different from the first attribute.

[0007] In some examples, a non-transitory computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting a region of a physical space to be illuminated; in response to detecting the region of the physical space to be illuminated: providing a first type of illumination via the light source based on determining that the region of the physical space has a first attribute; and abandoning the provision of the first type of illumination based on determining that the region of the physical space has a second attribute different from the first attribute.

[0008] In some examples, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting a region of a physical space to be illuminated; in response to detecting the region of the physical space to be illuminated: providing a first type of illumination via the light source based on determining that the region of the physical space has a first attribute; and abandoning the provision of the first type of illumination based on determining that the region of the physical space has a second attribute different from the first attribute.

[0009] In some examples, a computer system communicating with a light source is described. In some examples, the computer system communicating with the light source includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programs include instructions for: detecting a request for an area of ​​a physical space to be illuminated; in response to detecting the request for an area of ​​the physical space to be illuminated: providing a first type of illumination via the light source based on determining that the area of ​​the physical space has a first attribute; and abandoning the provision of the first type of illumination based on determining that the area of ​​the physical space has a second attribute different from the first attribute.

[0010] In some examples, a computer system that communicates with a light source is described. In some examples, the computer system that communicates with the light source includes components for performing each of the following steps: detecting a request for an area of ​​a physical space to be illuminated; in response to detecting the request for an area of ​​a physical space to be illuminated: providing a first type of illumination via the light source based on determining that the area of ​​the physical space has a first attribute; and abandoning the provision of the first type of illumination based on determining that the area of ​​the physical space has a second attribute different from the first attribute.

[0011] In some examples, a computer program product is described. In some examples, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting a request for an area of ​​a physical space to be illuminated; in response to detecting the request for an area of ​​the physical space to be illuminated: providing a first type of illumination via a light source based on determining that the area of ​​the physical space has a first attribute; and abandoning the provision of the first type of illumination based on determining that the area of ​​the physical space has a second attribute different from the first attribute.

[0012] In some examples, a method is described that is executed at a computer system communicating with a light source. In some examples, the method includes: detecting a change in user activity in the physical space when a user is detected in the physical space; and, in response to detecting the change in user activity in the physical space, changing the illumination of the physical space via the light source while users continue to be detected in the physical space.

[0013] In some examples, a non-transitory computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting changes in user activity in the physical space when a user is detected in the physical space; and, in response to detecting changes in user activity in the physical space, changing the illumination of the physical space via the light source while a user continues to be detected in the physical space.

[0014] In some examples, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting changes in user activity in the physical space when a user is detected in the physical space; and, in response to detecting changes in user activity in the physical space, changing the illumination of the physical space via the light source while a user continues to be detected in the physical space.

[0015] In some examples, a computer system communicating with a light source is described. In some examples, the computer system communicating with the light source includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programs include instructions for: detecting changes in user activity in the physical space when a user is detected in the physical space; and, in response to detecting changes in user activity in the physical space, changing the illumination of the physical space via a light source while a user continues to be detected in the physical space.

[0016] In some examples, a computer system communicating with a light source is described. In some examples, the computer system communicating with the light source includes components for performing each of the following steps: detecting changes in user activity in the physical space when a user is detected in the physical space; and changing the illumination of the physical space via the light source in response to detecting changes in user activity in the physical space while users continue to be detected in the physical space.

[0017] In some examples, a computer program product is described. In some examples, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting changes in user activity in the physical space when a user is detected in the physical space; and, in response to detecting changes in user activity in the physical space, changing the illumination of the physical space via the light source while a user continues to be detected in the physical space.

[0018] In some examples, a method is described that is executed at a computer system communicating with a light source. In some examples, the method includes: detecting an illumination request corresponding to a corresponding area of ​​an illuminated physical space; and in response to detecting the illumination request: illuminating a first area via a light source based on determining that the request corresponds to a first area of ​​the physical space; and illuminating a second area via a light source based on determining that the request corresponds to a second area of ​​the physical space different from the first area.

[0019] In some examples, a non-transitory computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting an illumination request corresponding to a corresponding area of ​​an illuminated physical space; and in response to detecting the illumination request: illuminating a first area via a light source based on determining that the request corresponds to a first area of ​​the physical space; and illuminating a second area via a light source based on determining that the request corresponds to a second area of ​​the physical space different from the first area.

[0020] In some examples, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting an illumination request corresponding to a corresponding area of ​​an illuminated physical space; and in response to detecting the illumination request: illuminating a first area via a light source based on determining that the request corresponds to a first area of ​​the physical space; and illuminating a second area via a light source based on determining that the request corresponds to a second area of ​​the physical space different from the first area.

[0021] In some examples, a computer system communicating with a light source is described. In some examples, the computer system communicating with the light source includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programs include instructions for: detecting an illumination request corresponding to a corresponding area of ​​an illuminated physical space; and in response to detecting the illumination request: illuminating a first area via a light source based on determining that the request corresponds to a first area of ​​the physical space; and illuminating a second area via a light source based on determining that the request corresponds to a second area of ​​the physical space different from the first area.

[0022] In some examples, a computer system communicating with a light source is described. In some examples, the computer system communicating with the light source includes components for performing each of the following steps: detecting an illumination request corresponding to a corresponding area of ​​an illuminated physical space; and in response to detecting the illumination request: illuminating a first area via the light source based on determining that the request corresponds to a first area of ​​the physical space; and illuminating a second area via the light source based on determining that the request corresponds to a second area of ​​the physical space different from the first area.

[0023] In some examples, a computer program product is described. In some examples, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting an illumination request corresponding to a corresponding area of ​​an illuminated physical space; and in response to detecting the illumination request: illuminating a first area via a light source based on determining that the request corresponds to a first area of ​​the physical space; and illuminating a second area via a light source based on determining that the request corresponds to a second area of ​​the physical space different from the first area.

[0024] In some examples, a method executed at a computer system communicating with a light source is described. In some examples, the method includes: detecting a request to illuminate a region of a first physical space; and in response to detecting the request to illuminate a region of the first physical space: illuminating a region of the first physical space via a light source to include a first abstract representation corresponding to the first scenario of the second physical space, wherein the second physical space is outside the first physical space, based on determining that the second physical space has a second scenario different from the first scenario; and illuminating a region of the first physical space via a light source to include a second abstract representation corresponding to the second scenario of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scenario of the second physical space, based on determining that the second physical space has a second scenario different from the first abstract representation corresponding to the first scenario of the second physical space.

[0025] In some examples, a non-transitory computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting a region of a first physical space to be illuminated; and in response to detecting the region of the first physical space to be illuminated: based on determining that a second physical space has a first scene, illuminating a region of the first physical space via the light source to include a first abstract representation corresponding to the first scene of the second physical space, wherein the second physical space is outside the first physical space; and based on determining that the second physical space has a second scene different from the first scene, illuminating a region of the first physical space via the light source to include a second abstract representation corresponding to the second scene of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scene of the second physical space.

[0026] In some examples, a transient computer-readable storage medium is described that stores one or more programs configured to be executed by one or more processors of a computer system in communication with a light source. In some examples, the one or more programs include instructions for: detecting a region of a first physical space to be illuminated; and in response to detecting the region of the first physical space to be illuminated: based on determining that a second physical space has a first scene, illuminating a region of the first physical space via the light source to include a first abstract representation corresponding to the first scene of the second physical space, wherein the second physical space is outside the first physical space; and based on determining that the second physical space has a second scene different from the first scene, illuminating a region of the first physical space via the light source to include a second abstract representation corresponding to the second scene of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scene of the second physical space.

[0027] In some examples, a computer system communicating with a light source is described. In some examples, the computer system communicating with the light source includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programs include instructions for: detecting a request to illuminate an area of ​​a first physical space; and in response to detecting the request to illuminate an area of ​​the first physical space: illuminating an area of ​​the first physical space via a light source to include a first abstract representation corresponding to the first scene of the second physical space, wherein the second physical space is outside the first physical space, based on determining that the second physical space has a second scene different from the first scene; and illuminating an area of ​​the first physical space via a light source to include a second abstract representation corresponding to the second scene of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scene of the second physical space, based on determining that the second physical space has a second scene different from the first abstract representation corresponding to the first scene of the second physical space.

[0028] In some examples, a computer system communicating with a light source is described. In some examples, the computer system communicating with the light source includes components for performing each of the following steps: detecting a request to illuminate a region of a first physical space; and in response to detecting the request to illuminate a region of the first physical space: illuminating a region of the first physical space via the light source to include a first abstract representation corresponding to the first scenario of the second physical space, wherein the second physical space is outside the first physical space, based on determining that the second physical space has a second scenario different from the first scenario; and illuminating a region of the first physical space via the light source to include a second abstract representation corresponding to the second scenario of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scenario of the second physical space.

[0029] In some examples, a computer program product is described. In some examples, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with a light source. In some examples, the one or more programs include instructions for: detecting a region of a first physical space to be illuminated; and in response to detecting the region of the first physical space to be illuminated: based on determining that a second physical space has a first scene, illuminating the region of the first physical space via the light source to include a first abstract representation corresponding to the first scene of the second physical space, wherein the second physical space is outside the first physical space; and based on determining that the second physical space has a second scene different from the first scene, illuminating the region of the first physical space via the light source to include a second abstract representation corresponding to the second scene of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scene of the second physical space.

[0030] In some examples, a method is described that is executed at a computer system communicating with a first device and a light source separate from the first device. In some examples, the method includes: receiving a request to extend content to be displayed on the first device into a physical space comprising a first region and a second region different from the first region; and in response to receiving the request to extend the content displayed on the first device and when the content is displayed on the first device: illuminating, via a light source, a first region of the physical space having a corresponding spatial arrangement relative to the first position in the physical space with a first light pattern based on the content displayed on the first device, without illuminating the second region of the physical space with the first light pattern via the light source; and illuminating, via a light source, a second region of the physical space having a corresponding spatial arrangement relative to the second position in the physical space with the first light pattern based on the content displayed on the first device, when the first device is determined to be located in a second position in the physical space.

[0031] In some examples, a non-transitory computer-readable storage medium is described, which stores one or more programs executed by one or more processors of a computer system configured to communicate with a first device and a light source separate from the first device. In some examples, the one or more programs include instructions for: receiving a request to expand content to be displayed on the first device into a physical space comprising a first region and a second region different from the first region; and in response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: illuminating, according to determining that the first device is located at a first position in the physical space, a first region of the physical space having a corresponding spatial arrangement relative to the first position in the physical space via a light source using a first light pattern based on the content displayed on the first device, without illuminating the second region of the physical space with the first light pattern via a light source; and illuminating, according to determining that the first device is located at a second position in the physical space, a second region of the physical space having a corresponding spatial arrangement relative to the second position in the physical space via a light source using the first light pattern based on the content displayed on the first device.

[0032] In some examples, a transient computer-readable storage medium is described that stores one or more programs executed by one or more processors of a computer system configured to communicate with a first device and a light source separate from the first device. In some examples, the one or more programs include instructions for: receiving a request to expand content to be displayed on the first device into a physical space comprising a first region and a second region different from the first region; and in response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: illuminating, via a light source, a first region of the physical space having a corresponding spatial arrangement relative to the first position in the physical space with a first light pattern based on the content displayed on the first device, without illuminating the second region of the physical space with the first light pattern via the light source, based on a first position in the physical space; and illuminating, via a light source, a second region of the physical space having a corresponding spatial arrangement relative to the second position in the physical space with the first light pattern based on the content displayed on the first device, based on a second position in the physical space.

[0033] In some examples, a computer system communicating with a first device and a light source separate from the first device is described. In some examples, the computer system communicating with the first device and a light source separate from the first device includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors. In some examples, the one or more programs include instructions for: receiving a request to expand content to be displayed on the first device into a physical space comprising a first region and a second region different from the first region; and in response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: based on determining that the first device is located at a first position in the physical space, illuminating via the light source a first region of the physical space having a corresponding spatial arrangement relative to the first position in the physical space with a first light pattern based on the content displayed on the first device, without illuminating the second region of the physical space with the first light pattern via the light source; and based on determining that the first device is located at a second position in the physical space, illuminating via the light source a second region of the physical space having a corresponding spatial arrangement relative to the second position in the physical space with the first light pattern based on the content displayed on the first device.

[0034] In some examples, a computer system communicating with a first device and a light source separate from the first device is described. In some examples, this computer system communicating with the first device and a light source separate from the first device includes components for performing each of the following steps: receiving a request to expand content to be displayed on the first device into a physical space comprising a first region and a second region different from the first region; and in response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: based on determining that the first device is located at a first position in the physical space, illuminating via the light source a first region of the physical space having a corresponding spatial arrangement relative to the first position in the physical space with a first light pattern based on the content displayed on the first device, without illuminating the second region of the physical space with the first light pattern via the light source; and based on determining that the first device is located at a second position in the physical space, illuminating via the light source a second region of the physical space having a corresponding spatial arrangement relative to the second position in the physical space with the first light pattern based on the content displayed on the first device.

[0035] In some examples, a computer program product is described. In some examples, the computer program product includes one or more programs configured to be executed by one or more processors of a computer system that communicates with a first device and a light source separate from the first device. In some examples, the one or more programs include instructions for: receiving a request to expand content to be displayed on the first device into a physical space comprising a first region and a second region different from the first region; and in response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: based on determining that the first device is located at a first position in the physical space, illuminating via the light source a first region of the physical space having a corresponding spatial arrangement relative to the first position in the physical space with a first light pattern based on the content displayed on the first device, without illuminating the second region of the physical space with the first light pattern via the light source; and based on determining that the first device is located at a second position in the physical space, illuminating via the light source a second region of the physical space having a corresponding spatial arrangement relative to the second position in the physical space with the first light pattern based on the content displayed on the first device.

[0036] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0037] Therefore, faster and more efficient methods and interfaces are provided for devices to deliver lighting effects, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace other methods used to deliver lighting effects. Attached Figure Description

[0038] To better understand the various described embodiments, reference should be made to the following detailed description in conjunction with the accompanying drawings, in which similar reference numerals indicate corresponding parts throughout the drawings.

[0039] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some implementation schemes.

[0040] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0041] Figure 2 Examples of portable multi-functional devices with touchscreens according to some implementation schemes are shown.

[0042] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0043] Figure 4A An exemplary user interface for a menu applied to a portable multi-functional device, according to some implementation schemes, is illustrated.

[0044] Figure 4B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is illustrated according to some embodiments.

[0045] Figure 5A Examples of personal electronic devices according to some implementation schemes are shown.

[0046] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.

[0047] Figures 6A to 6B Exemplary techniques for context-aware lighting are illustrated according to some examples.

[0048] Figure 7 This is a flowchart illustrating, based on some examples, a method for providing context-aware lighting.

[0049] Figures 8A to 8E Exemplary techniques for changing lighting based on detected user activity are illustrated in several examples.

[0050] Figure 9This is a flowchart illustrating methods for changing lighting, based on some examples.

[0051] Figures 10A to 10E Exemplary techniques for using lighting location to transmit information are illustrated according to some examples.

[0052] Figure 11 This is a flowchart illustrating methods for transmitting information based on some examples.

[0053] Figures 12A to 12D Exemplary techniques for providing a representation of a scenario for physical space are illustrated according to some examples.

[0054] Figure 13 This is a flowchart illustrating methods for providing scenarios based on some examples.

[0055] Figures 14A to 14D Exemplary techniques for extending content into physical space are illustrated according to some examples.

[0056] Figure 15 This is a flowchart illustrating methods for expanding content based on some examples. Detailed Implementation

[0057] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0058] There is a need for electronic devices that provide efficient methods and interfaces for delivering lighting effects. For example, light can be used to provide context-aware lighting, react to detected user activity, convey information, and / or expand content. Such technologies can reduce the cognitive burden on users in the physical environment, thereby increasing productivity. Furthermore, such technologies can reduce processor and battery power that would otherwise be wasted on redundant user input.

[0059] under Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B , Figures 5A to 5B A description of an exemplary device for performing techniques that provide lighting effects is provided. Figures 6A to 6B Exemplary techniques for context-aware lighting are illustrated according to some examples. Figure 7 This is a flowchart illustrating, based on some examples, a method for providing context-aware lighting. Figures 6A to 6B The user interface in the document is used to illustrate the process described below, including Figure 7 The process in. Figures 8A to 8E Exemplary techniques for changing lighting based on detected user activity are illustrated in several examples. Figure 9This is a flowchart illustrating methods for changing lighting, based on some examples. Figures 8A to 8E The user interface in the document is used to illustrate the process described below, including Figure 9 The process in. Figures 10A to 10E Exemplary techniques for using lighting location to transmit information are illustrated according to some examples. Figure 11 This is a flowchart illustrating methods for transmitting information based on some examples. Figures 10A to 10E The user interface in the document is used to illustrate the process described below, including Figure 11 The process in. Figures 12A to 12D Exemplary techniques for providing a representation of a scenario for physical space are illustrated according to some examples. Figure 13 This is a flowchart illustrating methods for providing scenarios based on some examples. Figures 12A to 12D The user interface in the document is used to illustrate the process described below, including Figure 13 The process in. Figures 14A to 14D Exemplary techniques for extending content into physical space are illustrated according to some examples. Figure 15 This is a flowchart illustrating methods for expanding content based on some examples. Figures 14A to 14D The user interface in the document is used to illustrate the process described below, including Figure 15 The process in.

[0060] The processes described below enhance device operability and make user-device interfaces more efficient through various technologies (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), including providing improved visual feedback to users, reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations when a set of conditions are met without requiring further user input and / or additional technologies. These technologies also reduce power consumption and extend device battery life by enabling users to use the device more quickly and efficiently.

[0061] Furthermore, in a method described herein where one or more steps depend on the satisfaction of one or more conditions, it should be understood that the described method may be repeated in multiple repetitions such that, during the repetitions, all conditions determining the steps in the method are satisfied in different repetitions of the method. For example, if the method requires performing a first step (if the conditions are satisfied) and a second step (if the conditions are not satisfied), those skilled in the art will know that the stated steps are repeated until both conditions are satisfied and not satisfied (in no particular order). Thus, a method described as having one or more steps depending on the satisfaction of one or more conditions can be rewritten as a method that repeats until each condition described in the method is satisfied. However, this does not require the system or computer-readable medium to declare that the system or computer-readable medium contains instructions for performing discretionary operations based on the satisfaction of the corresponding one or more conditions, and thus to determine whether possible conditions have been satisfied without explicitly repeating the steps of the method until all conditions determining the steps in the method are satisfied. Those skilled in the art will also understand that, similar to a method having discretionary steps, a system or computer-readable storage medium may repeat the steps of the method multiple times as needed to ensure that all discretionary steps have been performed.

[0062] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. In some embodiments, these terms are used to distinguish one element from another. For example, a first touch may be referred to as a second touch, and similarly, a second touch may be referred to as a first touch, without departing from the scope of the various described embodiments. In some embodiments, a first touch and a second touch are two separate references to the same touch. In some embodiments, both a first touch and a second touch are touches, but they are not the same touch.

[0063] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and in the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context expressly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0064] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "in response to," or "in response to detection." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "in response to determining..." or "in response to detecting [the stated condition or event]."

[0065] This document describes embodiments of electronic devices, user interfaces of such devices, and associated processes for using such devices. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functionalities such as PDA and / or music player functionality. Exemplary embodiments of portable multi-functional devices include, but are not limited to, iPhone® devices, iPod Touch® devices, and iPad® devices from Apple Inc. (Cupertino, California). Optionally, other portable electronic devices, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads), may be used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). In some embodiments, the electronic device is a computer system that communicates with a display generating component (e.g., via wireless or wired communication). The display generating component is configured to provide visual output, such as display via a CRT monitor, via an LED monitor, or via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. As used herein, “display” content includes content (e.g., video data rendered or decoded by display controller 156) that is visually generated by sending data (e.g., image data or video data) to an integrated or external display generating component via a wired or wireless connection.

[0066] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device may optionally include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0067] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0068] Various applications running on this device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or within the respective applications. In this way, the common physical architecture of the device (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0069] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1A This is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as or called a “touch-sensitive display system.” Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic outputs on device 100 (e.g., generating haptic outputs on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0070] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute (alternative) for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values ​​that includes at least four different values ​​and more typically hundreds of different values ​​(e.g., at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface are optionally used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or change of the contact area detected on the touch-sensitive surface, the capacitance and / or change of the touch-sensitive surface adjacent to the contact, and / or the resistance and / or change of the touch-sensitive surface adjacent to the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of the contact as an attribute of user input allows users to access additional device functionality that would otherwise be inaccessible to the user on a smaller device with limited physical space, which is used (e.g., on a touch-sensitive display) to display an indication and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0071] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with the touch-sensitive surface, which has been physically pressed (e.g., displaced) by the user's movement, does not move. As another example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or perceived by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of the user (e.g., "release click", "press click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception described by a typical (or common) user.

[0072] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.

[0073] Memory 102 optionally includes high-speed random access memory, and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0074] Peripheral interface 118 can be used to couple the device's input and output peripherals to CPU 120 and memory 102. The one or more processors 120 run or execute various software programs (such as computer programs (e.g., including instructions)) and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0075] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, subscriber identity module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks (such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs))) and other devices. RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as via short-range communication radio components. Wireless communication may optionally employ any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution, Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)) and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.

[0076] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and sends the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and sends the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or sent to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both outputs (e.g., a single-ear or dual-ear headset) and inputs (e.g., a microphone).

[0077] I / O subsystem 106 couples input / output peripherals (such as touchscreen 112 and other input control devices 116) on device 100 to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, depth camera controller 169, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / transmit electrical signals to the other input control device 116. Other input control devices 116 optionally include physical buttons (e.g., push-buttons, rocker buttons, etc.), dials, slide switches, joysticks, click dials, etc. In some embodiments, input controller 160 is optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing device such as mouse. One or more buttons (e.g., ... Figure 2 Optionally, 208) includes an increase / decrease button for volume control of speaker 111 and / or microphone 113. The one or more buttons optionally include a push-button (e.g., Figure 2(Ref. 206 in the original text). In some embodiments, the electronic device is a computer system that communicates with one or more input devices (e.g., via wireless communication, via wired communication). In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a touchpad, as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), such as for tracking user gestures (e.g., hand gestures and / or air gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system. In some implementations, air gestures are gestures detected without the user touching an input element that is part of the device (or independently of an input element that is part of the device) and based on the detected movement of a part of the user's body through the air (including movement of the user's body relative to an absolute reference (e.g., the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement relative to another part of the user's body (e.g., movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to the user's other hand, and / or movement of the user's fingers relative to another of the user's fingers or a part of the user's hand), and / or absolute movement of a part of the user's body (e.g., a tapping gesture that includes the hand moving a predetermined amount and / or rate in a predetermined pose, or a shaking gesture that includes a predetermined rate or amount of rotation of a part of the user's body)).

[0078] A quick press of a push button optionally disengages the touchscreen 112 from its lock or optionally initiates a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent No. 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of a push button (e.g., 206) optionally powers the device 100 on or off. The functionality of one or more of these buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0079] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0080] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.

[0081] Touchscreen 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology, such as that used in the iPhone® and iPod Touch® from Apple Inc. (Cupertino, California), is used.

[0082] In some embodiments of the touchscreen 112, the touch-sensitive display optionally resembles a multi-touch-sensitive touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.

[0083] The touch-sensitive display in some embodiments of the touchscreen 112 is described in the following applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, “Multipoint Touch Surface Controller”; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, “Multipoint Touchscreen”; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, “Mode-Based Graphical User Interfaces For Touch Sensitive Input”. (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, “Virtual Input Device Placement On A Touch Screen User Interface”; (7) U.S. Patent Application No. 11 / 228,700, filed September 16, 2005, “Operation Of A Computer With A Touch Screen Interface”; (8) U.S. Patent Application No. 11 / 228,737, filed September 16, 2005, “Activating Virtual Keys Of ATouch-Screen Virtual Keyboard”; and (9) U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, “Multi-Functional Hand-Held Device”. The full text of all these applications is incorporated herein by reference.

[0084] Touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users optionally use any suitable object or accessory such as a stylus, finger, etc., to interact with touchscreen 112. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positioning or commands for performing the user-desired actions.

[0085] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. Optionally, the touchpad is a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0086] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power fault detection circuitry, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0087] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 is shown coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also referred to as a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, facing away from a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the positioning of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensors within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.

[0088] The device 100 optionally also includes one or more depth camera sensors 175. Figure 1A A depth camera sensor is shown coupled to a depth camera controller 169 in I / O subsystem 106. Depth camera sensor 175 receives data from the environment to create a 3D model of an object (e.g., a face) within the scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with imaging module 143 (also referred to as camera module), depth camera sensor 175 is optionally used to determine depth maps of different portions of an image captured by imaging module 143. In some embodiments, the depth camera sensor is located at the front of device 100, such that user images with depth information are optionally acquired for video conferencing while a user views other video conferencing participants on a touchscreen display, and selfies with depth map data are captured. In some embodiments, depth camera sensor 175 is located at the rear of the device, or both the rear and front of device 100. In some embodiments, the positioning of depth camera sensor 175 can be changed by the user (e.g., by rotating a lens and sensor within the device housing), such that depth camera sensor 175 is used in conjunction with a touchscreen display for both video conferencing and still image and / or video image acquisition.

[0089] In some implementations, the depth map (e.g., a depth map image) contains information (e.g., values) relating to the distance of objects in the scene from the viewpoint (e.g., a camera, optical sensor, depth camera sensor). In one implementation of the depth map, each depth pixel defines the location of its corresponding two-dimensional pixel on the Z-axis of the viewpoint. In some implementations, the depth map is composed of pixels, where each pixel is defined by a value (e.g., 0 to 255). For example, a "0" value represents the pixel furthest from the viewpoint (e.g., a camera, optical sensor, depth camera sensor) in the "3D" scene, and a "255" value represents the pixel closest to the viewpoint in the "3D" scene. In other implementations, the depth map represents the distance between objects in the scene and the plane of the viewpoint. In some implementations, the depth map includes information about the relative depth of various features of the object of interest within the field of view of the depth camera (e.g., the relative depth of the eyes, nose, mouth, and ears of a user's face). In some implementations, the depth map includes information that enables the device to determine the contour of the object of interest along the z-direction.

[0090] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1A A contact strength sensor is shown coupled to a strength sensor controller 159 in I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0091] The device 100 optionally also includes one or more proximity sensors 166. Figure 1AA proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may optionally be configured as described in the following U.S. patent applications: 11 / 241,839, entitled "Proximity Detector In Handheld Device"; 11 / 240,788, entitled "Proximity Detector In Handheld Device"; 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", the entire contents of which are incorporated herein by reference. In some implementations, when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call), the proximity sensor is turned off and the touchscreen 112 is disabled.

[0092] The device 100 may optionally also include one or more tactile output generators 167. Figure 1A A haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components for converting electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.

[0093] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral device interface 118 is shown. Alternatively, the accelerometer 168 may be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may optionally be configured as described in the following U.S. Patent Publications: 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and 20060017692, entitled "Methods And Apparatuses For Operating A Portable DeviceBased On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. Device 100 may optionally include, in addition to the accelerometer 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation (e.g., portrait or landscape) of device 100.

[0094] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the figure. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, indicating what applications, views or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from various sensors and input control devices 116 of the device; and position information relating to the device's position and / or orientation.

[0095] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0096] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, Wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used on iPod® (Apple Inc. trademark) devices.

[0097] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., touchpads or physical click-based rotary dials). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0098] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by the user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).

[0099] The touch / motion module 130 optionally detects gesture input performed by the user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes: detecting a finger press event, and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes: detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.

[0100] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other displays, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0101] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from applications, etc., to specify the graphics to be displayed, and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data to output to the display controller 156.

[0102] The haptic feedback module 133 includes various software components for generating instructions which are used by the haptic output generator 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0103] The text input module 134, optionally a component of the graphics module 132, provides a soft keyboard for entering text in various applications, such as contacts 137, email 140, IM 141, browser 147, and any other application that requires text input.

[0104] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to telephone 138 for use in location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services, such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0105] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof: • Contacts module 137 (sometimes called address book or contact list); • Telephone module 138; • Video conferencing module 139; • Email client module 140; • Instant Messaging (IM) module 141; • Fitness support module 142; • Camera module 143 for still images and / or video images; • Image management module 144; • Video player module; • Music player module; • Browser module 147; • Calendar module 148; • Widget module 149, which optionally includes one or more of the following: weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5 and other widgets acquired by the user, as well as user-created widget 149-6; • Widget creator module 150 for creating user-created widgets 149-6; • Search module 151; • Video and music player module 152, which combines a video player module and a music player module; •Notepad module 153; • Map module 154; and / or • Online video module 155.

[0106] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0107] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing module 139, email 140, or IM 141; and so on.

[0108] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify input telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0109] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0110] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0111] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for performing the following operations: entering a character sequence corresponding to an instant message, modifying previously entered characters, sending a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing received instant messages. In some embodiments, the sent and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages delivered using SMS or MMS) and internet-based messages (e.g., messages delivered using XMPP, SIMPLE, or IMPS).

[0112] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for performing the following operations: creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (exercise equipment); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0113] In conjunction with the touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for performing the following operations: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0114] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for performing operations such as arranging, modifying (e.g., editing) or otherwise manipulating, marking, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0115] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for performing the following operations: browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as links to attachments and other files on web pages.

[0116] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0117] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is optionally a micro-application downloaded and used by a user (e.g., weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5) or a user-created micro-application (e.g., user-created widget 149-6). In some embodiments, the widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the widget includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! widgets).

[0118] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, widget creator module 150 is optionally used by the user to create widgets (e.g., turning user-specified portions of a webpage into widgets).

[0119] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for performing the following operations: searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0120] Combining touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0121] Combining the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.

[0122] Combining RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135 and browser module 147, map module 154 is optionally used to receive, display, modify and store maps and data associated with the maps (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.

[0123] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following operations: allowing users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Additional descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are incorporated herein by reference in their entirety.

[0124] Each of the modules and applications described above corresponds to an executable set of instructions for performing one or more functions described above and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A (e.g., video and music player module 152). In some embodiments, memory 102 optionally stores a subset of the modules and data structures identified above. Additionally, memory 102 optionally stores additional modules and data structures not described above.

[0125] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (such as push-buttons and dial pads) on device 100 is optionally reduced.

[0126] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, main desktop menu, or root menu. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0127] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any of the aforementioned applications 137 to 151, 155, 380 to 390).

[0128] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0129] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0130] Event monitor 171 receives event information from peripheral interface 118. The event information includes information about sub-events, such as a user touch on touch-sensitive display 112 as part of a multi-touch gesture. Peripheral interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.

[0131] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 sends event information. In other implementations, peripheral device interface 118 sends event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0132] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0133] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0134] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined, at least in part, based on the hit view of the initial touch that initiates the touch-based gesture.

[0135] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0136] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0137] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0138] In some implementations, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another implementation, event classifier 170 is a separate module or part of another module (such as contact / motion module 130) stored in memory 102.

[0139] In some implementations, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other implementations, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some implementations, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0140] The corresponding event identifier 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event identifier 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0141] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device orientation).

[0142] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in events (e.g., 187-1 and / or 187-2) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on the displayed object, movement of the touch on the touch-sensitive display 112, and lifting the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0143] In some implementations, event definition 186 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0144] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0145] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0146] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0147] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from delivering (and deferred delivering) the sub-events to the corresponding hit view. In some implementations, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag acquires the flag and performs a predefined process.

[0148] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined process.

[0149] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the positioning of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and transmits that display information to graphics module 132 for display on a touch-sensitive display.

[0150] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0151] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0152] Figure 2A portable multifunction device 100 with a touchscreen 112 is illustrated according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0153] Device 100 optionally also includes one or more physical buttons, such as a "main desktop" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0154] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headphone jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to: power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0155] Figure 3This is a block diagram of an exemplary multi-functional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the reference above). Figure 1A The described tactile output generator 167), sensor 359 (e.g., optical sensor, accelerometer, proximity sensor, touch sensor and / or contact intensity sensor (similar to the one described above)) Figure 1A The described contact strength sensor 165). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores information related to portable multifunction device 100. Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.

[0156] Figure 3Each of the elements described above is optionally stored in one or more memory devices of the previously mentioned memory devices. Each module described above corresponds to a set of instructions for performing the functions described above. The modules or computer programs described above (e.g., instruction sets or including instructions) need not be implemented as separate software programs (such as computer programs (e.g., including instructions)), processes, or modules, and therefore various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0157] Now let’s turn our attention to the implementation of the user interface, which is optionally implemented on, for example, a portable multifunction device 100.

[0158] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is illustrated. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof: • Signal strength indicator 402 for wireless communications such as cellular signals and Wi-Fi signals; • Time 404; • Bluetooth indicator 405; • Battery status indicator 406; • Tray 408 features icons for frequently used applications, such as: ○ The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemail messages; ○ An icon 418 labeled "Mail" in the email client module 140, which optionally includes an indicator 410 for the number of unread emails; ○ The icon 420 labeled "Browser" in browser module 147; and ○ The video and music player module 152 (also known as the iPod (Apple Inc. trademark) module 152) is marked with an icon 422 labeled "iPod"; and • Icons of other applications, such as: ○The icon 424 of the IM module 141 marked as "Message"; ○The calendar module 148 has an icon 426 labeled "Calendar"; ○ The icon 428 of the image management module 144 is labeled "Photo". ○ The icon 430 of camera module 143, which is labeled "camera"; ○ The icon 432 of the online video module 155, which is labeled "Online Video"; ○ The icon 434 labeled "Stock Market" in the Stock Market widget 149-2; ○The icon 436 of the map module 154 that is labeled "map"; ○The weather widget 149-1 has icon 438 labeled "weather"; ○ The alarm clock widget 149-4 has an icon 440 labeled "clock"; ○ The icon 442 of the fitness support module 142 is labeled "fitness support"; ○ The icon 444 labeled "Notepad" in Notepad module 153; and ○ Set an icon 446 labeled "Settings" for an application or module, which provides access to settings for device 100 and its various applications 136.

[0159] It should be pointed out that, Figure 4A The illustrated icon labels are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0160] Figure 4B An example is illustrated having a touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)). Figure 3 A tablet device or touchpad 355) device (e.g., Figure 3 An exemplary user interface on the device 300. The device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of the sensors 359) for detecting the intensity of contact on the tactile surface 451 and / or one or more tactile output generators 357 for generating tactile outputs for the user of the device 300.

[0161] While some examples of input on a touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations the device detects input on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450) corresponding to the main axis on the display (e.g., Figure 4B The main shaft of 453 in the middle (e.g., Figure 4B(452 in the example). According to these embodiments, the device detects the position corresponding to a specific location on the display (e.g., in...). Figure 4B In the diagram, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, when the touch-sensitive surface (e.g., ...) Figure 4B 451 in the middle) and the display of a multi-functional device (e.g., Figure 4B When 450 is separated from the touch-sensitive surface, user input detected by the device on that touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may be optionally used for other user interfaces described herein.

[0162] Additionally, while the examples below are given primarily with reference to finger input (e.g., finger touch, single-finger tap gesture, finger swipe gesture), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the touch). As another example, a tap gesture may optionally be replaced by a mouse click while the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0163] Figure 5A An exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include components relative to devices 100 and 300 (e.g., Figures 1A to 4B The device 500 may include some or all of the features described herein. In some embodiments, the device 500 has a touch-sensitive display 504, referred to below as a touchscreen 504. Alternatively, or in addition to the touchscreen 504, the device 500 may also have a display and a touch-sensitive surface. Similar to the cases of devices 100 and 300, in some embodiments, the touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). The one or more intensity sensors of the touchscreen 504 (or touch-sensitive surface) may provide output data representing the intensity of the touch. The user interface of the device 500 may respond to touches based on the intensity of the touch, meaning that touches of different intensities may invoke different user interface operations on the device 500.

[0164] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying UserInterface Objects Corresponding to an Application,” published as WIPO Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between TouchInput to Display Output Relationships,” published as WIPO Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.

[0165] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508, if included, can be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0166] Figure 5B An exemplary personal electronic device 500 is depicted. In some embodiments, device 500 may include, relative to... Figure 1A , Figure 1B and Figure 3Some or all of the components described. Device 500 has a bus 512 that operatively couples I / O portion 514 to one or more computer processors 516 and memory 518. I / O portion 514 may be connected to display 504, which may have touch-sensitive component 522 and optionally have intensity sensor 524 (e.g., contact intensity sensor). Furthermore, I / O portion 514 may be connected to communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 is optionally a rotatable input device. In some examples, input mechanism 508 is optionally a button.

[0167] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operatively connected to the I / O section 514.

[0168] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, may cause the computer processors to perform techniques including methods 700, 900, 1100, 1300, and 1500, for example. Figure 7 , Figure 9 , Figure 11 , Figure 13 and Figure 15 A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices 500 are not limited to... Figure 5B It can be the components and configurations, or it can include other components or additional components in a variety of configurations.

[0169] As used herein, the term "power indication" refers optionally to the power indication in devices 100, 300, and / or 500 ( Figure 1A , Figure 3 and Figures 5A to 5BA user-interactive graphical user interface object displayed on a screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) optionally each constitute a functional representation.

[0170] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of direct interaction with user interface elements on a touchscreen display) that enable direct interaction with user interface elements on the touchscreen display. Figure 1A The touch-sensitive display system 112 or Figure 4A In some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of ​​the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that delivers the user-expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0171] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted, before or after contact begins to move, before contact ends, before or after contact intensity is detected to increase, and / or before or after contact intensity decreases). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half maximum value of the contact intensity, or the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the contact intensity over time). In some implementations, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether a user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a characteristic intensity not exceeding the first threshold results in a first action, contact with a characteristic intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a characteristic intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the characteristic intensity and one or more thresholds is used to determine whether one or more actions should be performed (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.

[0172] As used herein, "installed application" refers to a software application that has been downloaded to an electronic device (e.g., device 100, 300, and / or 500) and is ready to be launched on the device (e.g., become open). In some implementations, the downloaded application becomes an installed application using an installer that extracts program portions from the downloaded software package and integrates the extracted portions with the computer system's operating system.

[0173] As used herein, the terms "open application" or "running application" refer to a software application that maintains state information (e.g., as part of device / global internal state 157 and / or application internal state 192). An open or running application is optionally any of the following types of applications: • Active app, which is currently displayed on the screen of the device that is using the app; • Background applications (or background processes) that are not currently displayed but whose one or more processes are being handled by one or more processors; and • Suspended or hibernating applications that are not currently running but have state information stored in memory (either volatile or non-volatile) that can be used to resume the application's execution.

[0174] As used herein, the term "closed application" refers to a software application that does not retain state information (e.g., the state information of a closed application is not stored in the device's memory). Therefore, closing an application includes: stopping and / or removing the application's process and removing the application's state information from the device's memory. Generally, opening a second application while the first application is running does not close the first application. When the second application is displayed and the first application stops displaying, the first application becomes a background application.

[0175] Now let’s turn our attention to the implementation of user interfaces (“UIs”) on electronic devices, such as portable multifunction devices 100, 300 or 500, and the associated processes.

[0176] Figures 6A to 6B Exemplary techniques for scene-aware lighting, according to some examples, are illustrated. The user interface in these figures is used to illustrate the processes described below, including... Figure 7 The process in.

[0177] Figure 6AA physical space 600, a room in a house, is illustrated. Physical space 600 includes light sources 601A, 601B, 601C, and 601D (collectively referred to below as light sources 601) to illuminate physical space 600. It should be understood that light sources 601 (including only one light source) may include more or fewer light sources, and the group of light sources can be arranged in any physical configuration. In some examples, there are no physical limitations on the physical placement, separation, orientation, and / or number of light sources. In some examples, each light source in light source 601 has a separate housing (e.g., as shown in 601A, 601B, 601C, and 601D). In other examples, one or more light sources in light source 601 share a common housing (e.g., a single luminaire for light sources 601A, 601B, 601C, and / or 601D). In some examples, performing the techniques described herein with a single light source and / or a single luminaire is easier to install, construct, move, and / or replace than performing the techniques described herein with multiple light sources and / or luminaires. In some examples, light source 601 includes one or more features described herein with respect to any one or more light sources, as described in Figures 8, 10, 12 and / or 14. In some examples, physical space 600 is the physical space of an area within another type of building, such as a hotel, office and / or enterprise.

[0178] In some examples, the light source (e.g., 601A, 601B, 601C, and / or 601D) includes one or more features of portable multifunction device 100, device 300, and / or device 500. For example, the light source may include or communicate with one or more processors and memories for storing and / or executing one or more instructions for performing the processes described herein. In some examples, one or more processors cause one or more light sources (e.g., 601) to perform operations (e.g., detecting input, illuminating an area, and / or determining properties of a physical space). In some examples, one or more processors communicate with one or more light sources (e.g., 601). In some examples, one or more processors are separate from one or more light sources (e.g., 601).

[0179] In some examples, light source 601 communicates with one or more other devices (e.g., computer systems). For example, light source 601 may communicate with one or more sensor devices (e.g., sensor devices that sense one or more properties of a physical space (e.g., physical space 600) and / or the environment). As another example, light source 601 may communicate with one or more processing devices (e.g., one or more processing devices that process sensor data, determine illumination levels, and / or process inputs that assist and / or instruct light source 601 to output illumination, as described in the examples described herein). For simplicity, various operations (e.g., outputting illumination, detecting inputs, and / or determining properties) are described below as being performed by light source 601. However, it should be recognized that one or more operations described below may be performed by devices other than light source 601, such as personal computing devices (e.g., telephones, tablets, laptops, desktop computers, and / or wearable devices) or public devices (e.g., smart speakers, televisions, routers, and / or hubs). Unless otherwise explicitly stated, this description should not be construed as limiting the scope of such operations performed by a single device (e.g., light source 601) or a specific combination of devices.

[0180] like Figure 6A As shown, physical space 600 includes several physical features (e.g., people, physical objects, and / or physical structures) that physically exist in and / or form part of the room. Such physical features include walls 612, floors 614, and windows 616, each of which is a physical feature constituting physical space 600 (e.g., defining the boundaries of physical space 600 and / or existing in physical space). Figure 6A Examples also include physical features such as display device 618 and person 620, each of which is a movable physical feature existing in physical space 600.

[0181] As described above, light source 601 can be used to intelligently illuminate physical space 600. In some examples, light source 601 provides illumination based on one or more properties of physical space 600 (e.g., physical characteristics). In some examples, light source 601 provides illumination in response to a request (e.g., in response to the detection of input, such as user input). In some examples, light source 601 provides illumination automatically based on one or more properties of physical space 600 (e.g., without detecting input, such as user input) (e.g., intelligently illuminating physical space 600 and / or a portion thereof (e.g., an area)). In some examples, properties of physical space 600 include physical characteristics of physical space 600, physical properties of physical characteristics (e.g., location, orientation, color, material, reflectivity, and / or opacity), and / or the context of physical space 600 (e.g., events occurring within physical space 600, user activity, weather, and / or time of day). The examples of properties provided are not intended to be exhaustive but are merely illustrative. Other examples described herein are intended to include those listed above.

[0182] In some examples, light source 601 provides illumination to a region of physical space 600 based on one or more properties of that region. In such examples, a region of physical space 600 may be a portion of the area and / or volume of physical space 600 (e.g., less than all and / or a subset).

[0183] In the various figures of this disclosure, reference is made to regions exemplified by dashed line boundaries. Unless otherwise expressly stated herein, dashed line boundaries are visual aids indicating the location and / or extent of the corresponding regions and should not be construed as being output by a device (e.g., light source 601) or otherwise visible within the corresponding physical space. For example, Figure 6A A physical space 600 is illustrated, comprising region 622 (e.g., a portion of wall 612), region 624 (e.g., a portion of floor 614), region 626 (e.g., a portion of wall 612 including window 616), region 628 (e.g., a portion of physical space 600 including display device 618), and region 630 (e.g., a portion of physical space 600 including the face of person 620). In some examples, light source 601 receives data representing regions of physical space 600 from one or more sensors (e.g., inside and / or outside light source 601). In some examples, the data representing regions includes data from one or more other devices (e.g., data received from another computing system such as another light source, server, and / or personal computing device).

[0184] In some examples, one or more properties of a specific area do not affect the lighting of another area of ​​physical space 600 and / or parts of physical space 600 outside the specific area. For example, the color of floor 614 may not affect the lighting of wall 612.

[0185] exist Figure 6AIn this embodiment, light source 601 detects a request to illuminate region 622 within physical space 600. In response to the detected request, light source 601 illuminates region 622 based on one or more detected attributes of the walls 612 within region 622. In some examples, attributes represent reflectivity (e.g., the reflectivity of a mirror, polished stone surface, and / or reflective metal surface), transparency (e.g., the transparency of a glass or plastic window, glass or plastic door, and / or glass or plastic table), color, material, time of day, object type, frequency of use of the object or region, user activity, the presence and / or absence of one or more people in the region, and / or the presence and / or absence of one or more people's faces in the region. In some examples, detecting a request to illuminate region 622 causes light source 601 to illuminate region 622 with light having a color and / or brightness determined based on the color of the walls 612. Thus, in some examples, light source 601 illuminates wall 612 with a color of light that will cause wall 612 to present its true color to a viewer (e.g., reducing and / or eliminating the effect of the color of the illumination on the viewer's perception of the object's color). In some examples, detecting a request to illuminate area 624 causes light source 601 to illuminate area 624 with light having a brightness determined based on the color of floor 614. In some examples, detecting a request to illuminate area 626 causes light source 601 to illuminate area 626 with light having a color and / or brightness determined based on the transparency of window 616 (e.g., light source 601 may reduce the illumination within area 626 to reduce internal glare from window 616 and / or avoid unnecessary energy use involved in generating illumination that will exit through window 616). In some examples, detecting a request to illuminate area 628 causes light source 601 to illuminate area 628 with light having a brightness determined based on the display device 618 identified as including surfaces for output content (e.g., screen and / or monitor) (e.g., light source 601 may reduce and / or lower the illumination within area 628 to reduce glare on the screen of display device 618 that would interfere with the viewer (e.g., person 620). In some examples, detecting a request to illuminate region 630 causes light source 601 to illuminate region 630 with light having a brightness determined based on identifying that region 630 includes a face. In some examples, light source 601 reduces the illumination within region 630, such that illumination directed towards the face of person 620 is reduced (e.g., to form a "mask" of reduced illumination that includes the area where the eyes of person 620 are located to limit light projected into the eyes of person 620). In some examples, region 630 is smaller than Figure 6A The area 630 shown is the size of a smaller area around the eyes of person 620 and / or includes the eyes of person 620.

[0186] In some examples, a region has more than one associated property. For example, window 616 is transparent, causing sensor readings to assign a color to region 626 that is outside window 616 (e.g., green if grass is visible, or blue if blue sky is visible). In some examples, because window 616 is transparent, region 626 is associated with a property indicating transparency. In some examples, light source 601 determines illumination based on a set of one or more properties associated with a region. This set of properties can be one property associated with the region, multiple properties (less than all and / or a subset), or all properties. In some examples, one or more properties may override other properties. In some examples, because region 626 includes transparent window 616, light source 601 ignores the color property (e.g., green) and does not provide illumination in region 626. In some examples, a reflector is treated similarly to a window (e.g., reduced or no illumination to reduce unwanted reflections entering physical space 600). In some examples, one property is ignored (and in some examples, does not affect illumination output), while another property affects illumination output. In some examples, several properties collectively affect the lighting. In some examples, light source 601 reduces the lighting in region 630 when the properties of region 630 indicate that a person's face is involved, in order to avoid shining light into the eyes of person 620, but provides a low level of lighting based on other properties of region 630, such as color temperature and / or time of day.

[0187] Figure 6B The physical space 600 is shown from different viewpoints and at different points in time. Figure 6B In the middle, the viewpoint is oriented towards and Figure 6A The opposite direction of the viewpoint shown makes it possible to... Figure 6B In the image, wall 612 is directly behind the viewpoint and opposite wall 634. (Example) Figure 6B As shown, physical space 600 includes physical features such as floor 632, walls 634, person 620 (now sitting in a chair) and person 636.

[0188] exist Figure 6BIn this process, light source 601 detects a request to illuminate region 642 within physical space 600. In response to the detected request, light source 601 illuminates region 642 based on one or more detected attributes of the floor 632 within region 642. For example, a request to illuminate region 642 may cause light source 601 to illuminate region 642 with light having a color and / or brightness determined based on the color of the floor 632 within region 642. Similarly, a request to illuminate region 644 may cause light source 601 to illuminate region 644 with light having a brightness determined based on the color of the walls 634 within region 644. And again, a request to illuminate region 646 may cause light source 601 to illuminate region 646 with light having a brightness determined based on identifying region 646 as including the face (of person 636) (e.g., light source 601 may reduce illumination within region 646 to reduce illumination directed at the face of person 636).

[0189] In some examples, light source 601 changes the illumination of a region in response to changes in one or more properties associated with a region of physical space 600. For example, in Figure 6B In this scenario, person 636 faces the direction of light source 601, which provides reduced illumination in region 646, as described above. If, for example, person 636 turns to face the opposite direction and is no longer facing light source 601, then region 646 will no longer be associated with the attribute indicating the detection of a person's face and therefore could have increased illumination, and light source 601 could change the illumination of region 646. In this example, if person 636 turns again to face light source 601, then the illumination in region 646 can return to a reduced illumination state because the attribute changed when a face was detected.

[0190] In some examples, light source 601 maintains illumination of the area in response to changes in one or more properties associated with the area of ​​physical space 600. For example, light source 601 may continue to illuminate the area. Figure 6B Reduced lighting is provided in area 630 (relative to) Figure 6A (Lighting in area 630). For example... Figure 6B As shown, person 620 has changed position (e.g., to a sofa) and posture (e.g., from standing to sitting) within physical space 600. In some examples, in response to detecting a change in one or more properties associated with area (e.g., 630), light source 601 changes the illumination of that area. In some examples, light source 601 tracks the change in position of area 630 due to movement of person 620 and, in response, follows the position of area 630 with determined illumination of area 630 (e.g., in real time or after the person has been positioned). In this example, light source 601 provides reduced illumination within area 630 (e.g., relative to the surrounding physical space).

[0191] In some examples, light source 601 adjusts the color temperature (and / or other properties of the lighting) based on detected environmental changes (e.g., within a physical space) and / or the time of day. For example, the color temperature of the lighting output by light source 601 may be adjusted to more closely resemble or match the color temperature of natural light throughout the day (e.g., having a cooler / bluer appearance at midday and / or a warmer / orange appearance at sunrise and / or sunset). In some examples, light source 601 (e.g., using a sensor) detects changes in the color temperature of the room. In some examples, the color temperature is based on the time of day and / or the day of the year (e.g., which can provide approximate sunrise and / or sunset times for estimating the color temperature).

[0192] Figure 7 This is a flowchart illustrating a method (method 700) for providing context-aware lighting according to some examples. Some operations in method 700 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0193] As described below, method 700 provides an intuitive way to provide context-aware lighting. Method 700 reduces the cognitive burden on the user when providing context-aware lighting, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to provide context-aware lighting more quickly and efficiently saves power and increases the time interval between battery charging sessions.

[0194] In some examples, method 700 is performed at a computer system (e.g., 100, 300, and / or 500) communicating with a light source (e.g., a lighting device, point light source, spotlight, and / or one or more light sources) (e.g., 601, 601A, 601B, 601C, and / or 601D). In some examples, the computer system is a telephone, watch, tablet computer, fitness tracker, wearable device, accessory, speaker, lamp, head-mounted display (HMD), and / or personal computing device. In some examples, the light source is not physically connected to and / or coupled to the computer system. In some examples, the computer system communicates with one or more cameras. In some examples, one or more cameras are not physically connected to the light source.

[0195] At 702, the computer system detects a request for lighting a region (e.g., location, area, section, and / or part) (e.g., general or specific area) (e.g., 622, 624, 626, 628, 630, 642, 644, and / or 646) of a physical space (e.g., a physical environment, room, office, and / or building) (e.g., 600). In some examples, detecting a request includes detecting input corresponding to the request (e.g., tap gesture, long press gesture, verbal request and / or command, physical button press, pointing input and / or air gesture, and / or rotation of a physical input mechanism). In some examples, detecting a request includes receiving a message from a different computer system indicating that the request has been received by that different computer system.

[0196] At 704, in response to a request to detect a region of the illuminated physical space and based on the determination that the region of the physical space has (e.g., 612, 614, 616, 618, 620, 632, 634 and / or 636) a first attribute (e.g., a first characteristic, a first state and / or a first scenario, such as the amount of translucency), the computer system provides a first type of illumination (e.g., the color, intensity and / or size of the illumination) via a light source. Figure 6A and / or Figure 6B As shown, providing less lighting to areas with televisions compared to areas without televisions to reduce glare, and / or providing different colored lighting to areas with walls of a specific color compared to areas with walls of a different color. In some examples, providing the first type of lighting includes activating a light source. In some examples, providing the first type of lighting includes changing the light output by the light source. In some examples, providing the first type of lighting includes sending a request to the light source to modify the light output by the light source. In some examples, the first type of lighting is provided until it is determined that an area of ​​the physical space does not have the first attribute. In some examples, the first type of lighting is provided until a request to stop the first type of lighting is received.

[0197] At 706, in response to a request to detect a region of the illuminated physical space and based on the determination that the region of the physical space has a second attribute (e.g., a second characteristic, a second state, and / or a second scenario) different from the first attribute (e.g., 612, 614, 616, 618, 620, 632, 634, and / or 636), the computer system waives the provision (e.g., via a light source) of the first type of illumination (e.g., in some examples, not providing another type of illumination, or in some examples, when providing different types of illumination) (e.g., as for...). Figure 6A and / or Figure 6BAs described, providing less illumination to areas with windows compared to areas without windows to reduce the amount of light leaving the windows, and / or providing less illumination to areas with human faces compared to areas without human faces to reduce the amount of illumination in the direction of human eyes. Providing a first type of illumination based on determining that the area has a first attribute allows the illumination to be automatically specific to and / or based on the attributes of the area without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions have been met.

[0198] In some examples, in response to a request to detect a region of the illuminated physical space and based on the determination that the region of the physical space has a third attribute (e.g., a second attribute or an attribute different from the first and second attributes) (e.g., reflections and / or surfaces that will affect and / or act on other lighting in the physical space) (e.g., reflections and / or surfaces that will affect and / or act on other lighting in the physical space), the computer system provides a second type of lighting (e.g., as targeted at...). Figure 6A and / or Figure 6B As described, lighting of a different color is provided in areas of the floor with a specific color compared to areas of a different color on the floor, and / or more lighting is provided in areas where people are present compared to areas where no people are present, so that people can be seen. The second type of lighting is less lighting than the first type of lighting (and / or has less lighting, a reduced amount of lighting, a lower amount of lighting, is dimmer, and / or less bright). In some examples, the first type of lighting includes a first amount of lighting, and the second type of lighting includes a second amount of lighting different from the first amount of lighting. In some examples, the first type of lighting includes at least some lighting. In some examples, the second type of lighting includes a reduced amount of lighting compared to the first type of lighting (e.g., relative and / or relative to). In some examples, the second type of lighting is different from the first type of lighting. Determining that an area has a third attribute that results in the second type of lighting allows lighting to be automatically specific to and / or based on the attributes of the area without user input, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0199] In some examples, in response to a request to detect an area of ​​the physical space to be illuminated and based on the determination that the area of ​​the physical space has a third attribute (e.g., a second attribute or an attribute different from the first and second attributes) (e.g., reflections and / or surfaces that would affect and / or act on other lighting in the physical space) (e.g., for 612, 614, 616, 618, 620, 632, 634 and / or 636), the computer system refrains from providing illumination to the area of ​​the physical space (e.g., for Figure 6A and / or Figure 6B (For example, abandoning lighting when the area includes a television, window, and / or a person's eye). Abandoning lighting to areas of the physical space based on determining that the area has a third attribute allows lighting to be automatically specific to and / or based on the attributes of the area without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions are met.

[0200] In some examples, in response to a request to detect an area of ​​the illuminated physical space and based on the determination that the area of ​​the physical space has a fourth attribute (e.g., a second attribute, a third attribute, or an attribute different from the first, second, and / or third attribute) (e.g., one or more specific colors and / or surface types) (e.g., 612, 614, 616, 618, 620, 632, 634, and / or 636), the computer system provides a third type of illumination (e.g., different colors and / or more or fewer lights) via a light source, different from the first type of illumination (e.g., different colors and / or more or fewer lights). Figure 6A and / or Figure 6B This involves providing less lighting to areas with windows compared to areas without windows to reduce the amount of light leaving the windows, and / or providing less lighting to areas with human faces compared to areas without human faces to reduce the amount of lighting in the direction of human eyes. In some examples, the third type of lighting differs from the first type of lighting. In some examples, the third type of lighting includes more or less lighting than the second type of lighting. In some examples, the third type of lighting includes the same amount of lighting as the first and / or second types of lighting, but includes colors different from the first and / or second types of lighting. Providing the third type of lighting based on determining that the area has a fourth attribute allows lighting to be automatically specific to and / or based on the attributes of the area without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions has been met.

[0201] In some examples, determining that a region of physical space has a first attribute includes determining that the region has a first reflectivity amount (e.g., the amount and / or direction of light reflected from a surface (e.g., 612, 614, 616, 618, 620, 632, 634 and / or 636)) (e.g., reflectivity factor) (e.g., as described above regarding television). In some examples, determining that a region of physical space has a second attribute includes determining that the region has a second reflectivity amount different from the first reflectivity amount (e.g., as described above regarding 612 and / or 614). In some examples, a third and / or fourth attribute is based on the reflectivity of the region. In some examples, determining that a region has a first reflectivity amount includes sensing the first reflectivity amount of the region of physical space via a sensor communicating with a computer system. In some examples, determining that a region has a first reflectivity amount includes identifying the type of object in the region (e.g., by identifying the object (e.g., 616, 618, 620 and / or 636) and identifying the type of object), and identifying a predefined (e.g., typical, average, hypothetical and / or predicted) reflectivity amount of the object of that type. The first property based on reflectivity allows lighting to be automatically specific to and / or targeted at reflective surfaces without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions are met.

[0202] In some examples, determining that a region of physical space has a first attribute includes determining that the region has a first transparency amount (e.g., the amount of light reflected in directions not away from the region) (e.g., a transparency factor) (e.g., as described above regarding windows). In some examples, determining that a region of physical space has a second attribute includes determining that the region has a second transparency amount different from the first transparency amount. In some examples, the first attribute and / or the second attribute are based on the transparency of the region (and in some examples, when the region corresponds to and / or a window, windowpane, glass surface, transparent and / or translucent surface and / or surface through which light passes). In some examples, the first attribute is a first corresponding attribute according to determining that the region has a first corresponding transparency amount; and the first corresponding attribute is a second corresponding attribute different from the first corresponding transparency amount according to determining that the region has a second corresponding transparency amount different from the first corresponding transparency amount. In some examples, determining that a region has a first transparency amount includes sensing a first transparency amount in the region of physical space via a sensor communicating with a computer system. In some examples, determining that a region has a first transparency amount includes identifying the type of object in the region (e.g., by identifying the object and identifying the type of object), and a predefined (e.g., typical, average, hypothetical, and / or predicted) transparency amount identifying the type of object. The first property based on the amount of transparency allows lighting to be automatically specific to and / or targeted at transparent surfaces without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions have been met.

[0203] In some examples, determining a region of physical space has a first attribute including determining whether a first person (e.g., a specific person and / or any person) (e.g., 620 and / or 636) exists in (e.g., is detected and / or identified as being in) the region. In some examples, determining a region of physical space has a second attribute including determining whether a first person exists in the region. In some examples, determining a region of physical space has a first attribute including determining whether a first person (e.g., any person and / or any specific person) exists in the region. In some examples, determining a region of physical space has a first attribute including determining whether a first person (e.g., a specific person and / or a specific person) exists in the region. In some examples, determining a region of physical space has a second attribute including determining whether a first person (e.g., any person and / or any specific person) does not exist in (e.g., is not and / or is not identified as being in) the region. In some examples, determining a region of physical space has a second attribute including determining whether a first person (e.g., a specific person and / or a specific person) does not exist in (e.g., is not and / or is not identified as being in) the region. In some examples, determining the presence of a first person in the area is based on information received in communications from different devices (e.g., user devices such as the person's own, user devices corresponding to the person, and / or user devices associated with the person). In some examples, determining the presence of a first person in the area is based on analysis of one or more images of the area to identify the first person. In some examples, determining the presence of a first person in the area is based on whether a motion sensor in the area has been triggered (e.g., motion has been detected). In some examples, the first attribute is a third corresponding attribute based on determining that the first person is present in the area; and the first attribute is a fourth corresponding attribute different from the first corresponding attribute based on determining that the first person is not present in the area. The first attribute based on the presence of a person allows lighting to automatically react differently depending on the number of people present without user input, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0204] In some examples, determining that a region of physical space has a second attribute includes determining whether the face (e.g., the entire face and / or a portion of the face, such as one or more eyes) of a second person (e.g., a specific person and / or a particular person) exists in (e.g., is detected and / or identified as being in) the region. In some examples, determining that a region of physical space has a first attribute includes determining whether the face of a second person exists in the region. In some examples, determining that a region of physical space has a first attribute includes determining whether the face of a second person (e.g., any person and / or any particular person) exists in the region. In some examples, determining that a region of physical space has a first attribute includes determining whether the face of a second person (e.g., a specific person and / or a particular person) exists in the region. In some examples, determining that a region of physical space has a second attribute includes determining that the face of a second person (e.g., any person and / or any particular person) does not exist in (e.g., is not and / or is not identified as being in) the region. In some examples, determining that a region of physical space has a second attribute includes determining that the face of a second person (e.g., a specific person and / or a particular individual) is not present in (e.g., not and / or not identified as being in) that region. In some examples, determining whether a second person's face is present in that region is based on analysis of one or more images of that region to identify the face. In some examples, the first attribute is a fourth corresponding attribute based on determining that a second person's face is in that region; and the first attribute is a fifth corresponding attribute different from the fourth corresponding attribute based on determining that a second person's face is in that region. In some examples, in conjunction with determining whether a second person's face is in that region, a computer system and / or another computer system creates a mask including the area where the second person's eyes are located to limit and / or reduce the light projected onto the second person's eyes (e.g., to provide less illumination in the second person's eyes (e.g., to avoid blinding and / or obstructing the second person's vision)). The primary property of the presence of a human face allows lighting to automatically take into account the human eye and / or protect the human eye from light without user input, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions have been met.

[0205] In some examples, in response to a request to detect a region of the physical space to be illuminated: regardless of the properties of the region of the physical space (e.g., a first property, a second property, and / or any other property) (e.g., not based on and / or not considering the property), the computer system provides a fourth type of illumination (e.g., color, intensity, and / or size of the illumination) relative to a second region of the physical space (e.g., 622, 624, 626, 628, 630, 642, 644, and / or 646), wherein the second region is different from the first region. In some examples, the second region is adjacent to and / or close to the first region. In some examples, providing the fourth type of illumination includes activating the light source. In some examples, providing the fourth type of illumination includes changing the light output by the light source. In some examples, providing the fourth type of illumination includes sending a request to the light source to modify the light output by the light source. In some examples, the fourth type of illumination is provided until a request to stop the fourth type of illumination is received. In some examples, the fourth type of illumination is provided with regard to (e.g., based on and / or considering) the properties of the second region (e.g., a second region with different properties makes it possible to provide different types of illumination). In some examples, the fourth type of lighting differs from the first type of lighting and / or the second type of lighting. Providing the fourth type of lighting relative to the second area in response to a request for lighting allows the lighting to be automatically specific to and / or based on the properties of the corresponding area in which the lighting is provided without user input, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0206] In some examples, the light source is a single light-emitting device. In some examples, the light source includes a single housing and / or receives commands to illuminate different areas (e.g., 622, 624, 626, 628, 630, 642, 644, and / or 646). In some examples, the light source includes multiple spotlights, light-emitting diodes, bulbs, and / or lasers located within a single housing and / or enclosure. In some examples, the light source includes multiple light sources (e.g., spotlights, light-emitting diodes, bulbs, and / or lasers) in wired communication (and in some examples, not wireless communication) (e.g., 601A, 601B, 601C, and / or 601D). In some examples, the light source, computer system, and / or the environment in which the light source is located include mirrors, lenses, and / or other optical components for changing the direction and / or pattern of the light emitted by the light source. In some examples, the light source moves (e.g., laterally, horizontally, vertically, inwardly, and / or outwardly) to illuminate different areas of a physical space. In some examples, the light source is configured to selectively illuminate portions of the light source to selectively illuminate different areas of the physical space. The light source as a single light-emitting device allows for less communication between devices, requires less setup compared to setting up multiple devices, and / or is easier to install, thereby reducing the amount of input required to perform the operation.

[0207] In some examples, after (and / or simultaneously) providing the first type of lighting and based on the determination that the area of ​​the physical space has changed from the first attribute to the fifth attribute (e.g., the determination that the area of ​​the physical space has changed from the first attribute to the fifth attribute occurs when the first type of lighting is provided) (e.g., people and / or objects move and / or people look in different directions) (e.g., which are different from the first and / or second attributes), the computer system provides a fifth type of lighting via a light source that is different from the first type of lighting (e.g., as described above regarding...). Figure 6A and / or Figure 6B The description, such as when a person 620 from Figure 6A Move to the position shown Figure 6B(When the location is shown). In some examples, the fifth type of lighting is different from the second, third, and / or fourth type of lighting. In some examples, the type of lighting provided changes in response to one or more properties of the area (e.g., lighting increases and / or decreases). In some examples, after (and / or simultaneously with) providing the first type of lighting and based on the determination that the area of ​​the physical space has not changed from the first property (and in some examples, changed to the fifth property and / or any other property), the computer system does not provide the fifth type of lighting, which is different from the first type of lighting. In some examples, the fifth property is the second property, and the fifth type of lighting is the second type of lighting. Providing the fifth type of lighting based on the determination that the area has changed from the first property to the fifth property allows the lighting to automatically adjust based on the change in physical space properties without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions has been met.

[0208] In some examples, providing a fifth type of lighting based on determining that a region of the physical space has changed from a first attribute to a fifth attribute includes changing the lighting from a first type of lighting to a fifth type of lighting via a light source (e.g., as mentioned above regarding...). Figure 6A and / or Figure 6B The description, such as when person 636 exists Figure 6B Not existing in Figure 6A (China Times). In some examples, changing from type 1 lighting to type 5 lighting involves a gradual change between different types of lighting (e.g., lighting with different types of lighting between type 1 and / or type 5). In some examples, changing from type 1 lighting to type 5 lighting involves switching from type 1 to type 5 without lighting with a type different from type 1 and / or type 5. In some examples, type 5 is type 2 lighting. In some examples, the computer system does not change from type 1 lighting to type 5 lighting via a light source if the area of ​​the physical space has not yet changed from the first attribute to the fifth attribute. Changing from type 1 lighting to type 5 lighting based on the determination that the area has changed from the first attribute to the fifth attribute allows lighting to automatically adjust based on changes in physical space attributes without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions has been met.

[0209] In some examples, after (and / or simultaneously) providing type 5 lighting, the computer system changes from type 5 lighting to type 1 lighting via a light source (e.g., as described above regarding...). Figure 6A and / or Figure 6B The description, such as if a person 620 is in a state of... Figure 6B Return to the location shown. Figure 6A (The location is shown). In some examples, changing from type 5 lighting to type 1 lighting involves a gradual change between different types of lighting (e.g., lighting with different types of lighting between type 5 and / or type 1). In some examples, changing from type 5 lighting to type 1 lighting involves switching from type 5 to type 1 without lighting with types different from type 1 and / or type 5. Changing back from type 5 lighting to type 1 lighting allows lighting to automatically adjust based on physical space property changes without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions has been met.

[0210] In some examples, providing first-type illumination via a light source includes: causing an output of a second color via the light source, based on a defined area including surfaces having a first color (e.g., tables, walls, solid flat and / or surfaces, windows, and / or object surfaces) (e.g., 612, 614, 616, 618, 620, 632, 634, and / or 636). In some examples, the second color is different from the first color. In some examples, providing first-type illumination via a light source includes: causing an output of a fourth color (e.g., without causing an output of the second and / or first colors), based on a defined area including surfaces having a third color different from the first color (e.g., tables, walls, solid flat and / or surfaces, windows, and / or object surfaces) (e.g., 612, 614, 616, 618, 620, 632, 634, and / or 636) (and in some examples, excluding surfaces having the first color), via the light source. In some examples, the first, second, third, and / or fourth colors are different colors. In some examples, one or more of the first, second, third, and / or fourth colors are different colors, and / or one or more of the first, second, third, and fourth colors are the same color. In some examples, depending on whether the defined area includes a surface with the first color but excludes a surface with the third color, the computer system causes the output of the second color via a light source (e.g., without causing the output of the fourth and / or third colors). Causing the output of the second or fourth color based on the color of the surface allows lighting to be automatically specific to and / or based on the properties of the corresponding area where illumination is provided without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions has been met.

[0211] In some examples, after (and / or when) a first type of lighting is provided and based on determining that the current time of day is the first time of day, the computer system changes the lighting from the first type of lighting to a sixth type of lighting via a light source, wherein the first type of lighting includes a first color temperature, and wherein the sixth type of lighting includes a second color temperature different from the first color temperature (e.g., as mentioned above regarding...). Figure 6BAs described, the lighting changes as the day progresses. In some examples, the color temperature of the lighting changes based on a first time of day (e.g., the color temperature changes over time (e.g., different color temperatures are used at different times)). In some examples, the first type of lighting is different from the sixth type of lighting. In some examples, the first type of lighting does not include a second color temperature, and the sixth type of lighting does not include a first color temperature. In some examples, after the first type of lighting is provided (and / or when the first type of lighting is provided) and based on determining that the current time of day is a second time of day different from the first time of day, the computer system changes the lighting from the first type of lighting to another type of lighting with a color temperature different from the first color temperature and / or the second color temperature (e.g., different from the sixth type of lighting and / or the first type of lighting) via a light source. Changing the lighting from the fifth type of lighting to the sixth type of lighting based on the time of day allows the lighting to be automatically specific to and / or based on the time of day and / or changes throughout the day without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions has been met.

[0212] In some examples, in response to detecting a change in the physical space and based on determining that the physical space has changed in a first manner (e.g., there is more and / or less light in the physical space, one or more objects and / or people have moved in the physical space), the computer system changes the lighting from the first type of illumination to the seventh type of illumination via a light source (as described above). Figure 6B As described, where, in addition to person 620, person 636 is also included in physical space 600, the first type of lighting includes a third color temperature, and the seventh type of lighting includes a fourth color temperature different from the third color temperature. In some examples, the color temperature of the lighting changes based on the properties of the changing physical space (e.g., the color temperature changes with one or more properties (e.g., different color temperatures are used when different properties are detected in the physical space)). In some examples, in response to detecting a change in the physical space and based on determining that the physical space has changed in a second manner different from the first manner, the computer system changes the lighting from the first type of lighting to an eighth type of lighting different from both the first and seventh types of lighting via a light source. Changing the lighting from the first type of lighting to the seventh type of lighting based on determining that the physical space has changed in a first manner allows the lighting to automatically adjust based on the change in physical space without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions has been met.

[0213] In some examples, determining that a region of physical space has a first attribute includes making the determination based on first data detected by a sensor (e.g., one or more sensors, such as a camera or a thermostat). In some examples, determining that a region of physical space has a second attribute includes making the determination based on second data detected by a sensor. In some examples, the second data is different from the first data. In some examples, the sensor communicates with a computer system. In some examples, the first attribute is determined based on data detected by one or more sensors communicating with a light source. In some examples, the second attribute is determined based on data detected by a sensor. In some examples, the first attribute is a sixth corresponding attribute based on determining that the sensor detected the first data; and the first attribute is a seventh corresponding attribute different from the sixth corresponding attribute based on determining that the sensor detected second data different from the first data. In some examples, the sensor is integrated into the computer system and / or the light source (e.g., physically included therein and / or not separated from it). In some examples, the sensor is separated from the computer system and / or the light source (e.g., not integrated therein and / or not physically included therein). The first and second attributes based on sensor-detected data ensure that the attributes are accurate, consistent, and / or based on real-world conditions, thereby reducing the amount of input required to perform the operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing the operation without further user input once a set of conditions has been met.

[0214] It should be noted that the process described above is relative to that described in method 700 (e.g., Figure 7 The details of this method also apply in a similar manner to the other methods described herein. For example, method 900 may optionally include one or more features of the various methods described above with reference to method 700. For example, the light source of method 900 may be the light source of method 700. For the sake of brevity, these details will not be repeated below.

[0215] Figures 8A to 8E Exemplary techniques for changing lighting based on detected user activity are illustrated in the accompanying figures. The user interface in these figures is used to illustrate the process described below, including... Figure 9 The process in.

[0216] Figures 8A to 8B An example is illustrated where light source 801 changes illumination based on detected user activity. Light source 801 may be the same as or similar to light source 601. For example, light source 801 may be a general term used to describe one or more light sources (such as light sources 801A, 801B, 801C, and / or 801D). In some examples, light source 801 includes one or more features described herein with respect to any one or more light sources as described in Figures 6, 10, 12, and / or 14.

[0217] Figure 8A An example is given of a physical space 800, a room where person 810 is seated on a sofa. In some examples, person 810's activities include whether person 810 is asleep or awake. Figure 8A In this system, light source 801 detects that person 810 is awake within physical space 800, and in response to this detection, illuminates physical space 800 based on the person 810 being awake. For example, as... Figure 8A As shown, light source 801 outputs a focused beam of illumination 822 based on the size of region 820. Region 820 represents a portion of the physical space 800 that is identified as corresponding to person 810 (e.g., their location, size, and / or activity (e.g., awake, asleep, sitting, standing, and / or lying down)).

[0218] In some examples, light source 801 detects changes in user activity within physical space 800 and, in response to this detection, alters the lighting in physical space 800. For example, in Figure 8B In this system, light source 801 detects that person 810 is sleeping and, in response to this detection, illuminates physical space 800 based on the fact that person 810 is sleeping. For example, as... Figure 8B As shown, light source 801 stops illuminating person 810 (e.g., no longer outputs light). Figure 8A Lighting 822). In Figure 8B In the process, person 810 is still detected, but their activity has changed (e.g., from awake to asleep), so light source 801 adjusts the illumination based on this change (e.g., adjusts to illumination suitable for the detected user activity). Figure 8B In this example, light source 801 turns off the lighting in response to detecting that person 810 is sleeping. In some examples, light source 801 reduces the lighting (e.g., dims its light output) in response to detecting that person 810 is sleeping. For example, light source 801 outputs a lower amount of lighting than before it detected that person 810 is sleeping.

[0219] In some examples, changes in user activity within physical space 800 represent changes in the number of people detected within physical space 800. For example, in Figure 8C In this example, light source 801 detects the presence of both person 810 and person 824 in physical space 800, and in response to this detection, illuminates physical space 800 based on user activity indicating the presence of both person 810 and person 824. In this example, person 810 has woken up from sleep (e.g., ...). Figure 8B (As shown) and now joined by 824, who are sitting next to them. Figure 8C In addition to detecting both people 810 and 824, light source 801 also detects that both people are awake, and in response, illuminates physical space 800 based on the presence of the two awake people. For example, the size of region 830 is compared to ( Figure 8AThe area 820 is larger to accommodate the addition of person 824. The light source 801 outputs a ratio based on the size of area 830. Figure 8A The focused beam of illumination 832 is wider than that of illumination 822. In this example, region 830 is larger than region 820, so light source 801 illuminates a larger area. Region 830 represents both person 810 and person 824. In some examples, if light source 801 detects... Figure 8C If person 824 leaves physical space 800, while person 810 remains in the same location and is awake, then in response to this detection, light source 801 can return to provide the light source. Figure 8A The lighting shown (e.g., lighting 822 based on region 820).

[0220] Figures 8D to 8E The example also illustrates how light source 802 changes the lighting based on detected user activity. Figure 8D An example of physical space 800 is shown, comprising a room with person 810, person 824, table 836, and chair 838 (which is not occupied by anyone). Figure 8D In this system, light source 801 detects the presence of two people (e.g., person 810 and person 824) in physical space 800 and detects that they are performing an activity (e.g., eating). Figure 8D In the process, light source 801 detects that chair 838 is not occupied (e.g., not occupied by anyone). For example... Figure 8D As shown, in response to detected user activity (e.g., people 810 and 824), light source 801 outputs illumination 842 based on area 840 that includes both people 810 and 824. It is noteworthy that chair 838 is outside of illumination 842 (e.g., because chair 838 is not within area 840). In some examples, portions of physical space 800 outside a specific area (e.g., 840) (e.g., chair 838) receive illumination of a different amount (e.g., less or more) and / or with one or more different characteristics (e.g., color, brightness, color temperature, focus, and / or diffusion). In some examples, light source 801 detects attributes of user activity and outputs illumination based on attributes of user activity (e.g., associated with user activity). For example, if two people are detected eating (e.g., similar to...), the light source outputs illumination based on these attributes. Figure 8D(As described), then light source 801 can determine that the meal is a date and, in response, adjust the lighting to be dimmer (e.g., create romantic lighting for the meal). In some examples, the attributes of user activity are determined based on one or more of the following: time of day, detection and / or identification of one or more people in physical space 800, detection of one or more features associated with the attribute. For example, a non-limiting list of attributes of user activity may include: the presence of candlelight and / or wine glasses for the date, the presence of board games and / or crafts for leisure activities, and / or the presence of books for reading. In some examples, the presence of candlelight results in reduced lighting compared to the absence of candlelight. In some examples, the presence of wine glasses for the date results in increased lighting compared to the absence of wine glasses for the date. In some examples, the presence of board games results in lighting in a more neutral color compared to the absence of board games. In some examples, the presence of crafts for leisure activities results in lighting in a cooler color compared to the absence of crafts for leisure activities. In some examples, the presence of books for reading results in lighting in a warmer color compared to the absence of books for reading.

[0221] In some examples, light source 801 changes the illumination based on the movement of a user (e.g., a person) within a predetermined distance of a location (e.g., associated with an object). For example, reference... Figure 8D In the scene shown, people 810 and 824 approach table 836 before sitting down at table 836. In some examples, light source 801 detects that the user has moved within a predetermined distance of the location and, in response, adjusts the lighting of physical space 800 (e.g., the area including that location). Figure 8D In the example, in response to detecting that person 810 and / or person 824 are within a threshold distance of table 836 (or one or more chairs associated with table 836), light source 801 adjusts the illumination of area 840 (e.g., adds additional lighting or reduces lighting). Whether the illumination is increased or decreased may depend on user activity. For example, if light source 801 detects that people 810 and 824 are about to play a board game at table 836, light source 801 may increase the illumination in response. If light source 801 detects that people 810 and 824 are about to eat, light source 801 may reduce the illumination in response.

[0222] In some examples, light source 801 detects movement of objects in physical space 800, and in response, the lighting of the moving object follows the object as it moves. For example, in Figure 8C and Figure 8D In the scene shown, the light source 801 can provide illumination for the movable figures 810 and 824 (e.g., based on...). Figure 8CIn area 830), so that they move from the sofa (where lighting is based on area 830) to their seating position at the table 836 (e.g., where lighting is based on area 836). Figure 8D The light source 802 (area 840) follows the object (e.g., person 810) throughout its movement through the physical space 800. In some examples, the light source 802 outputs the same illumination to the object (e.g., person 810) as it moves through the physical space 800 (e.g., the illumination is a spotlight that follows person 810 from sofa to table while maintaining the same lighting characteristics). In other examples, the light source 802 outputs different amounts of illumination to the object as it moves, such as by changing the size of the illumination (e.g., a spotlight changes size as the light follows the user and / or based on the space in which the light moves). In some examples, the different illumination is based on the final position of the movement (e.g., the sofa is illuminated differently than the table).

[0223] exist Figure 8E In this example, light source 801 detects that people 810, 824, and 844 (sitting in chair 838) are all present in physical space 800, and in response to this detection, illuminates physical space 800 based on user activity indicating the presence of these three people. In this example, person 838 has joined the other two people (persons 810 and 824) at table 836. Figure 8E In addition to detecting people 810, 824, and 844, light source 801 also detects multiple people eating and, in response, illuminates physical space 800 based on the activity of three people eating. For example, the size of area 850 is compared to ( Figure 8D The area 840 is larger to accommodate the addition of person 844. The light source 801 outputs a ratio based on the size of area 850. Figure 8D The illumination 842 is wider than the illumination 852. In this example, area 850 is larger than area 840, so light source 801 outputs a larger total illuminated area. Area 850 represents three people, including person 810, person 824, and person 844. In some examples, if light source 801 detects... Figure 8E Person 844 leaves physical space 800, while persons 810 and 824 remain in the same location and are awake. In response to this detection, light source 801 can return to provide... Figure 8D The lighting shown (e.g., lighting 842 based on region 840).

[0224] Figure 9 This is a flowchart illustrating methods for changing lighting (e.g., method 900) according to some examples. Some operations in method 900 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0225] As described below, Method 900 provides an intuitive way to change lighting. Method 900 reduces the cognitive burden on users when changing lighting, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to change lighting more quickly and efficiently saves power and increases the time interval between battery charging.

[0226] In some examples, method 900 is performed at a computer system (e.g., 100, 300, and / or 500) that communicates with a light source (e.g., a point light source, a spotlight, and / or one or more light sources) (e.g., 801, 801A, 801B, 801C, and / or 801D). In some examples, the computer system is a telephone, watch, tablet computer, fitness tracker, wearable device, accessory, speaker, lamp, head-mounted display (HMD), and / or personal computing device. In some examples, the light source is not physically connected to and / or coupled to the computer system. In some examples, the computer system communicates with one or more cameras. In some examples, one or more cameras are not physically connected to the light source. In some examples, the light source is a single light-emitting device (e.g., as described above with respect to method 700).

[0227] At 902, when a user (e.g., 810, 824, and / or 844) is detected in a physical space (e.g., a physical environment, room, office, and / or building) (e.g., 800), the computer system detects changes in user activity (e.g., human activity (e.g., 810, 824, and / or 844)) within the physical space. In some examples, detecting changes in user activity includes detecting the user's location. In some examples, detecting changes in user activity includes detecting the user's state. In some examples, detecting changes in user activity includes detecting objects near the user (e.g., 836 and / or 838). In some examples, changes in user activity are detected when illumination in the physical space via a light source has a first set of attributes (e.g., non-zero attributes (e.g., color, intensity, hue, and / or brightness)) (e.g., where at least one light source communicating with the computer system is emitting light).

[0228] At point 904, in response to detecting a change in user activity in the physical space, the computer system changes the lighting in the physical space via a light source, while continuing to detect users (e.g., people) in the physical space (e.g., such as...). Figure 8A and Figure 8B , Figure 8A and Figure 8C , Figure 8B and Figure 8C and / or Figure 8D and Figure 8E(As shown in the examples). In some examples, changing the lighting includes activating a light source. In some examples, changing the lighting includes changing the light output by the light source. In some examples, changing the lighting includes sending a request to the light source to modify the light output by the light source. In some examples, changing the lighting includes changing a first light source in a first manner (e.g., increasing / decreasing brightness, hue, intensity, and / or warmth and / or changing color), and changing a second light source in a second manner different from the first manner (e.g., increasing / decreasing brightness, hue, intensity, and / or warmth and / or changing color). In some examples, the lighting of the physical space is changed to have a second set of attributes different from the first set of attributes (e.g., where at least one light source communicating with the computer system is outputting light). In some examples, changes in user activity are detected by the same sensor that detects the user in the physical space. In some examples, changes in user activity are detected by a different sensor than the sensor that detects the user in the physical space. In some examples, changes in user activity do not correspond to changes in location (e.g., as shown in the examples). Figure 8A and Figure 8B (As shown in the examples). In some examples, the illumination changes based on specific user activity detected in the physical space. In some examples, the illumination is changed in a first manner (e.g., increasing and / or decreasing light intensity, color intensity, hue, color temperature, and / or brightness) depending on whether the change in user activity is determined to be a change in a second type of user activity different from the first type. In some examples, the illumination is changed in a third manner depending on whether the change in user activity is determined to be a change in a third type of user activity different from the third type. In some examples, the illumination is changed in a fourth manner depending on whether the change in user activity is determined to be a change in a fourth type of user activity different from the third type. In some examples, the user is detected in the physical space via a motion sensor, and the change in user activity is detected via a sensor of a different type than the motion sensor. As users continue to be detected in the physical space and the light in the physical space is changed in response to changes in detected user activity, this allows effects to occur automatically not only regarding the user's presence in the physical space but also regarding their activity without user input (e.g., separating the two types of detection into different determinations that can each cause different results), thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions has been met.

[0229] In some examples, detecting changes in user activity within a physical space includes detecting changes in the sleep state (or wakefulness or alertness) of a first user (e.g., the user and / or another user) (e.g., the first user has woken up (e.g., for a predetermined period of time (e.g., 1 second to 10,000 seconds) and / or has fallen asleep (e.g., 1 second to 10,000 seconds))) (e.g., based on changes in the first user's sleep state) (e.g., 810). In some examples, detecting changes in the first user's sleep state is based on analysis of one or more images of the physical space (e.g., detecting movement of the first user and / or no movement within a time period). In some examples, detecting changes in the first user's sleep state is based on detecting input from the first user (e.g., tap input and / or non-tap input (e.g., voice input, gaze input, air gestures, pointing gestures, swipe input, and / or mouse clicks)). In some examples, detecting changes in the first user's sleep state is based on whether motion sensors in the physical space have been triggered (e.g., motion has been detected). In some examples, detecting changes in sleep state is determined via one or more wearable fitness trackers and / or fixed devices (such as smartwatches and / or computers). Detecting changes in sleep state to alter the light in the physical space allows lighting to automatically adjust to changes in user activity without user input and / or ensures a comfortable environment for user adjustment to activity, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions has been met.

[0230] In some examples, detecting changes in user activity within physical activity includes detecting that a first number of users (e.g., people) detected in the physical space have changed from a first number (e.g., such as...). Figure 8A and Figure 8B As shown in 1 and / or as Figure 8D As shown in 2), the change (e.g., within a region and / or area of ​​physical space and / or throughout the entire physical space) is to a second quantity different from the first quantity (e.g., such as...). Figure 8C As shown in 2 and / or as Figure 8EAs shown in 3). In some examples, detecting a change in the number of users detected in the physical space from a first number to a second number is based on information received in communications from different devices, such as user devices of one or more users. In some examples, detecting a change in the number of users detected in the physical space from a first number to a second number is based on analysis of one or more images of the physical space. In some examples, detecting a change in the number of users detected in the physical space from a first number to a second number is based on whether motion sensors in the physical space have been triggered (e.g., motion has been detected). Detecting changes in the number of users to change the light in the physical space allows the lighting to automatically adjust to changes in user activity without user input and / or ensures a comfortable environment for users to adjust to their activities, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions has been met.

[0231] In some examples, the second number is greater than the first number (e.g., the number of users in the physical space and / or in the areas and / or zones of the physical space has increased) (e.g., as... Figure 8C As shown, with Figures 8A to 8B The person shown is compared to two people, and / or as Figure 8E As shown, with Figure 8D (The example shows two people compared to three). Detecting an increase in the number of users changes the lighting in the physical space, allowing the lighting to automatically adjust to changes in user activity without user input and / or ensuring a comfortable environment for users to adjust to their activities. This reduces the amount of input required to perform operations, provides additional control options without cluttering the user interface with additional displayed controls, and performs operations without further user input once a set of conditions are met.

[0232] In some examples, the second quantity is less than the first quantity (e.g., the number of users in the physical space and / or in areas and / or zones within the physical space has decreased). Detecting the decrease in the number of users changes the lighting in the physical space, which allows the lighting to automatically adjust to changes in user activity without user input and / or ensures a comfortable environment for users to adjust to their activities, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0233] In some examples, detecting changes in user activity within physical activity includes detecting a second number of users (e.g., the first number of users or a different number of users than the first number) in the physical space who are performing activities (e.g., specific and / or concrete activities, such as sitting together at a table). Figure 8D As shown), dancing, talking, sleeping (for example, as shown) Figure 8B (as shown) and / or watching TV on the sofa). In some examples, detecting a second number of users performing activities in the physical space is based on analysis of one or more images of the physical space. In some examples, detecting a second number of people performing activities in the physical space is based on the state (e.g., on and / or off) of devices in the physical space (e.g., user devices of users among the second number of users). Detecting changes in the number of users performing activities to change the light in the physical space allows the lighting to automatically adjust to changes in user activity without user input and / or ensures a comfortable environment for users to adjust to activities, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions has been met.

[0234] In some examples, changing the lighting of the physical space while a user continues to be detected in the physical space includes turning at least a portion of the physical space's lighting on or off (e.g., via a light source) (e.g., as described above regarding...). Figure 8B (As described). In some examples, the lighting in the physical space is turned off as a user continues to be detected in the physical space. By turning at least a portion of the physical space on or off as a user continues to be detected, this allows the lighting to automatically adjust to the user (e.g., with or without motion) without user input, thereby reducing the amount of input required to perform an action, providing additional control options without cluttering the user interface with additional displayed controls, and performing an action without further user input once a set of conditions has been met.

[0235] In some examples, changing the lighting in the physical space while a user continues to be detected in the physical space includes changing the level of illumination within the physical space (e.g., increasing or decreasing the level of light) (e.g., increasing and / or adding or decreasing and / or reducing the amount of light) (e.g., via a light source) (e.g., as described above regarding...). Figure 8BAs described, when a person 810 falls asleep and the lighting is reduced. In some examples, changing the level of light in a physical space includes increasing the amount of light output by one set of lights while decreasing, increasing, and / or maintaining the amount of light output by another set of lights. In some examples, changing the level of light in a physical space includes decreasing the amount of light output by one set of lights while decreasing, increasing, and / or maintaining the amount of light output by another set of lights. Instead of turning the lights off and / or on, changing the level of light in a physical space while a user is continuously detected in the physical space allows the lighting to automatically adjust to the user (e.g., with or without motion) without user input, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0236] In some examples, detecting changes in user activity within physical activity includes detecting a second user (e.g., user and / or another user) within a predetermined (e.g., 0.1 m to 20 m) distance (e.g., predefined and / or preconfigured) of their location in the physical space (e.g., near an object and / or device in the physical space) (e.g., a location where there is insufficient lighting for that user (and / or the second user) to see). In some examples, detecting the second user within a predetermined distance of that location is based on information received in communications from different devices (such as the second user's user device). In some examples, detecting the second user within a predetermined distance of that location is based on analysis of one or more images of the physical space. In some examples, detecting the second user within a predetermined distance of that location is based on whether a motion sensor in the physical space has been triggered (e.g., motion has been detected). Detecting the second user's position within a predetermined distance in the physical space to change the light in the physical space allows the lighting to automatically adjust to changes in user activity without user input and / or ensures a comfortable environment for the user to adjust to the activity, thereby reducing the amount of input required to perform the operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing the operation without further user input when a set of conditions has been met.

[0237] In some examples, changing the lighting of the physical space while a user continues to be detected in the physical space includes changing the illuminated area of ​​the physical space (as described above with respect to method 700) from a first area of ​​the physical space to a second area of ​​the physical space that is different from the first area (e.g., as the user moves in the physical space, such as moving from the first area of ​​the physical space to the second area of ​​the physical space) (e.g., as the person moves in the physical space, following the user from one location to another, such that, for example, the user can see) (e.g., as... Figure 8C, Figure 8D and / or Figure 8E (As shown). In some examples, the second region is the region that does not include the user (e.g., initially and / or when the lighting changes) (e.g., the second region is where the user is heading and / or looking). In some examples, the lighting is changed without the user moving (e.g., illuminating a new region). In some examples, the computer system detects user movement; and in response to detecting user movement, the computer system changes the currently illuminated area of ​​the physical space from that area to the second region of the physical space. In some examples, the first region is not included in the second region and / or does not include the second region, and / or vice versa. Changing the illuminated area of ​​the physical space from the first region to the second region as the user continues to be detected in the physical space allows the lighting to automatically adjust to the user's changing needs without user input (e.g., following the user from one region to another), thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0238] In some examples, the first and second areas are illuminated by the same light source (e.g., a single spotlight capable of targeting different areas and / or capable of being moved). In some examples, the intensity of the illumination varies as the light follows the user and / or is based on the area of ​​the physical space being illuminated and / or the movement of the light within it.

[0239] In some examples, the first area is illuminated by a first light source (e.g., 801, 801A, 801B, 801C, and / or 801D). In some examples, the second area is illuminated by a second light source (e.g., 801, 801A, 801B, 801C, and / or 801D) different from the first light source. In some examples, the computer system communicates with both the first and / or second light sources. In some examples, the first light source does not communicate with the second light source. In some examples, different light sources are used for different areas of the physical space (e.g., the lighting on a sofa is different compared to that on a table). In some examples, the areas of the physical space are defined by objects included within those areas (e.g., Figures 8A to 8C Sofas, 836 and / or 838) such as sofas and / or tables are limited.

[0240] It should be noted that the above is relative to method 900 (e.g., Figure 9 The details of the process described herein also apply in a similar manner to the methods described herein. For example, method 700 may optionally include one or more features of the various methods described above with reference to method 900. For example, areas in method 700 may include variations in user activity in method 900. For the sake of brevity, these details will not be repeated below.

[0241] Figures 10A to 10E Exemplary techniques for transmitting information using lighting location are illustrated according to some examples. The user interface in these figures is used to illustrate the processes described below, including... Figure 11 The process in.

[0242] Figure 10A A physical space 1000 is shown, comprising a room with a person 1002 (e.g., the same as person 620 and / or person 810). In some examples, a light source 1001 uses the location of illumination to transmit information (e.g., to a user). In some examples, the light source 1001 includes one or more features as described herein with respect to any one or more light sources of Figures 6, 8, 12, and / or 14.

[0243] exist Figure 10A In this design, light source 1001 detects a request from person 1002 to illuminate wall 1004. This request may include input, which is one or more of the following: input including one or more trigger words (e.g., “Hey, personal assistant”), input representing a physical gesture (e.g., a pointing gesture if person 1002, such as with a finger of person 1002's hand and / or a remote control, is pointing to a specific location on wall 1004), input representing a verbal request (e.g., “Please illuminate this area with blue light”), and / or other input (e.g., input from a device communicating with the light source, such as a remote control, controller device, and / or smartphone). In some examples, the request includes multiple different requests for different colors (e.g., color, color temperature, hue, intensity, and / or color saturation) and / or illumination locations. In some examples, the request indicates one or more physical locations rather than a physical area. In some examples, the request does not indicate one or more colors (e.g., color, color temperature, hue, intensity, and / or color saturation) of one or more specific light sources, but rather indicates one or more physical locations. In some examples, the request does not indicate one or more specific light sources, but rather indicates one or more physical locations. Figure 10A As shown, the request for lighting the wall 1004 includes input representing a pointing gesture made by a person 1002. Figure 10A In this illustration, light source 1001 detects a request to illuminate wall 1004 at the location (area 1014) where person 1002 is gesturing towards them; this area is shown within a dashed box for illustrative purposes. In response to the detected request, light source 1001 illuminates area 1014 with the requested type of illumination (e.g., blue light). In some examples, area 1014 covers the entire area of ​​wall 1004. In some examples, area 1014 covers an area smaller than the entire area of ​​wall 1004 (e.g.,...). Figure 10A (The area within the dashed line in the text).

[0244] Figure 10B This illustrates the lighting requested by the user at a second location within physical space 1000. Figure 10B In this system, light source 1001 detects a request from person 1002 to illuminate wall 1006 (a different wall at a different location from wall 1004) (e.g., one or more inputs indicating the request). For example, light source 1001 detects voice input including requests to move pointing gestures to different locations and / or voice commands indicating illumination of wall 1006 (e.g., "Please illuminate this area with blue light") and input indicating pointing gestures by person 1002 toward area 1016. Figure 10B As shown, in response to the detection of a request, the light source 1001 illuminates the area 1016 with the requested lighting type (e.g., blue light).

[0245] In some examples, in response to a request to illuminate a specific location, light source 1001 illuminates the specific location with the requested illumination, while maintaining some or all of the existing lighting in one or more other areas within physical space 1000. For example, in Figure 10B In this configuration, light source 1001 illuminates area 1016 while maintaining the previously requested illumination of area 1014 (e.g., both areas 1016 and 1014 are now illuminated with blue light according to their respective requests), and also maintains the illumination of the wall areas surrounding area 1016. In some examples, in response to a request to illuminate a specific location, light source 1001 illuminates the specific location with the requested illumination and alters some or all of the existing illumination in physical space 1000. For example, light source 1001 may stop illuminating area 1016 in response to a request to illuminate area 1014 (e.g., a subsequent request cancels the illumination generated by a previous request), and / or alter (e.g., dim) the illumination of the wall areas surrounding area 1016.

[0246] In some examples, the lighting request includes (for example, is) input indicating a pointing gesture. For example, Figure 10A and Figure 10B Each example shows a person 1002 making a pointing gesture, which a light source 1001 can detect as input indicating a request for illumination of a region. In some examples, if the request is an input indicating a pointing gesture without accompanying a second type of input, the light source 1001 stops outputting the requested illumination in response to stopping the detection of input indicating a pointing gesture toward that region. For example, in Figure 10BIn this scenario, if light source 1001 detects that the request to illuminate wall 1006 was made by person 1002 without accompanying and / or prior gestures, button presses, and / or voice input indicating a voice command, then light source 1001 stops outputting illumination in area 1016 in response to input indicating a pointing gesture. In some examples, if the request includes input indicating a pointing gesture accompanied or preceded by a second type of input (e.g., accompanying and / or prior gestures, button presses, and / or voice input indicating a voice command), then light source 1001 continues outputting the requested illumination in response to ceasing detection of input indicating a pointing gesture. For example, in Figure 10B If the request to illuminate wall 1006 is made by person 1002 with a second type of input (e.g., a voice command saying "Please illuminate this area with blue light"), then light source 1001 may continue to output illumination in area 1016 in response to the request (e.g., input indicating a pointing gesture) after it stops detecting input indicating a pointing gesture (e.g., the user stops pointing).

[0247] As should be understood from the above description, the user does not necessarily have to specify a specific unique name portion of the physical space 1000 for the light source 1001 to respond. For example, input representing a voice command could identify a general location / area / object (e.g., “that area,” “this,” “here,” and / or “that object”) and / or input representing a pointing gesture toward a general location, area, and / or object. This contrasts with some home control devices available today that require interaction with specific pre-programmed buttons (e.g., physical or virtual buttons) or the use of pre-programmed identifiers (e.g., names) associated with a device and / or room in order to adjust the lighting of that device and / or room (e.g., “turn off the living room light” or “turn off table lamp 2”).

[0248] In some examples, light source 1001 determines one or more characteristics of the illumination (e.g., size, shape, and / or intensity) based on one or more attributes of an object at the location associated with the request (e.g., dynamically). For example, in Figure 10B If wall 1006 includes a painting suspended thereon and person 1002 points to the painting and issues a voice command “Please illuminate the painting with bright white light”, then light source 1001 can detect the request (e.g., indicating input of a pointing gesture and voice command) and, in response, perform one or more operations: determine the location and / or area pointed to by the user, identify the painting, determine the size (e.g., shape) of the painting, and / or illuminate the painting with bright white light.

[0249] Figures 10C to 10D This illustrates an example technique for using lighting to identify the location of an object. Figure 10CA physical space 1000 is shown, containing a room with a person 1002 and a key 1020 (an example of an object). In some examples, a light source 1001 receives a request to identify the location of one or more objects. For example, in Figure 10C In this scenario, after person 1002 loudly says, "Where are my keys?", light source 1001 detects input indicating that person 1002 is looking for their keys 1020. In some examples, this is in response to receiving a request to identify the location of one or more objects and when wall 1006 is illuminated (e.g., as mentioned above regarding...). Figure 10B As described, light source 1001 determines the position of one or more objects. For example, in Figure 10C In this embodiment, light source 1001 uses data from one or more sensors communicating with it to determine the location of key 1020 within physical space 1000. Examples of sensors include image sensors and position sensors. Light source 1001 may also use stored position data to determine the location of an object (e.g., the object's location is tracked and recorded, so determining the location includes accessing data from such records). In some examples, light source 1001 uses visual illumination (e.g., a spot of light output to physical space 1000) to indicate the object's location. For example, in... Figure 10C In the diagram, illumination 1022 is shown. Illumination 1022 can be used to indicate the current location of the search (e.g., light source 1001 has determined the location of key 1020). Figures 10B to 10D As shown in the diagram, spotlight 1022 can also be displayed to transition from the initial position (e.g., wall 1006) to the current position of the search by moving the lighting between the two positions.

[0250] Figure 10D Illumination 1022 of illumination key 1020 is shown. In some examples, in response to receiving a request to identify one or more objects, light source 1001 illuminates one or more objects. For example, in Figure 10D In this configuration, light source 1001 determines the position of key 1020 and illuminates key 1020 by outputting illumination 1022. As described above, in some examples, light source 1001 moves illumination (e.g., a spotlight) to the position of one or more objects within physical space 1000. For example, in... Figure 10CIn this embodiment, illumination 1022 begins at wall 1006 and can then move continuously (e.g., smoothly) toward key 1020 until key 1020 is illuminated. In some examples, continuous movement includes not stopping before reaching the destination. In some examples, continuous movement follows the shortest line from the starting point to the destination. In some examples, smooth movement includes moving at a constant and / or consistent rate. In some examples, smooth movement includes moving in a straight line. In some examples, smooth movement includes moving along a specific surface and / or surface type. In some examples, smooth movement includes moving at a consistent and / or constant distance from one or more people (e.g., 1002) in physical space 1000. In some examples, smooth movement includes moving at a rate corresponding to a mathematical equation (e.g., a bell curve, exponential, linear, and / or other mathematical equation). The movement using illumination allows light source 1001 to effectively draw the user's attention to the requested location of one or more objects (e.g., the user can more quickly recognize the movement and follow it to the object).

[0251] In some examples, the lighting changes size as it moves through physical space. For example, in Figure 10C In this design, the lighting 1022 is a small square, and as it moves toward the key 1020, the size of the lighting expands and / or changes shape to match the requested object position. Figure 10D In this process, the light source 1001 changes the shape and / or size of the illumination as the illumination 1022 moves (and / or in response to the illumination (e.g., reaching) the position of the key 1020), and the illumination is now larger than Figure 10C Lighting 1022. In some examples, the initial size of lighting 1022 depends on the size of the object at the initial position and / or the lighting at the initial position. For example, Figure 10C The size of the lighting 1022 can be based on a first object (e.g., wall 1006) in an initial position that is requested to be illuminated. After illuminating the first object, the light source 1001 receives a request to locate the key 1020 and moves from the initial position of the first object until it reaches the position of the key 1020, i.e., the second object at the second position. During this movement (and / or in response to reaching the position of the key 1020), the size, shape, and / or properties of the lighting 1022 can be changed (e.g., to match the size of the key 1020).

[0252] In some examples, as the illumination changes (e.g., moves), the light source 1001 remains stationary (e.g., does not move). In some examples, the light source 1001 consists of multiple light sources that output illumination in different directions (e.g., sufficient to cover the field of view of physical space 1000 from the angle of the light source 1001), and illumination 1022 can be formed by selectively adjusting (e.g., turning on or off and / or changing illumination properties) one or more subsets of the multiple light sources constituting the light source 1001 to form illumination 1022 (e.g., in...). Figure 10C and Figure 10D (In the middle). The light source 1001, which remains stationary when the lighting is changed (e.g., moved), contrasts with other techniques for moving lighting that require moving components (e.g., moving the light source and / or one or more lenses and / or shutters).

[0253] In some examples, one or more lighting properties can be configured based on the confidence level of the object's location at the illuminated position. For example, in Figure 10D In this scenario, key 1020 is not obstructed by one or more image sensors communicating with light source 1001 (e.g., located on the ceiling), and light source 1001 has high confidence that key 1020 is in a location illuminated by illumination 1022, which may have bright illumination to indicate high confidence. If light source 1001 has less confidence in the location of key 1020 than high confidence (e.g., if key 1020 is partially or completely obstructed by a piece of furniture), the illumination at that location may appear larger (e.g., covering a larger area) and / or darker (e.g., compared to the situation described for a high-confidence scenario).

[0254] In some examples, one or more characteristics of the lighting are based on the location of the device that person 1002 is interacting with. For example, such as Figure 10E As shown, the smart speaker 1030 is illuminated by lighting 1032. In this example, a person 1002 interacts with the smart speaker 1030 using voice input, which requests the smart speaker 1030 to cause the display device 1040 to start playing media (e.g., playing a movie). Figure 10E As shown, the smart speaker 1030 is illuminated (e.g., when its output voice input is received and / or understood as a response or instruction). In some examples, the illumination moves to another device in response to an event (e.g., a change indicating that person 1002 should interact with it). For example, for Figure 10E In the scenario described, after the smart speaker 1030 completes its response to the person 1002 and / or successfully instructs the display device 1040 to start media playback as requested, the lighting 1032 may move to the position of the display device 1040 (or may stop the display and turn on a new spotlight at the position of the display device 1040).

[0255] Figure 11 This is a flowchart illustrating a method for transmitting information (e.g., method 1100) according to some examples. Some operations in method 1100 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0256] As described below, method 1100 provides an intuitive way to transmit information. Method 1100 reduces the cognitive burden on users when transmitting information, thereby creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to transmit information faster and more efficiently, saving power and increasing the time interval between battery charging.

[0257] In some examples, method 1100 is performed at a computer system (e.g., 100, 300, and / or 500) that communicates with a light source (e.g., a lighting device, point light source, spotlight, and / or one or more light sources integrated into a single device) (e.g., 1001, 1001A, 1001B, 1001C, and / or 1001D). In some examples, the computer system is a telephone, watch, tablet computer, fitness tracker, wearable device, accessory, speaker, lamp, head-mounted display (HMD), and / or personal computing device. In some examples, the light source is not physically connected to and / or coupled to the computer system. In some examples, the computer system communicates with one or more cameras. In some examples, one or more cameras are not physically connected to the light source.

[0258] At 1102, the computer system detects a lighting request (e.g., such as...). Figure 10A and / or Figure 10B As shown in 1002, the lighting request corresponds to a request for a corresponding area (e.g., a corresponding location, area, section, and / or component) (e.g., 1004 and / or 1006) of a physical space (e.g., a physical environment, room, office, and / or building) (e.g., 1000). In some examples, detecting a lighting request includes detecting input corresponding to the request (e.g., tap gesture, long press gesture, verbal request and / or command, physical button press, pointing input and / or air gesture, and / or rotation of a physical input mechanism). In some examples, detecting a lighting request includes receiving a message from a different computer system indicating that the request has been received by the different computer system. In some examples, a request for a corresponding area of ​​the physical space does not include requests for a first area (e.g., 1004 and / or 1006) and / or a second area (e.g., 1004 and / or 1006) of the physical space.

[0259] At 1104, in response to detecting a lighting request and based on determining that the request corresponds to a first area of ​​physical space (e.g., 1004 and / or 1006) (and / or the corresponding area includes and / or is the first area) (e.g., as described above with respect to method 700), the computer system illuminates the first area via a light source (e.g., without illuminating (and / or directly illuminating) the second area) (e.g., as...). Figure 10A , Figure 10B , Figure 10C , Figure 10D and / or Figure 10E (As shown). In some examples, illuminating the first region includes activating a light source. In some examples, illuminating the first region includes changing the light output by the light source. In some examples, illuminating the first region includes sending a request to the light source to modify the light output by the light source. In some examples, illuminating the first region includes causing a first light source (e.g., 1001, 1001A, 1001B, 1001C and / or 1001D) to be changed in a first manner (e.g., increasing / decreasing brightness, hue, intensity, and / or warmth and / or changing color), and causing a second light source (e.g., 1001, 1001A, 1001B, 1001C and / or 1001D) to be changed in a second manner different from the first manner (e.g., increasing / decreasing brightness, hue, intensity, and / or warmth and / or changing color).

[0260] At 1106, in response to detecting an illumination request and based on determining that the request corresponds to a second region in physical space that is different from (e.g., does not overlap, at least does not partially overlap, and / or is separate from) the first region (and / or the corresponding region includes and / or is the second region) (e.g., as described above with respect to method 700) (e.g., 1004 and / or 1006), the computer system illuminates the second region via a light source (e.g., without illuminating (and / or directly illuminating) the first region) (e.g., as described above with respect to method 700). Figure 10A , Figure 10B , Figure 10C , Figure 10D and / or Figure 10E(As shown). In some examples, the second illumination area includes activating a light source. In some examples, the second illumination area includes changing the light output by the light source. In some examples, the second illumination area includes sending a request to the light source to modify the light output by the light source. In some examples, the second illumination area is different from the first illumination area. In some examples, the second illumination area includes causing a third light source (e.g., 1001, 1001A, 1001B, 1001C, and / or 1001D) to be changed in a third manner (e.g., increasing / decreasing brightness, hue, intensity, and / or warmth and / or changing color), and causing a fourth light source (e.g., 1001, 1001A, 1001B, 1001C, and / or 1001D) to be changed in a fourth manner different from the third manner (e.g., increasing / decreasing brightness, hue, intensity, and / or warmth and / or changing color). In some examples, the third light source is the same as the first or second light source. In some examples, one or more of the first and second light sources are different from the third and / or fourth light sources. In some examples, the third light source is different from the fourth light source. Based on the determined request corresponding to the corresponding area, different areas of the physical space are illuminated by a light source. This allows lighting to occur automatically in the appropriate area instead of other areas without user input, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions have been met.

[0261] In some examples, detecting a lighting request includes (e.g., via one or more sensors (e.g., a camera and / or depth sensor communicating with a computer system) detecting a first pointing input in the direction (and / or pointing to) a corresponding area in the physical space (e.g., the orientation and / or pointing of an object (e.g., 1002), such as a device (e.g., a portable device, fitness tracker, wearable device, and / or remote control) (e.g., a computer system), a user's finger, hand, arm, and / or nod) (e.g., an air pointing gesture and / or input detected by a fitness tracker and / or wearable device) (or, in some examples, non-pointing input, such as a mouse click, swipe gesture / input, tap gesture / input, and / or voice command). Detecting the first pointing input in the direction of the corresponding area to illuminate a first or second area allows the user to indicate the content to be illuminated via the first pointing input, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0262] In some examples, detecting a lighting request includes a request to detect (e.g., receive, obtain, and / or acquire) the location (e.g., current, precise, and / or last identified location) (e.g., 1020 and / or 1030) of an identifiable object (e.g., movable objects such as portable devices, keys, books, people, and / or tablets, or immovable objects such as walls, rooms, areas, sofas, or tables) (e.g., 1022 and / or 1032). Detecting the location of an identifiable object to illuminate a first or second area allows the user to identify where an object is located via a lighting request, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0263] In some examples, the location of an object is specified in a lighting request (e.g., and not solely determined by a computer system) (e.g., a user requests lighting on a painting on a wall). In other examples, the location of an object is determined based on a lighting request (e.g., without any other input).

[0264] In some examples, the object's location is determined by the computer system (e.g., without the user specifying the location in the lighting request). In some examples, the object's location is determined after a lighting request is detected. In some examples, the lighting request includes an identification of the object. In some examples, after a lighting request is detected, the computer system determines the object's location by locating the object in physical space (e.g., 1022 and / or 1032). A computer system that determines the object's location allows the user to identify the object's location even when the user is unaware of its location, thereby reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input once a set of conditions has been met.

[0265] In some examples, in response to detecting a lighting request, the first region is illuminated in a first manner (e.g., the color, intensity, color temperature, and / or size of the lighting) based on determining that the request corresponds to a first region and determining that the object has a first probability of being in the first region (e.g., a first confidence level and / or accuracy level). In some examples, the first region is illuminated in a second manner (e.g., the color, intensity, color temperature, and / or size of the lighting) different from the first manner (e.g., not illuminating the first region in the first manner) based on determining that the request corresponds to a first region and determining that the object has a second probability of being in the first region (e.g., different from the first probability) (e.g., a second confidence level and / or accuracy level). In some examples, the same object is illuminated differently depending on the computer system's confidence level regarding the object's position in the corresponding region. In some examples, the second region is illuminated in a first manner based on determining that the request corresponds to a second region and determining that the object has a third probability of being in the second region (e.g., a third confidence level and / or accuracy level); and in a second manner based on determining that the request corresponds to a second region and determining that the object has a fourth probability of being in the second region (e.g., different from the third probability) (e.g., a fourth confidence level and / or accuracy level). In some examples, the first probability is above a threshold confidence level, while the second probability is below a threshold confidence level, and / or vice versa. Illuminating the first region differently based on the probability of the determined object allows users to understand the probability at a glance, thus providing improved visual feedback to the user and enabling them to perform actions when a set of conditions has been met without further user input.

[0266] In some examples, illuminating a first region via a light source includes: moving the illumination from a first portion of the first region to a second portion of the first region at a first rate within a first time range (e.g., as shown in the image). Figure 10B and Figure 10C (as shown in the diagram); and after the first time range and within the second time range, the illumination of the second part of the first region is moved to the third part of the first region at a first rate (e.g., as shown in the diagram). Figure 10C and Figure 10D(as shown in the diagram), wherein the second portion of the first region is adjacent to the first portion and the third portion of the first region (e.g., sharing sides and / or boundaries and / or being adjacent to them). In some examples, the second portion of the first region differs from (e.g., not included in and / or excluded from) the first portion of the first region, the first portion of the first region differs from the third portion of the first region, and the third portion of the first region differs from the second portion of the first region. In some examples, the first time range and the second time range are of the same duration. In some examples, the light source moves over the first region at a constant and / or regular rate. In some examples, each of the multiple regions is adjacent to at least one other region among the multiple regions. In some examples, the first region is adjacent to one of the multiple regions. In some examples, the multiple regions include the first region. In some examples, each of the multiple regions is illuminated for the same amount of time. In some examples, the multiple regions are illuminated continuously and sequentially. In some examples, sequentially illuminating the multiple regions includes illuminating the first region without illuminating the second region, and illuminating the second region without illuminating the first region after illuminating the first region. In response to the detection of a lighting request, different portions of a first area are illuminated at a first rate, allowing the user to follow the position of the lighting as it moves, thereby providing the user with improved visual feedback and enabling the user to perform an action when a set of conditions has been met without requiring further user input.

[0267] In some examples, the light source remains in a specific position (and / or is positioned) when the illumination of a first portion of a first region is moved to a second portion of the first region and the illumination of the second portion of the first region is moved to a third portion of the first region. In some examples, the light source does not move while sequentially illuminating portions of the region. Maintaining a specific position of the light source while moving the illumination allows for less movement of the light source, thereby reducing wear on the components of the light source.

[0268] In some examples, the detection lighting request includes detecting input (e.g., pointing input (as described above) and / or non-pointing input (as described above)) (e.g., pointing... Figure 10A 1004 and Figure 10B(As shown in Figure 1006, 1002). In some examples, one or more portions of the first region are identified based on input. In some examples, the lighting request identifies multiple locations corresponding to multiple regions and / or one or more portions of a corresponding region. In some examples, the multiple regions and / or one or more portions of a corresponding region are regions and / or portions of regions between the corresponding region and the location illuminated when a lighting request is detected. In some examples, different regions and / or portions of regions are identified when different inputs are detected. Identifying one or more portions of the first region based on the detected input allows lighting movement to align with where the user is looking, thereby providing the user with improved visual feedback and performing actions without further user input once a set of conditions have been met.

[0269] In some examples, the size of an object is determined to be a first size in a first portion of a first region, and the size of a second object (e.g., the first object, another portion of the first object, and / or an object different from the first object) is a second size in a second portion of the first region, where the first size is smaller than the second size, and the lighting intensity of the first portion of the first region is smaller than the lighting intensity of the second portion of the first region. In some examples, the size of an object is determined to be a first size in a first portion of a first region, and the size of a second object is a third size in a second portion of the first region, where the first size is larger than the third size, and the lighting intensity of the first portion of the first region is greater than the lighting intensity of the second portion of the first region (e.g., as shown in the image). Figure 10B and Figure 10D As shown in the figure, where Figure 10B 1016 in the middle is greater than Figure 10D(1022 in the text). In some examples, based on determining that a first region (e.g., the illuminated area) will be illuminated by a first amount (e.g., the intensity and / or size of the illumination) and based on determining that a third region (e.g., the currently illuminated area) will be illuminated when an illumination request is detected by a second amount (e.g., the intensity and / or size of the illumination) (e.g., the same as and / or different from the first amount), a fourth region among multiple regions (e.g., the region between the first and third regions) is illuminated by a third amount (e.g., the intensity and / or size of the illumination) different from the first and / or second amounts; and based on determining that the first region will be illuminated by a first amount and based on determining that the third region will be illuminated by a fourth amount (e.g., the intensity and / or size of the illumination) different from the second amount. The illumination may be a fifth quantity (e.g., the intensity and / or size of the illumination) that differs from the first and / or third quantities in intensity and / or size; or a sixth quantity (e.g., the intensity and / or size of the illumination) that differs from the first quantity in intensity and / or size in ... Figure 10B , Figure 10C and Figure 10D As shown in the figure, where Figure 10B 1016 in Figure 10C It becomes smaller in 1022, and then Figure 10D (In 1022, the size changes), where the size variation depends on the size of the lighting at the original and new positions. Having different sizes of lighting while moving allows for signaling the size of an object (or the probability that the object is in a lit position) before reaching it, thus providing improved visual feedback to the user and enabling actions to be performed when a set of conditions are met without further user input.

[0270] In some examples, the lighting request corresponds to a request for output content (e.g., visual and / or auditory content) from a device that is different from a computer system (e.g., a smart speaker, television, telephone, smartwatch, and / or wearable device) (e.g., 1030 and / or 1040). In some examples, the device that is different from a computer system is a smart speaker (e.g., 1030). In some examples, the device that is different from a computer system is a television (e.g., 1040). In some examples, the corresponding area is the first area (e.g., as determined by the determination that a first area includes the device). Figure 10E(As shown in 1032). In some examples, the second area is defined as including the device, and the corresponding area is the second area. In some examples, the lighting is based on the location of the device with which the user is interacting and / or requesting to perform an action. In some examples, the device is not a computer system and / or the device interpreting the voice request. In some examples, detecting the lighting request includes detecting user interaction with a device other than a computer system. In some examples, user interaction is a request to cause the device to output and / or adjust content, the user looking at and / or gazing at the device, and / or the user providing input to the device (e.g., via touch input / gesture, air gesture, voice command, and / or another type of gesture).

[0271] In some examples, illuminating the first area includes: providing a first type of illumination (e.g., size, shape, and / or intensity) via a light source based on determining that the illumination request corresponds to a first object (e.g., as described above with respect to method 700); and providing a second type of illumination (e.g., size, shape, and / or intensity) different from the first type of illumination via a light source based on determining that the illumination request corresponds to a second object different from the first object (e.g., as described above with respect to method 700) (e.g., as described above with respect to method 700). Figure 10D As shown in 1022, it is similar to Figure 10E (Different from 1032 in the original text). In some examples, the lighting type is dynamically determined based on the object's location. In other examples, the lighting type is dynamically determined based on the object's type. Providing different types of lighting based on the determined lighting request corresponds to different objects allows users to identify the object's type through the lighting request, thereby reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input when a set of conditions has been met.

[0272] In some examples, in response to detecting that the second pointing input is no longer facing the first area and based on determining that a corresponding input has been detected (e.g., locking the input and / or indicating that the lighting should not only correspond to the direction of the pointing input) (e.g., when illuminating the first area and / or relative to a lighting request), the computer system continues to illuminate the first area via a light source (and in some examples, maintains the illumination of the first area and / or the same type of lighting) (e.g., as described above regarding...). Figure 10B (As described above). In some examples, in response to detecting that the second pointing input is no longer facing the first area and based on determining that no corresponding input has been detected (e.g., when illuminating the first area and / or relative to an illumination request), the computer system stops illuminating the first area via a light source (e.g., as described above regarding...). Figure 10B(As described). The first area stops lighting based on the determination that no corresponding input has been detected, allowing the user to choose when to keep the lighting on, thereby reducing the amount of input required to perform an operation. It provides additional control options without cluttering the user interface with additional displayed controls, and performs an operation without further user input once a set of conditions has been met.

[0273] In some examples, the lighting request does not include an identifier for the corresponding area (e.g., a first area or a second area). In some examples, the request corresponding to the first area does not include a request to identify the first area (e.g., by an identifier, name, text representation, and / or naming associated with the first area). In some examples, the request corresponding to the second area does not include a request to identify the second area (e.g., by an identifier, name, text representation, and / or naming associated with the second area). In some examples, the corresponding area is automatically identified (e.g., by a computer system and / or another computer system).

[0274] It should be noted that the above is relative to method 1100 (e.g., Figure 11 The details of the process described herein also apply in a similar manner to the methods described herein. For example, method 700 optionally includes one or more features of the various methods described above with reference to method 1100. For example, the light source of method 700 may be the light source of method 1100. For the sake of brevity, these details will not be repeated below.

[0275] Figures 12A to 12D Exemplary techniques for providing a representation of a physical space scenario are illustrated according to some examples. The user interface in these figures is used to illustrate the processes described below, including... Figure 13 The process in.

[0276] Figure 12A A physical space 1200 is shown, comprising a room with a light source 1201 and walls 1202 (physical features). In some examples, the light source 1201 includes one or more features as described herein with respect to any one or more light sources, as described herein with respect to Figures 6, 8, 10 and / or 14. Figure 12A An environment 1210 (shown in box A) including people 1212 and 1214 is also shown, and an environment 1220 (shown in box B) including a tree 1222 is also shown.

[0277] In some examples, light source 1201 illuminates an area of ​​physical space 1200 (e.g., wall 1202) using a representation of the environment (e.g., external environment). For example, Figure 12AThe diagram shows light source 1201 with output representations 1204A and 1204B, both indicating the context of environment 1210. In this example, environment 1210 is the external environment (e.g., outside physical space 1200). Environment 1210 can be the environment directly behind wall 1202, or any other arbitrary physical location outside physical space 1200. In some examples, the context of the environment includes one or more of the following: the number of people in the environment, the overall activity level in the environment, lighting conditions (e.g., the position of the sun and / or the color of sunlight) and / or weather conditions (e.g., windy, rainy, sunny, and / or cloudy). Figure 12AIn the above, 1204 (used collectively to refer to 1204A and 1204B) each represents a person in environment 1210. Specifically, 1204A represents person 1212 in environment 1210, and 1204B represents person 1214 in environment 1210. In some examples, light source 1201, in conjunction with (e.g., in response to, as part of, simultaneously with, and / or with) an external computer system that receives a communication request (e.g., a computer system outside of light source 1201 and / or a computer system communicating with light source 1201), outputs and / or modifies the appearance of 1204A and 1204B. For example, in response to an external computer system receiving a communication request, light source 1201 outputs representations 1204A and 1204B. Light source 1201 changes the appearance of representations 1204A and / or 1204B to an appearance associated with the communication request (e.g., the environment from which the communication request originates and / or the current lighting from the external computer system). The external computer system receives the request to change the appearance of representations 1204A and / or 1204B upon receiving the communication request and / or after the communication request is accepted, and / or the external computer system can detect the request to change representations 1204A and / or 1204B based on the communication request. In some examples, the communication request originates from environment 1210 and / or a computer system within environment 1210. For example, the communication request is initiated by a user located in environment 1210 and / or a computer system located within an external environment. In some examples, the communication request includes an identifier of environment 1210. For example, a user of another computer system (e.g., a computer system located within environment 1202 and / or a computer system outside environment 1202) may select environment 1210 as the basis for illuminating environment 1202 relative to a communication request, and an instruction for environment 1210 is sent along with the communication request. In some examples, the external computer system receiving the communication request may detect input for selecting environment 1210 to be used as the basis for illuminating environment 1202 before or after receiving the communication request. For example, when the external computer receives a communication request from a specific user, using a specific application, and / or for a specific type of communication (e.g., audio or video), the user may configure environment 1210 as the basis for illuminating environment 1202. In some examples, in response to the communication request being accepted (e.g., as discussed above) (e.g., accepted by a user of the external computer system), the computer system located within environment 1210 connects to the external computer system located in the external environment. In some examples, when the computer system located within environment 1210 connects to the external computer system, light source 1201 illuminates environment 1202 based on the illumination of the environment of the external computer system.

[0278] In some examples, in response to an external computer system receiving and / or generating a notification, light source 1201 modifies the appearance of representations 1204A and / or 1204B. For example, in response to an external computer system receiving and / or generating a notification, the appearance of representations 1204A and / or 1204B will pulse. In some examples, in response to an external computer system detecting input corresponding to a selection of a user interface element, light source 1201 illuminates environment 1202 based on the current lighting of the external environment. In some examples, the external computer system displays user interface objects as part of a preview of communication between the computer system located within environment 1210 and the external computer system. For example, in response to receiving a communication request (e.g., as described above), the external computer system displays user interface objects with representations of the user of the external computer system (e.g., text representations and / or graphical representations), real-time feeds from the user of the external computer system, and / or real-time feeds from the user of the computer system located within environment 1210. In some examples, the external computer system displays user interface elements as part of a representation of communication between the external computer system and the computer system located within environment 1210.

[0279] In some examples, light source 1201 outputs a representation of a scene (e.g., 1204A) as an area with reduced illumination (e.g., less direct illumination than the area surrounding the representation). In some examples, light source 1201 outputs a representation of a scene (e.g., 1204A) as an unilluminated area (e.g., an area without direct illumination from light source 1201). For example, a representation formed as an area with reduced illumination or no illumination may appear darker than the area surrounding the representation (e.g., having an appearance with outlines and / or shadows). This can be achieved by light source 1201 illuminating the area surrounding the representation but not illuminating the area within the representation (or illuminating it with less or different illumination). In some examples, light source 1201 outputs a representation by projecting an image and / or illumination within the area of ​​the representation. For example, light source 1201 may output representation 1204 as an image and / or representation of person 1212, rather than as an outline.

[0280] like Figure 12AAs shown, 1204A is an example of an abstract representation of person 1212, and 1204B is an example of an abstract representation of person 1214. 1204A and 1204B together form an abstract representation of a scenario 1210, where the scenario is user activity. 1204A and 1204B generally represent activities, but are not detailed representations of people 1212 and 1214. For example, 1204A and 1204B represent the corresponding sizes of people 1212 and 1214 (e.g., person 1212 is larger than person 1214, therefore 1204A is larger than 1204B), the corresponding positions of people 1212 and 1214 (e.g., 1204A is to the left of 1204B, which matches the position of the person in environment 1210), and the corresponding movements of people 1212 and 1214 (e.g., 1204A and 1204B move to the left to match the movement of people 1212 and 1214 walking to the left in environment 1210). 1204A and 1204B represent the size, position, and movement of people 1212 and 1214, but do not include... Figure 12A The environment 1210 shows people 1212 and 1214 at the same level of detail. For example, representations 1204A and 1204B do not include details revealing shape, facial features, gender, or allow for unique identification. Figure 12A Details of other characteristics of the person in environment 1210. In some examples, the representation (e.g., 1204A and / or 1204B) is an abstract representation of the scene (e.g., a representation of at least one visible attribute of the scene in the abstract environment). In some examples, the representation is not an abstract representation of the scene (e.g., it does not abstract at least one visible attribute) (e.g., the representation is a captured image or video of the scene). In some examples, when a user requests representation 1204A and / or representation 1204B to have an appearance simulating a specific time of day, light source 1201 outputs representation 1204A and / or representation 1204B with an appearance based on the time of day. For example, when a user requests representation 1204A and / or representation 1204B to have an appearance based on sunrise, light source 1201 outputs representation 1204A and / or representation 1204B with an appearance based on sunrise. In some examples, when a user requests that representation 1204A and / or representation 1204B have an appearance simulating a specific event, light source 1201 outputs representation 1204A and / or representation 1204B with an event-based appearance. For example, when a user requests that representation 1204A and / or representation 1204B have an eclipse-based appearance, light source 1201 outputs representation 1204A and / or representation 1204B with an eclipse-based appearance.

[0281] In some examples, a user can select a location within environment 1202 (e.g., the ceiling and / or a wall) where light is simulated to originate from that location. For example, light source 1201 can simulate light originating from the left wall of environment 1202 based on user input. In some examples, a representation (e.g., 1204A and / or 1204B) is an abstract representation of the shadows of one or more objects positioned within environment 1202 based on the user-defined location of the light source. For example, when the user selects a light source originating from the right wall of environment 1202, the representation may appear to the left of the object.

[0282] In some examples, light source 1201 illuminates environment 1202 based on illumination (e.g., current illumination, previous illumination, and / or future illumination) of the external environment (e.g., the environment outside environment 1202) (e.g., environment 1210 and / or environment 1220). When light source 1201 illuminates environment 1202 based on illumination from the external environment, light source 1201 simulates the position of the light source within the external environment. For example, if it is noon in the external environment, light source 1201 will simulate the sun at the center of environment 1202. In some examples, when light source 1201 illuminates environment 1202 based on illumination from the external environment, light source 1201 simulates the shadows of objects in the external environment. For example, if the external environment includes buildings with shadows pointing west, light source 1201 will simulate the shadows of west-pointing buildings within environment 1202.

[0283] Figure 12B The diagram illustrates physical space 1200, containing a room designated 1204C. In some examples, the level of abstraction for the representation (e.g., 1204C) can be a higher level of abstraction (e.g., higher than...). Figure 12A (as shown) or a lower level of abstraction (e.g., below) Figure 12A Any level between (as shown). In some examples, the abstract quantity represented is configurable (e.g., can be selected and / or adjusted by the user). Figure 12B In the text, it indicates that 1204C is... Figure 12B Box A represents an abstract representation of the scenario (general state of activity) of environment 1210. At least for the reasons discussed below, representation 1204C has a higher level of abstraction than representation 1204B. Figure 12BIn this context, 1204C represents an abstraction of person 1214. Specifically, 1204C abstracts the position and movement of person 1214. Representation 1204C shows person 1214 on the left side of wall 1202 and moving to the left, while person 1214 is on the right side of environment 1210 and moving to the right (e.g., walking). It should be understood that representation 1204C indicates that a person exists in the environment, but does not specifically indicate where they are or their particular direction or speed of movement. By not indicating the specific position or movement of the corresponding person, representation 1204C is a higher-level abstraction of position and location than representation 1204B. In other words, representation 1204C indicates that a person is located in the environment and is moving, but not necessarily where or how fast they are moving. In some examples, the level of abstraction of one or more attributes can be combined with any other level of abstraction of one or more other attributes. For example, representation 1204C could represent the speed of movement of person 1214, but not location information (e.g., representing 1204C moving to the left (or in any arbitrary direction) at the same speed as person 1214 walking to the right).

[0284] Representation 1204C also abstracts the identity of person 1214 (as described above regarding representation 1204B). Representation 1204C also abstracts the size of person 1214. In some examples, the size of representation 1204C does not correspond to the size of person 1214. For example, if both person 1212 and person 1214 are instantiated as... Figure 12B If the representation in the table is the same, then its corresponding representation can be of the same size. In some examples, the representation 1204C represents multiple people (e.g., such as...). Figure 12A The environment 1210 is an abstract representation of both people 1212 and people 1214. For example, a single representation 1204C can be used to represent more than one person, object, or a combination thereof. In some examples, the size of the abstract representation (e.g., 1204C) indicates the context of the environment. For example, a light source 1201 could increase the size of representation 1204C to reflect a high level of activity in the environment 1210 (e.g., appearing larger when more people are detected and smaller when fewer people are detected).

[0285] In some examples, light source 1201 receives a request to illuminate an abstract representation of a different environment (e.g., a physical space). This request could be to change the current abstract representation to represent an environment different from the currently represented environment. For example, the different environment could be another physical space (e.g., a room in a different house than the house where physical space 1200 is located) or an environment (e.g., an outdoor space at any location, not limited to the vicinity of physical space 1200). For example, the request could originate from a user in physical space 1200. For instance, a user of light source 1201 could request to view environment 1220.

[0286] Figure 12CA physical space 1200 is shown, containing a room designated 1206. A light source 1201 outputs light onto a wall 1202, representing 1206 as an area with reduced illumination (e.g., an outline). Figure 12C In the middle, it means that 1206 is... Figure 12C Box B represents an abstract representation of the scenario (e.g., weather) of environment 1220. Specifically, representation 1206 is an abstract representation of tree 1222 in environment 1220. As shown, representation 1206 is the outline of tree 1222, which resembles the shape of tree 1222 but does not include all details (e.g., details of individual leaves).

[0287] In some examples, representation 1206 includes environmental context, such as weather, climate, and / or lighting conditions. For example, light source 1201 may output representation 1206 to indicate how the windy environment 1220 moves (e.g., swaying) (e.g., swaying motion increases with more windy conditions). In some examples, light source 1201 may also output representation 1206 to indicate the direction of wind, such as by swaying in a certain direction and / or outputting additional representations of wind moving in a particular direction (e.g., one or more arrows and / or lines with spiral ends, typically used to exemplify wind).

[0288] In some examples, representation 1206 includes a representation of the sun's position within environment 1220. For example, light source 1201 may output representation 1206 as including a representation of the time of day (or a new representation that outputs a representation of the time of day). The representation of the time of day may include a representation of the sun (e.g., represented as an area of ​​increased illumination of the photosphere). Light source 1201 may output a representation of the sun (and / or wall 1202 or the remainder of physical space 1200) to resemble or match the color temperature of sunlight currently in environment 1220. For example, light source 1201 illuminates physical space 1200 with bright white light during the middle of the day, but with a dimmer orange-red light during the period before and after sunset. In some examples, light source 1201 uses representations of sunrise and / or sunset to illuminate physical space 1200. For example, the representation of the sun may appear on a simulated horizon (e.g., the bottom of wall 1202) and rise simultaneously with sunrise in the environment (e.g., based on geographic location or sensor data associated with environment 1220). In some examples, without detecting user input, light source 1201 moves the representation of the sun within environment 1202 based on detected time passage and / or a predicted, estimated, and / or determined position of the sun within environment 1202. For example, light source 1201 may initially output a representation of the sun within the left side of environment 1202 and gradually move the representation of the sun toward the right side of environment 1202 as the day progresses. In some examples, in response to a computer system located within environment 1202 detecting input, light source 1201 may illuminate a representation of the sun having an appearance corresponding to different times of day (e.g., sunrise, noon, and / or sunset). For example, a user can cause light source 1201 to illuminate the representation of the sun with an appearance corresponding to the sun at sunrise or sunset by selecting settings of the computer system located within environment 1202. It should be recognized that the sun is only one example of an object represented by light source 1201, and other objects, including other celestial objects (e.g., the moon or stars), may be represented by light source 1201.

[0289] like Figure 12CAs shown, representation 1206 excludes an abstract representation of a person 1212 in environment 1210 shown in box A. For example, the scenario selected to be represented by representation 1206 is a scenario of environment 1220 different from environment 1210. In some examples, if person 1212 is present in environment 1220, light source 1201 does not output an abstract representation of person 1212 in physical space 1200 (e.g., on a portion of wall 1202, such as representation 1206). For example, if the selected scenario to be represented is weather, light source 1201 does not include an abstract representation of a person (e.g., light source 1201 ignores person 1212 during the creation of one or more abstract representations of the weather scenario of environment 1220). In some examples, environment 1210 and environment 1220 are the same environment, where box A shows a first scenario (e.g., may include people and exclude general activity of static objects such as tree 1222), and where box B shows a second scenario (e.g., may include trees and exclude weather for people and / or buildings). In some examples, when environment 1210 is different from and / or distinguished from environment 1220, light source 1201 simultaneously illuminates abstract representations of the scene from both environment 1210 and environment 1220 within environment 1202. In some examples, light source 1201 transitions from outputting a representation corresponding to environment 1210 to outputting a representation corresponding to environment 1220 at a first rate. In some examples, light source 1201 transitions from outputting a representation corresponding to environment 1210 and / or environment 1220 to outputting a representation corresponding to environment 1202 at a second rate faster than the first rate. In some examples, light source 1201 gradually transitions from outputting a representation corresponding to environment 1210 and / or environment 1220 to outputting a representation corresponding to environment 1202 over a time period.

[0290] Figure 12D A physical space 1200 is shown, containing a room with representation 1208. A light source 1201 outputs representation 1208 on a wall 1202 as an area with reduced illumination (e.g., an outline). Representation 1208 is... Figure 12D The image in box B is an abstract representation of the weather scenario of environment 1220. Specifically, representation 1208 represents raindrops representing the current weather of environment 1220 (e.g., a place where it is currently raining). Figure 12D In this context, light source 1201 does not output a representation of general activity (e.g., such as person 1212 in environment 1220) because it is based on the context of environment 1210. Figure 12C The difference lies in Figure 12D In this context, light source 1201 does not output a representation of the tree (within environment 1220). Light source 1201 can output lighting representing the scene in many ways. For example... Figure 12DAs shown, the representation of weather does not necessarily include objects in the environment (e.g., tree 1222). In some examples, light source 1201 outputs an abstract representation of tree 1222 as part of or supplement to the rain representation 1208. For example, Figure 12D The scenarios shown may additionally include those from Figure 12C The representation is 1206, and it represents a later time within environment 1220 (e.g., in...). Figure 12C In the middle, the condition is that there is wind, and in Figure 12D In the example above, the condition changes to rain. In some examples, the representation of the scenario changes over time. For example, as described above, the weather can change, general activities can change, and / or the color of the light can change.

[0291] Figure 13 This is a flowchart illustrating a method for providing a scenario based on some examples (e.g., method 1300). Some operations in method 1300 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0292] As described below, method 1300 provides an intuitive way of providing scenarios. Method 1300 reduces the cognitive burden on the user when providing scenarios, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to provide scenarios more quickly and efficiently saves power and increases the time interval between battery charging.

[0293] In some examples, method 1300 is performed at a computer system (e.g., 100, 300, and / or 500) communicating with a light source (e.g., a lighting device, point light source, spotlight, and / or one or more light sources) (e.g., 1201, 1201A, 1201B, 1201C, and / or 1201D). In some examples, the computer system is a telephone, watch, tablet computer, fitness tracker, wearable device, accessory, speaker, lamp, head-mounted display (HMD), and / or personal computing device. In some examples, the light source is not physically connected to and / or coupled to the computer system. In some examples, the computer system communicates with one or more cameras. In some examples, one or more cameras are not physically connected to the light source.

[0294] At 1302, the computer system detects a request to illuminate an area (e.g., as described above with respect to method 700) of the first physical space (e.g., a physical environment, a at least partially enclosed area, a room, an office, and / or a building) (e.g., 1200). In some examples, detecting the request includes detecting input corresponding to the request (e.g., a tap gesture, a long press gesture, a verbal request and / or command, a physical button press, a pointing gesture and / or an air gesture, and / or a rotation of a physical input mechanism). In some examples, detecting the request includes receiving a message from a different computer system indicating that the request has been received by that different computer system. In some examples, detecting the request is independent of input. In some examples, detecting the request includes detecting events that have occurred in the first physical space and / or the second physical space.

[0295] At 1304, in response to a request to detect an area of ​​the first physical space being illuminated and based on the determination that the second physical space (e.g., 1210 and / or 1220) has a first scenario (e.g., a scenario including one or more lighting attributes and / or light attributes, such as the color, brightness, intensity, warmth, and / or hue of the light and / or illumination of the physical space), the computer system illuminates the area of ​​the first physical space via a light source to include (e.g., and / or cause the area of ​​the first physical space to include) a first abstract representation (e.g., 1204A, 1204B, 1204C, 1204B) corresponding to the first scenario of the second physical space. 06 and / or 1208 (e.g., including a representation of one or more lighting attributes (e.g., color, hue, brightness level, and / or intensity level) of a first scene corresponding to and / or matching a second physical space and / or a second physical space at a time instance) (e.g., excluding an abstract representation of a second scene corresponding to a second physical space (e.g., described below), wherein the second physical space is outside the first physical space (e.g., different from, independent of, on opposite sides of a surface, and / or at a different location in the physical environment) (and in some examples, the first physical space is not included within the second physical space). In some examples, the first abstract representation corresponding to the first scene includes a representation of one or more objects (e.g., 1212, 1214, and / or 1220) detected in the second physical space. In some examples, the first abstract representation corresponding to the first scene includes one or more objects not detected in the second physical space. In some examples, the first abstract representation corresponding to the first scene indicates user activity in the second physical space. In some examples, the first abstract representation is generated from one or more images of the second physical space, such that the first abstract representation differs from one or more images. In some examples, the first abstraction indicates an activity occurring in the second physical space without displaying one or more images of the second physical space. In some examples, the first abstraction includes a blurred version of an object included in an image of the second physical space. In some examples, the first abstraction uses shadows to represent objects in the second physical space. In some examples, the first abstraction represents a camera feed different from that of the second physical space.

[0296] At 1306, in response to a request to detect an area of ​​the first physical space being illuminated and based on the determination that the second physical space has a second scenario different from the first scenario, the computer system illuminates an area of ​​the first physical space via a light source to include (e.g., and / or cause the area of ​​the first physical space to include) a second abstract representation (e.g., 1204A, 1204B, 1204C, 1206, and / or 1208) corresponding to the second scenario of the second physical space, which is different from the first abstract representation corresponding to the first scenario of the second physical space (e.g., including representations of one or more illumination attributes (e.g., color, hue, brightness level, and / or intensity level) corresponding to and / or matching the first scenario of the second physical space and / or the second physical space at a time instance) (e.g., excluding the abstract representation corresponding to the first scenario of the second physical space). In some examples, the second abstract representation corresponding to the second scenario includes representations of one or more objects detected in the second physical space. In some examples, the second abstract representation corresponding to the second scenario includes one or more objects not detected in the second physical space (e.g., as described above regarding...). Figure 12B (As described above). In some examples, the second abstract representation corresponding to the second scenario indicates user activity in a second physical space (e.g., as described above regarding...). Figures 12A to 12B (As described). In some examples, illuminating the first region includes activating a light source. In some examples, illuminating the first region includes altering the light output by the light source. In some examples, illuminating the first region includes sending a request to the light source to modify the light output by the light source. In some examples, the second abstract representation is generated from one or more images of a second physical space, such that the second abstract representation differs from one or more images. In some examples, the second abstract representation indicates an activity occurring in the second physical space without displaying one or more images of the second physical space. In some examples, the second abstract representation includes blurred versions (e.g., 1204A, 1204B, and / or 1206) of objects (e.g., 1212, 1214, and / or 1222) included in the images of the second physical space. In some examples, the second abstract representation uses shadows to represent objects in the second physical space. In some examples, the second abstract representation differs from the camera feed of the second physical space. Illuminating the area of ​​the first physical space to include different abstract representations of the scenario corresponding to the second physical space allows information about the second physical space to be known even when the second physical space is outside the first physical space, thereby providing the user with improved visual feedback, reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input when a set of conditions has been met.

[0297] In some examples, the illuminated area includes a portion of the illuminated area to form a simulated outline representing a first person (e.g., 1212 and / or 1214) (e.g., the shape of a person and / or the shape of a representation of a person (such as a spot or other object)). This outline is used in the negative space of the area and / or in portions that are darker than the rest of the area (e.g., 1204A, 1204B, and / or 1204C). In some examples, multiple simulated outlines are used to represent different people (e.g., such as...). Figure 12A (As shown). In some examples, illuminating a portion of the area to form a simulated outline involves providing some light to the portion of the area that includes the simulated outline and providing more light to the portion of the area that does not include the simulated outline (e.g., outside the simulated outline). Illuminating a portion of the area to form a simulated outline representing a first person allows the user to identify when the person is in a second physical space, thereby providing the user with improved visual feedback, reducing the amount of input required to perform an action, providing additional control options without cluttering the user interface with additional displayed controls, and performing an action when a set of conditions has been met without requiring further user input.

[0298] In some examples, the simulated silhouette corresponds to (e.g., indicates and / or represents) a second person (e.g., 1212 and / or 1214) detected in a second physical space (e.g., via sensors such as cameras and / or motion detectors). The simulated silhouette corresponding to a second person detected in the second physical space allows the user to identify when the person is in the second physical space, thereby providing improved visual feedback to the user, reducing the amount of input required to perform an action, providing additional control options without cluttering the user interface with additional displayed controls, and performing an action without further user input once a set of conditions has been met.

[0299] In some examples, the first person was not detected in the second physical space. In some examples, the first abstract representation indicates the general level of activity in the second physical space (e.g., as mentioned above regarding...). Figure 12B (As described), such as representations excluding objects in a second physical space. In some examples, the simulated profile corresponds to the general activity level of a second person. The first person not detected in the second physical space allows the abstract representation to maintain the anonymity of people in the second physical space, thereby improving privacy.

[0300] In some examples, the illuminated area includes a portion of the illuminated area to form a simulated outline (e.g., 1222) representing a tree (e.g., the shape of one or more trees and / or the shape of a representation of one or more trees (such as a spot or other object)). This outline is used to represent different trees. Illuminating a portion of the area to form a simulated outline representing a tree allows the user to identify content in a second physical space and its current state, thus providing improved visual feedback, reducing the amount of input required to perform actions, providing additional control options without cluttering the user interface with additional displayed controls, and performing actions without further user input once a set of conditions are met.

[0301] In some examples, the lighting changes over time based on weather conditions in a second physical space (e.g., wind and / or the amount and / or direction of the sun). In some examples, the lighting changes in a first manner (e.g., as determined by a first weather condition) based on the determination that the current weather is in a first state. Figure 12C or Figure 12D As shown); and depending on the current weather being in a second state different from the first state, the lighting is changed in a second manner different from the first manner (e.g., as shown). Figure 12C or Figure 12D (As shown). Lighting based on weather changes over time allows users to identify content in a second physical space and its current state, thus providing improved visual feedback to the user, reducing the amount of input required to perform operations, providing additional control options without cluttering the user interface with additional displayed controls, and performing operations without further user input once a set of conditions have been met.

[0302] In some examples, the first abstract representation (and / or the second abstract representation) indicates the time of day (e.g., the time of day in a first physical space or the time of day in a second physical space) (e.g., morning, evening, afternoon, night, and / or 7 a.m. to 8 a.m.). In some examples, the first abstract representation includes light to indicate the current position of the sun. In some examples, the first abstract representation includes color to indicate the time of day. The first abstract representation indicating the time of day allows users to identify the time of day, thereby providing improved visual feedback to the user, reducing the amount of input required to perform an action, providing additional control options without cluttering the user interface with additional displayed controls, and performing an action without further user input once a set of conditions has been met.

[0303] In some examples, the color characteristics (e.g., color temperature, hue, intensity, and / or color saturation) of the first abstract representation (and / or the second abstract representation) are based on (e.g., in a first physical space and / or a second physical space) the time of day (e.g., the current and / or present time). In some examples, the color characteristic is a first color characteristic depending on whether the time of day is determined to be a first time of day; and a second color characteristic is a different color characteristic depending on whether the time of day is determined to be a second time of day different from the first time of day. Color characteristics with a first abstract representation based on the time of day allow users to identify the time of day, thereby providing users with improved visual feedback, reducing the amount of input required to perform an action, providing additional control options without cluttering the user interface with additional displayed controls, and performing an action without further user input once a set of conditions has been met.

[0304] In some examples, a first abstract representation (and / or a second abstract representation) indicates the weather in the second physical space (e.g., wind, rain, snow, tornado, hurricane, sun, and / or clouds) (e.g., 1208). This abstract representation indicating the weather in the second physical space allows the user to identify the current state of the second physical space, thereby providing improved visual feedback, reducing the amount of input required to perform an action, providing additional control options without cluttering the user interface with additional displayed controls, and performing an action without further user input once a set of conditions has been met.

[0305] In some examples, when an area of ​​a first physical space is illuminated with a first abstract representation including a first scenario corresponding to a second physical space, the computer system detects that the scenario of the second physical space has changed from the first scenario to a third scenario different from the first scenario (and / or the second scenario). In some examples, in response to detecting that the scenario of the second physical space has changed to the third scenario, the computer system illuminates an area of ​​the first physical space via a light source with a third abstract representation including a third scenario corresponding to the second physical space (e.g., 1204C and / or 1208), wherein the third abstract representation is different from the first abstract representation (and / or the second abstract representation). After illuminating the first abstract representation, illuminating an area of ​​the first physical space with a third abstract representation including a third scenario corresponding to the second physical space allows the user to identify the change in scenario through illumination, thereby providing the user with improved visual feedback, reducing the amount of input required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, and performing an operation without further user input when a set of conditions has been met.

[0306] In some examples, the third abstract representation includes a representation of the sun's current position (and / or location) (e.g., in a first physical space and / or a second physical space). In some examples, the third abstract representation includes a represent...

Claims

1. A method, the method comprising: At the computer system communicating with the light source: A request to inspect the area of ​​the physical space where lighting is located; as well as In response to the request that the area of ​​the physical space being illuminated is detected: Based on the determination that the area of ​​the physical space has a first attribute, a first type of illumination is provided via the light source; as well as Based on the determination that the area of ​​the physical space has a second attribute different from the first attribute, the provision of the first type of lighting is abandoned.

2. The method according to claim 1, further comprising: In response to the request that the area of ​​the physical space being illuminated is detected: Based on the determination that the area of ​​the physical space has a third attribute, a second type of lighting is provided, wherein the second type of lighting is less lighting than the first type of lighting.

3. The method according to claim 1, further comprising: In response to the request that the area of ​​the physical space being illuminated is detected: Based on the determination that the area of ​​the physical space has a third attribute, illumination is not provided to the area of ​​the physical space.

4. The method according to any one of claims 1 to 3, further comprising: In response to the request that the area of ​​the physical space being illuminated is detected: Based on the determination that the area of ​​the physical space has a fourth attribute, a third type of lighting, different from the first type of lighting, is provided via the light source.

5. The method according to any one of claims 1 to 4, wherein determining that the region of the physical space has the first attribute includes determining that the region has a first reflectivity quantity, and wherein determining that the region of the physical space has the second attribute includes determining that the region has a second reflectivity quantity different from the first reflectivity quantity.

6. The method according to any one of claims 1 to 5, wherein determining that the region of the physical space has the first attribute includes determining that the region has a first transparency amount, and wherein determining that the region of the physical space has the second attribute includes determining that the region has a second transparency amount different from the first transparency amount.

7. The method according to any one of claims 1 to 6, wherein determining that the region of the physical space has the first attribute includes determining whether a first person is present in the region.

8. The method according to any one of claims 1 to 7, wherein determining that the region of the physical space has the second attribute includes determining whether the face of a second person is present in the region.

9. The method according to any one of claims 1 to 8, further comprising: In response to the request that the area of ​​the physical space being illuminated is detected: Without considering the properties of the area of ​​the physical space, a fourth type of illumination is provided via the light source for a second area of ​​the physical space, wherein the second area is different from the area.

10. The method according to any one of claims 1 to 9, wherein the light source is a single light-emitting device.

11. The method according to any one of claims 1 to 10, further comprising: After providing the first type of lighting and based on the determination that the area of ​​the physical space has changed from the first attribute to the fifth attribute, a fifth type of lighting different from the first type of lighting is provided via the light source.

12. The method of claim 11, wherein providing the fifth type of lighting based on determining that the area of ​​the physical space has changed from the first attribute to the fifth attribute includes changing the lighting from the first type to the fifth type via the light source.

13. The method according to any one of claims 11 to 12, further comprising: After the fifth type of lighting is provided, the lighting is changed from the fifth type of lighting to the first type of lighting via the light source.

14. The method according to any one of claims 1 to 13, wherein providing the first type of illumination via the light source comprises: Based on the determination that the region includes a surface having a first color, the output of a second color is induced via the light source; as well as Based on the determination that the region includes a surface having a third color different from the first color, a fourth color different from the third color is generated via the light source.

15. The method according to any one of claims 1 to 14, the method further comprising: After providing the first type of lighting and determining that the current time of day is the first time of day, the lighting is changed from the first type of lighting to a sixth type of lighting via the light source, wherein the first type of lighting includes a first color temperature, and wherein the sixth type of lighting includes a second color temperature different from the first color temperature.

16. The method according to any one of claims 1 to 15, further comprising: After providing the first type of lighting, the changes in the physical space are detected; as well as In response to detecting the change in the physical space: Based on the determination that the physical space has been changed in a first manner, the illumination has been changed from the first type of lighting to the seventh type of lighting via the light source, wherein the first type of lighting includes a third color temperature, and wherein the seventh type of lighting includes a fourth color temperature different from the third color temperature.

17. The method according to any one of claims 1 to 16, wherein: Determining that the region of the physical space has the first attribute includes making the determination based on first data detected by a sensor; Determining that the region of the physical space has the second attribute includes making the determination based on second data detected by the sensor; The second data is different from the first data; and The sensor communicates with the computer system.

18. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for performing the method according to any one of claims 1 to 17.

19. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 1 to 17.

20. A computer system for communicating with a light source, the computer system comprising: Components for performing the method according to any one of claims 1 to 17.

21. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 17.

22. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for: A request to inspect the area of ​​the physical space where lighting is located; and In response to the request that the area of ​​the physical space being illuminated is detected: Based on the determination that the area of ​​the physical space has a first attribute, a first type of illumination is provided via the light source; and Based on the determination that the area of ​​the physical space has a second attribute different from the first attribute, the provision of the first type of lighting is abandoned.

23. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: A request to inspect the area of ​​the physical space where lighting is located; and In response to the request that the area of ​​the physical space being illuminated is detected: Based on the determination that the area of ​​the physical space has a first attribute, a first type of illumination is provided via the light source; as well as Based on the determination that the area of ​​the physical space has a second attribute different from the first attribute, the provision of the first type of lighting is abandoned.

24. A computer system for communicating with a light source, the computer system comprising: Components used for: detecting requests to illuminate a region of a physical space; as well as In response to the request that the area of ​​the physical space being illuminated is detected: Components for the following operation: providing a first type of illumination via the light source based on determining that the area of ​​the physical space has a first attribute; as well as The component is used to: abandon providing the first type of lighting based on a second attribute determined to be different from the first attribute of the area of ​​the physical space.

25. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the following operations: A request to inspect the area of ​​the physical space where lighting is located; and In response to the request that the area of ​​the physical space being illuminated is detected: Based on the determination that the area of ​​the physical space has a first attribute, a first type of illumination is provided via the light source; and Based on the determination that the area of ​​the physical space has a second attribute different from the first attribute, the provision of the first type of lighting is abandoned.

26. A method, the method comprising: At the computer system communicating with the light source: When a user is detected in the physical space, changes in the user's activity in the physical space are detected; as well as In response to the detection of changes in user activity in the physical space, the illumination of the physical space is changed via the light source while users continue to be detected in the physical space.

27. The method of claim 26, wherein detecting the change in user activity during the physical activity includes detecting a change in the sleep state of the first user.

28. The method of any one of claims 26 to 27, wherein detecting the change in user activity in the physical activity comprises detecting that a first number of users detected in the physical space has changed from a first number to a second number different from the first number.

29. The method of claim 28, wherein the second quantity is greater than the first quantity.

30. The method of claim 28, wherein the second quantity is less than the first quantity.

31. The method according to any one of claims 26 to 30, wherein detecting the change in user activity in the physical activity includes detecting a second number of users performing activities detected in the physical space.

32. The method according to any one of claims 26 to 31, wherein changing the illumination of the physical space while the user continues to be detected in the physical space includes turning on or off at least a portion of the illumination of the physical space.

33. The method according to any one of claims 26 to 32, wherein changing the illumination of the physical space while the user continues to be detected in the physical space includes changing the illumination level within the physical space.

34. The method according to any one of claims 26 to 33, wherein detecting the change in user activity in the physical activity includes detecting a predetermined distance from the location of a second user within the physical space.

35. The method according to any one of claims 26 to 34, wherein changing the illumination of the physical space while the user continues to be detected in the physical space comprises changing the illuminated area of ​​the physical space from a first area of ​​the physical space to a second area of ​​the physical space that is different from the first area of ​​the physical space.

36. The method of claim 35, wherein the first region and the second region are illuminated by the same light source.

37. The method of claim 35, wherein the first region is illuminated by a first light source, and wherein the second region is illuminated by a second light source different from the first light source.

38. The method according to any one of claims 26 to 37, wherein the light source is a single light-emitting device.

39. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for performing the method according to any one of claims 26 to 38.

40. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 26 to 38.

41. A computer system for communicating with a light source, the computer system comprising: Components for performing the method according to any one of claims 26 to 38.

42. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the method according to any one of claims 26 to 38.

43. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for: When a user is detected in the physical space, changes in the user's activity in the physical space are detected; and In response to the detection of changes in user activity in the physical space, the illumination of the physical space is changed via the light source while users continue to be detected in the physical space.

44. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: When a user is detected in the physical space, changes in the user's activity in the physical space are detected; and In response to the detection of changes in user activity in the physical space, the illumination of the physical space is changed via the light source while users continue to be detected in the physical space.

45. A computer system for communicating with a light source, the computer system comprising: Components used for the following operations: when a user is detected in a physical space, detecting changes in user activity in the physical space; as well as Components for the following operation: in response to the detection of a change in user activity in the physical space, changing the illumination of the physical space via the light source while the user continues to be detected in the physical space.

46. ​​A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the following operations: When a user is detected in the physical space, changes in the user's activity in the physical space are detected; and In response to the detection of changes in user activity in the physical space, the illumination of the physical space is changed via the light source while users continue to be detected in the physical space.

47. A method comprising: At the computer system communicating with the light source: Detect lighting requests corresponding to the relevant area of ​​the physical space being illuminated; as well as In response to the detection of the lighting request: Based on the determination that the request corresponds to a first region of the physical space, the first region is illuminated via the light source; as well as Based on the determination that the request corresponds to a second region of the physical space that is different from the first region, the second region is illuminated via the light source.

48. The method of claim 47, wherein detecting the lighting request includes detecting a first pointing input in the direction of the corresponding area in the physical space.

49. The method of any one of claims 47 to 48, wherein detecting the lighting request includes a request to detect the location of the identified object.

50. The method of claim 49, wherein the position of the object is specified in the lighting request.

51. The method of claim 49, wherein the location of the object is determined via the computer system.

52. The method according to claim 49, wherein: In response to the detection of the lighting request: The first area is illuminated in a first manner based on determining that the request corresponds to the first area and determining that the object has a first possibility of being in the first area; as well as Based on determining that the request corresponds to the first region and based on determining that the object has a second possibility of being in the first region, the first region is illuminated in a second manner different from the first manner.

53. The method according to any one of claims 47 to 52, wherein illuminating the first region via the light source comprises: Within a first time frame, the illumination of a first portion of the first region is moved to a second portion of the first region at a first rate. as well as After the first time range and within the second time range, the illumination of the second portion of the first region is moved to the third portion of the first region at the first rate, wherein the second portion of the first region is adjacent to the first portion of the first region and the third portion of the first region.

54. The method of claim 53, wherein when the illumination of the first portion of the first region is moved to the second portion of the first region and the illumination of the second portion of the first region is moved to the third portion of the first region, the light source remains in a specific position.

55. The method of any one of claims 53 to 54, wherein detecting the lighting request includes detecting input, and wherein one or more portions of the first region are identified based on the input.

56. The method according to any one of claims 53 to 55, wherein: Based on the determination that the size of an object is a first size in the first part of the first region and the size of a second object is a second size in the second part of the first region, wherein the first size is smaller than the second size, the illumination size of the first part of the first region is smaller than the illumination size of the second part of the first region; as well as Based on the determination that the size of the object in the first part of the first region is the first size and the size of the second object in the second part of the first region is the third size, wherein the first size is greater than the third size, the size of the illumination of the first part of the first region is greater than the size of the illumination of the second part of the first region.

57. The method according to any one of claims 47 to 56, wherein: The lighting request corresponds to a request for content output by a device that is different from that of the computer system; Based on the determination that the first region includes the device, the corresponding region is the first region; and The second region is determined to include the device, and the corresponding region is the second region.

58. The method of claim 57, wherein the device, which is different from the computer system, is a smart speaker.

59. The method of claim 57, wherein the device, which is different from the computer system, is a television.

60. The method according to any one of claims 47 to 59, wherein illuminating the first region comprises: Based on the determination that the lighting request corresponds to a first object, a first type of lighting is provided via the light source; as well as Based on the determination that the lighting request corresponds to a second object different from the first object, a second type of lighting different from the first type of lighting is provided via the light source.

61. The method according to any one of claims 47 to 60, the method further comprising: When the first area is illuminated, the second pointing input is detected to no longer be facing the first area; as well as In response to detecting that the second pointing input is no longer facing the first area: Based on the determination that a corresponding input has been detected, the first area continues to be illuminated via the light source; and If it is determined that the corresponding input is not detected, the illumination of the first area via the light source is stopped.

62. The method according to any one of claims 47 to 61, wherein the lighting request does not include an identifier of the corresponding area.

63. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for performing the method according to any one of claims 47 to 62.

64. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 47 to 62.

65. A computer system for communicating with a light source, the computer system comprising: Components for performing the method according to any one of claims 47 to 62.

66. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the method according to any one of claims 47 to 62.

67. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for: Detect lighting requests corresponding to the relevant area of ​​the physical space being illuminated; as well as In response to the detection of the lighting request: Based on the determination that the request corresponds to a first region of the physical space, the first region is illuminated via the light source; as well as Based on the determination that the request corresponds to a second region of the physical space that is different from the first region, the second region is illuminated via the light source.

68. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: Detect lighting requests corresponding to the relevant area of ​​the physical space being illuminated; as well as In response to the detection of the lighting request: Based on the determination that the request corresponds to a first region of the physical space, the first region is illuminated via the light source; as well as Based on the determination that the request corresponds to a second region of the physical space that is different from the first region, the second region is illuminated via the light source.

69. A computer system for communicating with a light source, the computer system comprising: Components used for the following operations: detecting lighting requests corresponding to the appropriate area of ​​the physical space to be lit; as well as In response to the detection of the lighting request: A component for the following operation: illuminating the first region via the light source based on determining that the request corresponds to a first region of the physical space; as well as The component is used to illuminate the second region via the light source, based on the determination that the request corresponds to a second region of the physical space that is different from the first region.

70. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the following operations: Detect lighting requests corresponding to the relevant area of ​​the physical space being illuminated; as well as In response to the detection of the lighting request: Based on the determination that the request corresponds to a first region of the physical space, the first region is illuminated via the light source; as well as Based on the determination that the request corresponds to a second region of the physical space that is different from the first region, the second region is illuminated via the light source.

71. A method, the method comprising: At the computer system communicating with the light source: A request to detect the area of ​​the first physical space illuminated; as well as In response to the request that the area of ​​the first physical space is illuminated: Based on the determination that the second physical space has a first scenario, the area of ​​the first physical space is illuminated via the light source to include a first abstract representation of the first scenario corresponding to the second physical space, wherein the second physical space is outside the first physical space; as well as Based on the determination that the second physical space has a second scenario different from the first scenario, the area of ​​the first physical space is illuminated via the light source to include a second abstract representation corresponding to the second scenario of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scenario of the second physical space.

72. The method of claim 71, wherein illuminating the region to include the first abstract representation includes illuminating a portion of the region to form a simulated silhouette representing a first person.

73. The method of claim 72, wherein the simulated profile corresponds to a second person detected in the second physical space.

74. The method of claim 72, wherein the first person is not detected in the second physical space.

75. The method according to any one of claims 71 to 74, wherein illuminating the region to include the first abstract representation includes illuminating a portion of the region to form a simulated outline representing a tree.

76. The method of claim 75, wherein the lighting is based on weather changes over time in the second physical space.

77. The method according to any one of claims 71 to 76, wherein the first abstract representation indicates the time of day.

78. The method of claim 77, wherein the color characteristics of the first abstract representation are based on the time of day.

79. The method according to any one of claims 71 to 78, wherein the first abstract representation indicates the weather in the second physical space.

80. The method according to any one of claims 71 to 79, the method further comprising: When the area of ​​the first physical space is illuminated to include the first abstract representation of the first scenario corresponding to the second physical space, it is detected that the scenario of the second physical space has changed from the first scenario to a third scenario different from the first scenario. as well as In response to detecting that the scenario of the second physical space has changed to the third scenario, the area of ​​the first physical space is illuminated via the light source to include a third abstract representation corresponding to the third scenario of the second physical space, wherein the third abstract representation is different from the first abstract representation.

81. The method of claim 80, wherein the third abstract representation includes the current position of the sun.

82. The method according to any one of claims 80 to 81, wherein: The first abstract representation includes a first indication of the weather in the second physical space; The third abstract representation includes a second indication of the weather in the second physical space; The second instruction differs from the first instruction; and The second indication represents the change in weather in the second physical space.

83. The method according to any one of claims 71 to 82, wherein the first abstract representation changes over a period of time.

84. The method according to any one of claims 71 to 83, wherein: Based on the determination that the object in the second physical space is of the first size, the first abstract representation is of the second size; as well as Based on the determination that the object in the second physical space is a third size different from the first size, the first abstract representation is a fourth size different from the second size.

85. The method according to any one of claims 71 to 84, wherein: Based on the determination that the user has selected a first setting, the first abstract representation includes first details of the second physical space; and Based on the determination that the user has selected a second setting different from the first setting, the first abstract representation does not include the first details of the second physical space.

86. The method according to any one of claims 71 to 85, the method further comprising: After illuminating the area of ​​the physical space to include an abstract representation corresponding to the second physical space, a request to change the abstract representation to correspond to a third physical space different from the second physical space is detected. as well as In response to detecting the request to change the abstract representation to correspond to the third physical space, the region of the first physical space is illuminated via the light source to include a fourth abstract representation of a scenario corresponding to the third physical space, wherein the third physical space is outside the first physical space and the second physical space.

87. The method of claim 86, wherein the region of the first physical space is gradually illuminated over a period of time to include the fourth abstract representation of the scenario corresponding to the third physical space.

88. The method according to any one of claims 86 to 87, wherein: Based on the determination that the third physical space corresponds to the first physical space, the area of ​​the first physical space is illuminated at a first rate; as well as Based on the determination that the third physical space does not correspond to the first physical space, the area of ​​the first physical space is illuminated at a second rate, wherein the first rate is faster than the second rate.

89. The method according to any one of claims 71 to 88, wherein: The first abstract representation includes the representation of celestial objects; Based on the determination that the celestial object is in a first location relative to the second physical space, the representation of the celestial object is located at a first position within the first abstract representation; as well as Based on the determination that the celestial object is in a second location relative to the second physical space, the representation of the celestial object is located at a second location within the first abstract representation, wherein the second location is different from the first location, and wherein the first location is different from the second location.

90. The method according to claim 89, further comprising: The passage of time is detected when the representation of the celestial object is located at the first position within the first abstract representation; as well as In response to the detection of the passage of time and the absence of corresponding user input, the representation of the celestial object is moved from the first position within the first abstract representation to the third position within the first abstract representation.

91. The method of any one of claims 71 to 90, wherein the region of the first physical space is illuminated to include the first abstract representation including a third portion of the illuminated region to form a set of one or more outlines representing a first group of one or more objects, wherein the first group of one or more objects is positioned within the first physical space.

92. The method according to any one of claims 71 to 91, wherein the region of the first physical space is illuminated to include the first abstract representation including a fourth portion of the illuminated region to form a simulated outline representing a second group of one or more objects, wherein the second group of one or more objects is positioned within the second physical space.

93. The method according to any one of claims 71 to 92, wherein the computer system is a first computer system, and the method further comprises: Prior to detecting the request to illuminate the area of ​​the first physical space, a request to establish communication between the first computer system and the second computer system is received from a second computer system different from the first computer system, wherein the request to illuminate the area of ​​the first physical space is associated with the communication.

94. The method of claim 93, wherein the second physical space corresponds to the communication with the second computer system.

95. The method according to any one of claims 93 to 94, wherein the communication includes a first user and a second user different from the first user, and wherein the second physical space is selected by the first user or the second user.

96. The method according to any one of claims 71 to 95, wherein the computer system is a third computer system, wherein the third computer system communicates with a display generation component, the method further comprising: Before the request to illuminate the area of ​​the first physical space is detected, a second request to establish communication between the third computer system and the fourth computer system is received from a fourth computer system, which is different from the third computer system. as well as Upon receiving the second request, a user interface element is displayed via the display generation component, wherein the request to illuminate the area of ​​the first physical space corresponds to the selection of the user interface element.

97. The method according to claim 96, further comprising: Upon receiving the second request and before establishing the communication between the third computer system and the fourth computer system, a preview of the communication between the third computer system and the fourth computer system is displayed via the display generation component, wherein the preview is displayed when the user interface elements are displayed.

98. The method of claim 96, wherein the user interface elements are displayed after the communication between the third computer system and the fourth computer system is established.

99. The method according to any one of claims 71 to 98, the method further comprising: In response to the detection of the request to illuminate the area of ​​the first physical space and when the area of ​​the first physical space is illuminated via the light source to include the first abstract representation, the area of ​​the first physical space is illuminated via the light source to include a fifth abstract representation corresponding to a fourth physical space, wherein the fourth physical space is different from the second physical space.

100. The method according to any one of claims 71 to 99, the method further comprising: In conjunction with the request to detect the area of ​​the first physical space that is illuminated, an input corresponding to the corresponding time indication is detected; as well as In response to the request that the area of ​​the first physical space is illuminated: Based on determining that the corresponding time indication is a first time indication, the area of ​​the first physical space is illuminated via the light source to include a sixth abstract representation corresponding to the first time indication; as well as Based on the determination that the corresponding time indication is a second time indication different from the first time indication, the area of ​​the first physical space is illuminated via the light source to include a seventh abstract representation corresponding to the second time indication, wherein the seventh abstract representation is different from the sixth abstract representation.

101. The method according to any one of claims 71 to 100, the method further comprising: In conjunction with the request to detect the area of ​​the first physical space being illuminated, an input corresponding to the corresponding event is detected; as well as In response to detecting the input corresponding to the corresponding event: Based on determining that the corresponding event is the first event, the area of ​​the first physical space is illuminated via the light source to include an eighth abstract representation corresponding to the first event; as well as Based on the determination that the corresponding event is a second event different from the first event, the area of ​​the first physical space is illuminated via the light source to include a ninth abstract representation corresponding to the second event, wherein the eighth abstract representation is different from the ninth abstract representation.

102. The method of any one of claims 71 to 101, wherein the region of the first physical space is illuminated to include the first abstract representation including various visual states of advancing through the first abstract representation, the method further comprising: The selection of a setting corresponding to the region of the first physical space illuminated by the light source to include the first abstract representation; as well as In response to detecting the selection of the setting: Based on the determination that the selection of the setting corresponds to a first setting, the area of ​​the first physical space is illuminated to include the various visual states of the first abstract representation advancing through the first abstract representation at a first rate; as well as Based on the determination that the selection of the setting corresponds to a second setting different from the first setting, the area of ​​the first physical space is illuminated to include the various visual states of the first abstract representation advancing through the first abstract representation at a second rate different from the first rate.

103. The method according to any one of claims 71 to 102, the method further comprising: When the area of ​​the first physical space corresponding to the first scenario in the second physical space is illuminated by the light source, a detection notification is generated; as well as In response to the detection of the notification, the lighting in the area of ​​the first physical space is modified.

104. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for performing the method according to any one of claims 71 to 103.

105. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 71 to 103.

106. A computer system for communicating with a light source, the computer system comprising: Components for performing the method according to any one of claims 71 to 103.

107. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the method according to any one of claims 71 to 103.

108. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs including instructions for: A request to detect the area of ​​the first physical space illuminated; and In response to the request that the area of ​​the first physical space is illuminated: Based on the determination that the second physical space has a first scenario, the area of ​​the first physical space is illuminated via the light source to include a first abstract representation of the first scenario corresponding to the second physical space, wherein the second physical space is outside the first physical space; and Based on the determination that the second physical space has a second scenario different from the first scenario, the area of ​​the first physical space is illuminated via the light source to include a second abstract representation corresponding to the second scenario of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scenario of the second physical space.

109. A computer system for communicating with a light source, the computer system comprising: one or more processors; as well as The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: A request to detect the area of ​​the first physical space illuminated; and In response to the request that the area of ​​the first physical space is illuminated: Based on the determination that the second physical space has a first scenario, the area of ​​the first physical space is illuminated via the light source to include a first abstract representation of the first scenario corresponding to the second physical space, wherein the second physical space is outside the first physical space; as well as Based on the determination that the second physical space has a second scenario different from the first scenario, the area of ​​the first physical space is illuminated via the light source to include a second abstract representation corresponding to the second scenario of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scenario of the second physical space.

110. A computer system for communicating with a light source, the computer system comprising: Components used for: detecting requests to illuminate the area of ​​the first physical space; as well as In response to the request that the area of ​​the first physical space is illuminated: Components for the following operation: based on determining that a second physical space has a first scene, illuminating the area of ​​the first physical space via the light source to include a first abstract representation of the first scene corresponding to the second physical space, wherein the second physical space is outside the first physical space; as well as The component is used to: illuminate the region of the first physical space via the light source to include a second abstract representation corresponding to the second scenario of the second physical space, based on the determination that the second physical space has a second scenario different from the first scenario, the second abstract representation being different from the first abstract representation corresponding to the first scenario of the second physical space.

111. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a light source, the one or more programs comprising instructions for performing the following operations: A request to detect the area of ​​the first physical space illuminated; and In response to the request that the area of ​​the first physical space is illuminated: Based on the determination that the second physical space has a first scenario, the area of ​​the first physical space is illuminated via the light source to include a first abstract representation of the first scenario corresponding to the second physical space, wherein the second physical space is outside the first physical space; and Based on the determination that the second physical space has a second scenario different from the first scenario, the area of ​​the first physical space is illuminated via the light source to include a second abstract representation corresponding to the second scenario of the second physical space, the second abstract representation being different from the first abstract representation corresponding to the first scenario of the second physical space.

112. A method, the method comprising: At the computer system communicating with the first device and the light source separate from the first device: Receive a request to expand the content to be displayed on the first device to include a physical space comprising a first region and a second region different from the first region; and In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on determining that the first device is located at a first position in the physical space, the first area of ​​the physical space having a corresponding spatial arrangement relative to the first position in the physical space is illuminated by the light source with a first light pattern based on the content displayed on the first device, without illuminating the second area of ​​the physical space with the first light pattern via the light source; as well as Based on the determination that the first device is located at a second position in the physical space, the second area of ​​the physical space having the corresponding spatial arrangement relative to the second position in the physical space is illuminated by the light source with a first light pattern based on the content displayed on the first device.

113. The method according to claim 112, further comprising: In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on the determination that the first device is located at the second position in the physical space, the first area of ​​the physical space is abandoned from being illuminated by the first light pattern via the light source.

114. The method according to any one of claims 112 to 113, wherein the first device is a television.

115. The method according to any one of claims 112 to 114, the method further comprising: In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on the determination that the first device is located at the first position in the physical space, a third area of ​​the physical space is illuminated via the light source, wherein the third area is in front of the first device.

116. The method according to any one of claims 112 to 115, the method further comprising: In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on the determination that the first device is located at the first position in the physical space, a fourth area of ​​the physical space is illuminated via the light source, wherein the fourth area is behind the first device.

117. The method according to any one of claims 112 to 116, wherein the first light pattern includes content different from the content displayed on the first device, and wherein the different content is synchronized with the content displayed on the first device.

118. The method according to any one of claims 112 to 117, wherein the first light pattern comprises a representation of the content displayed on the first device.

119. The method according to any one of claims 112 to 118, wherein the first light pattern comprises a simulation of light emitted from the content displayed on the first device.

120. The method of any one of claims 112 to 119, wherein the content displayed on the first device is part of a multidimensional representation of the environment, and wherein the first light pattern is based on content of the multidimensional representation of the environment that is currently not visible on the first device.

121. The method according to claim 120, further comprising: In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on determining that the first device is located at the first position in the physical space and based on determining that the user is located at the third position in the physical space, the area relative to the third position is illuminated via the light source with a lower fidelity than the first area.

122. A non-transitory computer-readable medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a first device and a light source separate from the first device, the one or more programs including instructions for performing the method according to any one of claims 112 to 121.

123. A computer system communicating with a first device and a light source separate from the first device, the computer system comprising: one or more processors; as well as A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 112 to 121.

124. A computer system communicating with a first device and a light source separate from the first device, the computer system comprising: Components for performing the method according to any one of claims 112 to 121.

125. A computer program product comprising one or more programs executed by one or more processors of a computer system configured to communicate with a first device and a light source separate from the first device, the one or more programs comprising instructions for performing the method according to any one of claims 112 to 121.

126. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a first device and a light source separate from the first device, the one or more programs including instructions for: Receive a request to expand the content to be displayed on the first device to include a physical space comprising a first region and a second region different from the first region; and In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on determining that the first device is located at a first position in the physical space, the first area of ​​the physical space having a corresponding spatial arrangement relative to the first position in the physical space is illuminated via the light source using a first light pattern based on the content displayed on the first device, without illuminating the second area of ​​the physical space using the first light pattern via the light source; and Based on the determination that the first device is located at a second position in the physical space, the second area of ​​the physical space having the corresponding spatial arrangement relative to the second position in the physical space is illuminated by the light source with a first light pattern based on the content displayed on the first device.

127. A computer system communicating with a first device and a light source separate from the first device, the computer system comprising: one or more processors; as well as The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: Receive a request to expand the content to be displayed on the first device to include a physical space comprising a first region and a second region different from the first region; and In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on determining that the first device is located at a first position in the physical space, the first area of ​​the physical space having a corresponding spatial arrangement relative to the first position in the physical space is illuminated by the light source with a first light pattern based on the content displayed on the first device, without illuminating the second area of ​​the physical space with the first light pattern via the light source; as well as Based on the determination that the first device is located at a second position in the physical space, the second area of ​​the physical space having the corresponding spatial arrangement relative to the second position in the physical space is illuminated by the light source with a first light pattern based on the content displayed on the first device.

128. A computer system communicating with a first device and a light source separate from the first device, the computer system comprising: A component for the following operation: receiving a request to expand the content to be displayed on the first device to a physical space including a first area and a second area different from the first area; as well as In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: The component is used to: illuminate a first area of ​​the physical space having a corresponding spatial arrangement relative to the first location in the physical space via the light source using a first light pattern based on the content displayed on the first device, without illuminating a second area of ​​the physical space via the light source using the first light pattern, based on the first device being located at a first position in the physical space. as well as The component is used to: based on determining that the first device is located at a second position in the physical space, illuminate, via the light source, a second area of ​​the physical space having the corresponding spatial arrangement relative to the second position in the physical space with a first light pattern based on the content displayed on the first device.

129. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a first device and a light source separate from the first device, the one or more programs comprising instructions for performing the following operations: Receive a request to expand the content to be displayed on the first device to include a physical space comprising a first region and a second region different from the first region; and In response to receiving the request to expand the content displayed on the first device and when the content is displayed on the first device: Based on determining that the first device is located at a first position in the physical space, the first area of ​​the physical space having a corresponding spatial arrangement relative to the first position in the physical space is illuminated via the light source using a first light pattern based on the content displayed on the first device, without illuminating the second area of ​​the physical space using the first light pattern via the light source; and Based on the determination that the first device is located at a second position in the physical space, the second area of ​​the physical space having the corresponding spatial arrangement relative to the second position in the physical space is illuminated by the light source with a first light pattern based on the content displayed on the first device.

Images