User interface for managing controllable external devices

Abstract

This invention provides technology for managing controllable external devices in relation to computer user interfaces. [Solution] An electronic device comprising a display device, a communication device, one or more processors, and memory displays a first representation of a first controllable external device when the first controllable external device is located at a specific location. The device detects a first input corresponding to a selection of the first display of the first controllable external device. In response to detecting the first input, the device uses the communication device to send a command to the first controllable external device to perform a function. The first controllable external device performs a function if it is a first type of device, or uses the display device to display a user interface for controlling the characteristics of the first controllable external device.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims priority to U.S. Provisional Patent Application No. 62 / 855,895, titled "User Interface for Managing a Controllable External Device," filed on May 31, 2019; U.S. Patent Application No. 16 / 584,743, titled "User Interface for Managing a Controllable External Device," filed on September 26, 2019; and U.S. Patent Application No. 16 / 586,002, titled "User Interface for Managing a Controllable External Device," filed on September 27, 2019, each of which is hereby incorporated by reference in its entirety.

[0002] The present disclosure generally relates to computer user interfaces, and more specifically, to techniques for managing controllable external devices.

Background Art

[0003] Controllable (e.g., remotely controllable) electronic devices, including Wi - Fi - enabled lighting, power outlets, thermostats, and household appliances, provide users with a combination of functionality and convenience. Controllable electronic devices can be managed using a computer interface.

Summary of the Invention

[0004] However, some technologies for managing controllable external devices using electronic devices are generally cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may involve multiple key presses or taps, and / or touch inputs. As another example, some existing technologies fail to provide a wide range of functionality or sufficient feedback regarding the status and function of various controllable devices. As a result, existing technologies take more time than necessary to manage controllable external devices, wasting both user time and device energy. The latter problem is particularly critical in battery-powered devices.

[0005] Therefore, this technology provides electronic devices with faster and more efficient methods and interfaces for managing controllable external devices. Such methods and interfaces optionally complement or replace other methods for managing controllable external devices. Such methods and interfaces reduce the cognitive burden on the user and create a more efficient human-machine interface. In the case of battery-operated computing devices, such methods and interfaces conserve power and extend the interval between battery charges. In some embodiments, the user experience is simplified by having a user interface for managing controllable external devices instead of various different applications for managing each controllable external device, thus reducing the reception of unnecessary / heterogeneous / repetitive inputs that were required to manage controllable external devices.

[0006] A method is described according to some embodiments. The method includes an electronic device comprising a display device and a communication device, which includes displaying a user interface on the display device that includes a first accessory affordance associated with a first controllable external device; detecting a first input corresponding to the selection of the first accessory affordance; in response to the detection of the first input, transmitting a command to the first controllable external device via the communication device to perform a first function, in accordance with the determination that a set of command transmission criteria is satisfied; and displaying a first accessory interface on the display device, in accordance with the determination that the set of command transmission criteria is not satisfied, wherein the first accessory interface includes a first control affordance which, when selected, transmits a first command to the controllable external device to perform a first function; and a second control affordance which, when selected, transmits a command to the first controllable external device to perform a second function different from the first function.

[0007] According to some embodiments, a non-temporary computer-readable storage medium is described. The non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and a communication device, the one or more programs including instructions to cause a user interface to be displayed on the display device, which includes a first accessory affordance associated with a first controllable external device; to detect a first input corresponding to the selection of the first accessory affordance; to send a command to the first controllable external device via the communication device in response to the detection of the first input, in accordance with a determination that a set of command transmission criteria has been met; and to display a first accessory interface on the display device in accordance with a determination that the set of command transmission criteria has not been met, the first accessory interface including a first control affordance which, when selected, sends a first command to the controllable external device to perform a first function; and a second control affordance which, when selected, sends a command to the first controllable external device to perform a second function different from the first function.

[0008] According to some embodiments, a temporary computer-readable storage medium is described. The temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and a communication device, the one or more programs including instructions to cause a user interface to be displayed on the display device, which includes a first accessory affordance associated with a first controllable external device; to detect a first input corresponding to the selection of the first accessory affordance; to send a command to the first controllable external device via the communication device to perform a first function in response to the detection of the first input, in accordance with a determination that a set of command transmission criteria has been met; and to display a first accessory interface on the display device in accordance with a determination that the set of command transmission criteria has not been met, the first accessory interface including a first control affordance which, when selected, sends a first command to the controllable external device to perform a first function; and a second control affordance which, when selected, sends a command to the first controllable external device to perform a second function different from the first function.

[0009] According to some embodiments, an electronic device is described. The electronic device includes a display device and a communication device, one or more processors, and a memory for storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for displaying a video media user interface, which include a user interface on the display device that includes a first accessory affordance associated with a first controllable external device, detecting a first input corresponding to the selection of the first accessory affordance, transmitting a command to the first controllable external device via the communication device to perform a first function in response to the detection of the first input, in accordance with the determination that a set of command transmission criteria has been met, and displaying a first accessory interface on the display device in accordance with the determination that the set of command transmission criteria has not been met, the first accessory interface including a first control affordance which, when selected, transmits a first command to the controllable external device to perform a first function, and a second control affordance which, when selected, transmits a command to the first controllable external device to perform a second function different from the first function.

[0010] According to some embodiments, an electronic device is described. The electronic device includes a display device and a communication device; means for displaying a user interface on the display device, which includes a first accessory affordance associated with a first controllable external device; means for detecting a first input corresponding to the selection of the first accessory affordance; and means that, in response to detecting the first input, transmit a command to the first controllable external device via the communication device to perform a first function, according to a determination that a set of command transmission criteria is satisfied, and display a first accessory interface on the display device according to a determination that the set of command transmission criteria is not satisfied, wherein the first accessory interface includes a first control affordance, which, when selected, transmits a command to the first controllable external device to perform a first function; and a second control affordance, which, when selected, transmits a command to the first controllable external device to perform a second function different from the first function.

[0011] A method is described according to some embodiments. The method relates to an electronic device having a display device, which includes displaying a user interface on the display device including a first accessory affordance associated with a first controllable external device; detecting a first input corresponding to the selection of the first accessory affordance; and, in response to the detection of the first input, displaying on the display device a first area of ​​the first accessory interface which includes a first control affordance that, when selected, controls a first function of the first controllable external device without displaying a second area of ​​the first accessory interface; and while the first area of ​​the first accessory interface is displayed on the display device, The method includes detecting a second input, and in response to the detection of the second input, updating the user interface display to show the second region, which includes one or more settings for a first controllable external device, at the top of the first accessory interface, while the display of the first region is stopped, in accordance with the determination that the second input causes a portion of the second region to exceed a threshold amount, and updating the display of the first accessory interface based on a second input to a second control affordance in the first region of the first accessory interface, without displaying the second region of the first accessory interface.

[0012] According to some embodiments, a non-temporary computer-readable storage medium is described. The non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs being instructions to cause the display device to display a user interface including a first accessory affordance associated with a first controllable external device, to detect a first input corresponding to the selection of the first accessory affordance, and in response to the detection of the first input, to display on the display device a first area of ​​the first accessory interface including a first control affordance which, when selected, controls a first function of the first controllable external device without displaying a second area of ​​the first accessory interface, and the first access The process includes: detecting a second input while a first region of the Sali interface is being displayed on a display device; updating the user interface display to show a second region, which includes one or more settings for a first controllable external device, at the top of the first accessory interface, in response to the detection of the second input, according to a determination that the second input causes a portion of the second region to be displayed beyond a threshold amount; and updating the display of the first accessory interface based on a second input to a second control affordance in the first region of the first accessory interface, without displaying the second region of the first accessory interface, according to a determination that the second input does not cause a portion of the second region to be displayed beyond a threshold amount.

[0013] According to some embodiments, a temporary computer-readable storage medium is described. The temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs causing the display device to display a user interface including a first accessory affordance associated with a first controllable external device, detecting a first input corresponding to the selection of a first accessory affordance, and in response to the detection of the first input, displaying on the display device a first area of ​​the first accessory interface which includes a first control affordance which, when selected, controls a first function of the first controllable external device without displaying a second area of ​​the first accessory interface, and the first accessory input The method includes detecting a second input while a first area of ​​a surface is being displayed on a display device, updating the user interface display while the display of the first area is stopped in response to the detection of the second input, according to a determination that the second input causes a portion of the second area to be displayed beyond a threshold amount, and displaying a second area, which includes one or more settings for a first controllable external device, above the first accessory interface, and updating the display of the first accessory interface based on a second input to a second control affordance in the first area of ​​the first accessory interface without displaying the second area of ​​the first accessory interface, according to a determination that the second input does not cause a portion of the second area to be displayed beyond a threshold amount.

[0014] According to some embodiments, an electronic device is described. This electronic device includes a display device, one or more processors, and a memory for storing one or more programs configured to be executed by the one or more processors, the one or more programs being instructions for displaying a video media user interface, which includes a user interface on the display device including a first accessory affordance associated with a first controllable external device, detecting a first input corresponding to the selection of the first accessory affordance, and, in response to detecting the first input, on the display device a first area of ​​the first accessory interface which includes a first control affordance that, when selected, controls a first function of the first controllable external device without displaying a second area of ​​the first accessory interface. The function includes displaying, detecting a second input while displaying a first area of ​​a first accessory interface on a display device, updating the user interface display to display a second area above the first accessory interface, which includes one or more settings for a first controllable external device, while the display of the first area is stopped, in response to the detection of the second input and according to a determination that the second input causes a portion of the second area to be displayed beyond a threshold amount, and updating the display of the first accessory interface based on a second input to a second control affordance in the first area of ​​the first accessory interface without displaying the second area of ​​the first accessory interface, in accordance with a determination that the second input does not cause a portion of the second area to be displayed beyond a threshold amount.

[0015] According to some embodiments, an electronic device is described. This electronic device includes a display device, means for displaying a user interface on the display device including a first accessory affordance associated with a first controllable external device, means for detecting a first input corresponding to the selection of the first accessory affordance, means for displaying a first area of ​​the first accessory interface on the display device in response to the detection of the first input, which includes a first control affordance that, when selected, controls a first function of the first controllable external device without displaying a second area of ​​the first accessory interface, and means for detecting a second input on the display device while the first area of ​​the first accessory interface is being displayed. The system includes means for outputting, and means for updating the display of the user interface while the display of the first region is stopped, in response to the detection of a second input and according to the determination that the second input causes a portion of the second region to be displayed beyond a threshold amount, to display the second region, which includes one or more settings for the first controllable external device, above the first accessory interface, and means for updating the display of the first accessory interface based on a second input to a second control affordance in the first region of the first accessory interface, without displaying the second region of the first accessory interface, in accordance with the determination that the second input does not cause a portion of the second region to be displayed beyond a threshold amount.

[0016] According to some embodiments, a method is described. The method includes, in an electronic device having a display device, detecting a first set of one or more inputs corresponding to a request to configure the electronic device to control a first controllable external device while the electronic device is not configured to control a first controllable external device, and in response to the detection of the first set of one or more inputs, initiating a process to configure the electronic device to control the first controllable external device, the process including receiving information corresponding to the first controllable external device, and, based on the received information, determining that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device, displaying on the display device a first accessory affordance corresponding to a group including the first controllable external device and a plurality of sub-components, and, based on the received information, determining that the first controllable external device is not associated with a plurality of sub-components, The present invention includes: displaying a second accessory affordance corresponding to a first controllable external device; detecting a second input on a display device while displaying the first or second accessory affordance; and, in response to the detection of the second input, displaying on the display device a first accessory interface including a plurality of control affordances corresponding to a first controllable external device and a plurality of subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process of removing at least a first control affordance from the plurality of control affordances from the first accessory interface; and, in accordance with the determination that the second input corresponds to the selection of a second accessory affordance, displaying on the display device a second accessory interface including control of the first controllable external device without displaying the control reconfiguration affordance.

[0017] According to some embodiments, a non-temporary computer-readable storage medium is described.A non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs causing the electronic device to detect a first set of one or more inputs corresponding to a request to configure the electronic device to control a first controllable external device while the electronic device is not configured to control a first controllable external device, and in response to the detection of the first set of one or more inputs, initiating a process to configure the electronic device to control the first controllable external device, the process including receiving information corresponding to the first controllable external device, and, based on the received information, determining that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device, displaying on the display device a first accessory affordance corresponding to a group including the first controllable external device and a plurality of sub-components, and based on the received information, determining that the first controllable external device is associated with a plurality of sub-components of the first controllable external device The present invention includes: displaying a second accessory affordance on a display device corresponding to a first controllable external device in accordance with a determination that it is not associated with a component; detecting a second input on the display device while displaying the first or second accessory affordance; and, in response to the detection of the second input, displaying on the display device a first accessory interface including a plurality of control affordances corresponding to a first controllable external device and a plurality of subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process of removing at least a first control affordance from the plurality of control affordances from the first accessory interface; and displaying a second accessory interface including control of the first controllable external device on the display device without displaying the control reconfiguration affordance, in accordance with a determination that the second input corresponds to a selection of the second accessory affordance.

[0018] According to some embodiments, a temporary computer-readable storage medium is described.A temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs causing the electronic device to detect a first set of one or more inputs corresponding to a request to configure the electronic device to control a first controllable external device while the electronic device is not configured to control a first controllable external device, and in response to the detection of the first set of one or more inputs, initiating a process to configure the electronic device to control the first controllable external device, the process including receiving information corresponding to the first controllable external device, and, based on the received information, determining that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device, displaying on the display device a first accessory affordance corresponding to a group including the first controllable external device and a plurality of sub-components, and based on the received information, determining that the first controllable external device is associated with a plurality of sub-components of the first controllable external device The present invention includes: displaying a second accessory affordance on a display device corresponding to a first controllable external device in accordance with a determination that it is not associated with a component; detecting a second input on the display device while displaying the first or second accessory affordance; and, in response to the detection of the second input, displaying on the display device a first accessory interface including a plurality of control affordances corresponding to a first controllable external device and a plurality of subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process of removing at least a first control affordance from the plurality of control affordances from the first accessory interface; and displaying a second accessory interface including control of the first controllable external device on the display device without displaying the control reconfiguration affordance, in accordance with a determination that the second input corresponds to a selection of the second accessory affordance.

[0019] According to some embodiments, an electronic device is described. This electronic device comprises a display device, one or more processors, and a memory for storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs are instructions for displaying a video media user interface, which cause the electronic device to detect a first set of inputs corresponding to a request to configure the electronic device to control a first controllable external device while the electronic device is not configured to control a first controllable external device, and in response to the detection of the first set of inputs, initiate a process to configure the electronic device to control the first controllable external device, the process comprising receiving information corresponding to the first controllable external device, and, based on the received information, determining that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device, displaying a first accessory affordance corresponding to a group including the first controllable external device and the plurality of sub-components. To display, and based on the received information, in accordance with the determination that the first controllable external device is not associated with multiple subordinate components, to display on the display device a second accessory affordance corresponding to the first controllable external device; to detect a second input on the display device while displaying the first or second accessory affordance; and in response to the detection of the second input, in accordance with the determination that the second input corresponds to the selection of the first accessory affordance, to display on the display device a first accessory interface including multiple control affordances corresponding to the first controllable external device and multiple subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process of removing at least the first control affordance from the multiple control affordances from the first accessory interface; and in accordance with the determination that the second input corresponds to the selection of the second accessory affordance, to display on the display device without displaying the control reconfiguration affordance.This includes displaying a second accessory interface that includes control of a first controllable external device.

[0020] According to some embodiments, an electronic device is described. This electronic device includes a display device, means for detecting a first set of one or more inputs corresponding to a request to configure the electronic device to control a first controllable external device while the electronic device is not configured to control a first controllable external device, and means for initiating a process to configure the electronic device to control a first controllable external device in response to detecting a first set of one or more inputs, the process comprising: receiving information corresponding to a first controllable external device; displaying on the display device a first accessory affordance corresponding to a group including a first controllable external device and a plurality of sub-components, in accordance with a determination based on the received information that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device; and displaying on the display device a first accessory affordance corresponding to a group including a first controllable external device and a plurality of sub-components, in accordance with a determination based on the received information that the first controllable external device is not associated with a plurality of sub-components. The display device includes a means for displaying a second accessory affordance corresponding to a controllable external device, and for detecting a second input while displaying a first accessory affordance or a second accessory affordance, and means for displaying a first accessory interface including a plurality of control affordances corresponding to a first controllable external device and a plurality of subordinate components, in response to the detection of a second input, according to a determination that the second input corresponds to the selection of a first accessory affordance, on the display device, and a control reconfiguration affordance, which, when selected, initiates a process of removing at least a first control affordance from the plurality of control affordances from the first accessory interface, and means for displaying a second accessory interface including control of the first controllable external device without displaying the control reconfiguration affordance, according to a determination that the second input corresponds to the selection of a second accessory affordance.

[0021] According to some embodiments, a method is described. The method includes, in an electronic device having a display device, displaying on the display device a temperature mode creation user interface including a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device; detecting a first set of one or more inputs while the temperature mode creation user interface is displayed on the display device; selecting a temperature range including a minimum temperature and a maximum temperature different from a minimum temperature range in response to the first set of one or more inputs; associating the temperature range with a first temperature mode; displaying on the display device a temperature scheduling user interface including a second set of scheduling operation creation controls for an external thermostat device after associating the temperature range with the first temperature mode; detecting a second set of one or more inputs while the temperature scheduling user interface is displayed on the display device; selecting a first criterion for activating a first scheduled operation in response to the detection of the second set of one or more inputs; and associating the first temperature mode with a first scheduled operation.

[0022] According to some embodiments, a non-temporary computer-readable storage medium is described. A non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions to cause the display device to display a temperature mode creation user interface including a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device; detecting a first set of one or more inputs while the temperature mode creation user interface is displayed on the display device; selecting a temperature range including a minimum and maximum temperature different from a minimum temperature range in response to the first set of one or more inputs; associating the temperature range with a first temperature mode; displaying a temperature scheduling user interface including a second set of scheduled operation creation controls for an external thermostat device after associating the temperature range with the first temperature mode; detecting a second set of one or more inputs while the temperature scheduling user interface is displayed on the display device; selecting a first criterion for activating a first scheduled operation in response to the detection of the second set of one or more inputs; and associating the first temperature mode with a first scheduled operation.

[0023] According to some embodiments, a temporary computer-readable storage medium is described. A temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs being instructions that include displaying a temperature mode creation user interface on the display device, which includes a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device; detecting a first set of one or more inputs while the temperature mode creation user interface is displayed on the display device; selecting a temperature range including a minimum and maximum temperature different from a minimum temperature range in response to the first set of one or more inputs; associating the temperature range with a first temperature mode; displaying a temperature scheduling user interface on the display device, which includes a second set of scheduled operation creation controls for an external thermostat device after associating the temperature range with the first temperature mode; detecting a second set of one or more inputs while the temperature scheduling user interface is displayed on the display device; selecting a first criterion for activating a first scheduled operation in response to the detection of the second set of one or more inputs; and associating the first temperature mode with a first scheduled operation.

[0024] According to some embodiments, an electronic device is described. The electronic device comprises a display device, one or more processors, and a memory for storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs are instructions for displaying a video media user interface, which include causing the display device to display a temperature mode creation user interface including a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device; detecting a first set of one or more inputs while the temperature mode creation user interface is displayed on the display device; selecting a temperature range including a minimum and maximum temperature different from a minimum temperature range in response to the first set of one or more inputs; associating the temperature range with a first temperature mode; displaying a temperature scheduling user interface including a second set of scheduled operation creation controls for an external thermostat device after associating the temperature range with the first temperature mode; detecting a second set of one or more inputs while the temperature scheduling user interface is displayed on the display device; selecting a first criterion for activating a first scheduled operation in response to the detection of the second set of one or more inputs; and associating the first temperature mode with a first scheduled operation.

[0025] According to some embodiments, an electronic device is described. This electronic device includes a display device; means for displaying a temperature mode creation user interface on the display device, which includes a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device; means for detecting a first set of one or more inputs while the temperature mode creation user interface is displayed on the display device; means for selecting a temperature range including a minimum and maximum temperature different from a minimum temperature range, and associating the temperature range with a first temperature mode, in response to the first set of one or more inputs; means for displaying a temperature scheduling user interface on the display device, which includes a second set of scheduled operation creation controls for an external thermostat device, after associating the temperature range with the first temperature mode; means for detecting a second set of one or more inputs while the temperature scheduling user interface is displayed on the display device; and means for selecting a first criterion for activating a first scheduled operation, and associating the first temperature mode with the first scheduled operation, in response to the detection of the second set of one or more inputs.

[0026] According to some embodiments, a method is described. The method includes, in an electronic device including a display device, displaying, via the display device, a first accessory affordance corresponding to a first controllable external device; detecting one or more inputs for adding the first controllable external device to a first set of controllable external devices; in response to detecting the one or more inputs for adding the first controllable external device to the first set of controllable external devices, updating the first set of controllable external devices to include the first controllable external device; simultaneously displaying, in accordance with a determination that the first controllable external device is of a first type, a second accessory affordance corresponding to the first controllable external device and a configuration affordance that, when selected, initiates a process of configuring the audio settings of the first controllable external device; and displaying, in accordance with a determination that the first controllable external device is not of the first type, the second accessory affordance corresponding to the first controllable external device without the configuration affordance that, when selected, initiates a process of configuring the audio settings of the first controllable external device.

[0027] According to some embodiments, a non-temporary computer-readable storage medium is described. The non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs being instructions to cause a first accessory affordance corresponding to a first controllable external device via the display device to detect one or more inputs for adding the first controllable external device to a first set of controllable external devices, and to update the first set of controllable external devices to include the first controllable external device in response to the detection of one or more inputs for adding the first controllable external device to a first set of controllable external devices. The method includes, in accordance with the determination that the first controllable external device is of a first type, simultaneously displaying a second accessory affordance corresponding to the first controllable external device and a configuration affordance, which, when selected, initiates the process of configuring the audio settings of the first controllable external device; and, in accordance with the determination that the first controllable external device is not of a first type, displaying the second accessory affordance corresponding to the first controllable external device without the configuration affordance, which, when selected, initiates the process of configuring the audio settings of the first controllable external device.

[0028] According to some embodiments, a non - persistent computer - readable storage medium is described. The non - persistent computer - readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device. The one or more programs include instructions to display, via the display device, a first accessory affordance corresponding to a first controllable external device, and to detect one or more inputs for adding the first controllable external device to a first set of controllable external devices; in response to detecting the one or more inputs for adding the first controllable external device to the first set of controllable external devices, to update the first set of controllable external devices to include the first controllable external device; according to a determination that the first controllable external device is of a first type, to simultaneously display a second accessory affordance corresponding to the first controllable external device and a configuration affordance that, when selected, starts a process of configuring the audio settings of the first controllable external device; and according to a determination that the first controllable external device is not of the first type, to display the second accessory affordance corresponding to the first controllable external device without the configuration affordance that, when selected, starts a process of configuring the audio settings of the first controllable external device.

[0029] According to some embodiments, an electronic device is described. This electronic device comprises a display device, one or more processors, and a memory for storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs are instructions that cause the display device to display a first accessory affordance corresponding to a first controllable external device, detect one or more inputs for adding the first controllable external device to a first set of controllable external devices, and update the first set of controllable external devices to include the first controllable external device in response to the detection of one or more inputs for adding the first controllable external device to a first set of controllable external devices. The method includes, in accordance with the determination that the first controllable external device is of a first type, simultaneously displaying a second accessory affordance corresponding to the first controllable external device and a configuration affordance, which, when selected, initiates the process of configuring the audio settings of the first controllable external device; and, in accordance with the determination that the first controllable external device is not of a first type, displaying the second accessory affordance corresponding to the first controllable external device without the configuration affordance, which, when selected, initiates the process of configuring the audio settings of the first controllable external device.

[0030] According to some embodiments, an electronic device is described. This electronic device includes a display device; means for displaying a first accessory affordance corresponding to a first controllable external device via the display device; means for detecting one or more inputs for adding the first controllable external device to a first set of controllable external devices; means for updating the first set of controllable external devices to include the first controllable external device in response to detecting one or more inputs for adding the first controllable external device to a first set of controllable external devices, and simultaneously displaying a second accessory affordance corresponding to the first controllable external device and a configuration affordance, which, when selected, initiates a process of configuring the audio settings of the first controllable external device, in accordance with a determination that the first controllable external device is of a first type; and displaying the second accessory affordance corresponding to the first controllable external device without the configuration affordance, which, when selected, initiates a process of configuring the audio settings of the first controllable external device, in accordance with a determination that the first controllable external device is not of a first type.

[0031] According to some embodiments, a method is described. The method includes, in an electronic device equipped with a display device, simultaneously displaying one or more sets of playback controls and alarm configuration affordances for performing a playback function relating to a media item via the display device; detecting user input; in response to the detection of user input, initiating a process to perform a playback function on a controllable external device based on the selected playback control, according to a determination that the detected user input corresponds to the selection of a playback control from one or more sets of playback controls; and replacing the simultaneous display of one or more sets of playback controls and alarm configuration affordances with a set of time affordances for setting the time of a first alarm, according to a determination that the detected user input corresponds to the selection of an alarm configuration affordance.

[0032] According to some embodiments, a non-temporary computer-readable storage medium is described. The non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions that simultaneously display one or more sets of playback controls and alarm configuration affordances for performing a playback function relating to a media item via the display device and detect user input; in response to the detection of user input, initiate a process to perform a playback function on a controllable external device based on the selected playback control, according to a determination that the detected user input corresponds to a selection of a playback control from one or more sets of playback controls; and, according to a determination that the detected user input corresponds to a selection of alarm configuration affordances, replace the simultaneous display of one or more sets of playback controls and alarm configuration affordances with a set of time affordances for setting the time of a first alarm.

[0033] According to some embodiments, a temporary computer-readable storage medium is described. The temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions that simultaneously display one or more sets of playback controls and alarm configuration affordances for performing a playback function relating to a media item via the display device and detect user input; in response to the detection of user input, initiate a process to perform a playback function on a controllable external device based on the selected playback control, according to a determination that the detected user input corresponds to a selection of a playback control from one or more sets of playback controls; and replace the simultaneous display of one or more sets of playback controls and alarm configuration affordances with a set of time affordances for setting the time of a first alarm, according to a determination that the detected user input corresponds to a selection of alarm configuration affordances.

[0034] According to some embodiments, an electronic device is described. This electronic device includes a display device, one or more processors, and a memory for storing one or more programs configured to be executed by the one or more processors, the one or more programs being instructions that simultaneously display, via the display device, one or more sets of playback controls and alarm configuration affordances for performing a playback function relating to a media item, and detect user input; in response to the detection of user input, initiating a process to perform a playback function on a controllable external device based on the selected playback control, according to a determination that the detected user input corresponds to a selection of a playback control from the one or more sets of playback controls; and replacing the simultaneous display of one or more sets of playback controls and alarm configuration affordances with a set of time affordances for setting the time of a first alarm, according to a determination that the detected user input corresponds to a selection of alarm configuration affordances.

[0035] According to some embodiments, an electronic device is described. This electronic device includes a display device, means for simultaneously displaying one or more sets of playback controls and alarm configuration affordances for performing a playback function relating to a media item via the display device, means for detecting user input, and means for, in response to detecting user input, initiating a process to perform a playback function on a controllable external device based on the selected playback control, according to a determination that the detected user input corresponds to the selection of a playback control from one or more sets of playback controls, and replacing the simultaneous display of one or more sets of playback controls and alarm configuration affordances with a set of time affordances for setting the time of a first alarm, according to a determination that the detected user input corresponds to the selection of an alarm configuration affordance.

[0036] A method is described according to some embodiments. A method implemented in an electronic device having a display is a request to play an audio media item of a first type, the request being to identify an audio media item of a first type but not an individual audio media item of a first type, to play a first audio media item of a first type in response to the receipt of a request to play an audio media item of a first type, and to display an affordance, which, when selected, allows for the playback of a second audio media item of a first type, the second audio media item of a first type to be different from the first audio media item of a first type.

[0037] According to some embodiments, a non-temporary computer-readable storage medium is described. The non-temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs include instructions which

[0038] According to some embodiments, a temporary computer-readable storage medium is described. The temporary computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs include instructions that cause a request to be received which identifies a first type of audio media item, but does not identify individual audio media items of the first type, and in response to the receipt of a request to play a first type of audio media item, cause a first audio media item of the first type of audio media item to be played, and cause an affordance to be displayed which, when selected, causes a second audio media item of the first type of audio media item to be played, the second audio media item of the first type of audio media item to be different from the first audio media item of the first type of audio media item.

[0039] According to some embodiments, an electronic device is described. This electronic device comprises a display device, one or more processors, and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for causing a request to play a first type of audio media item, the request identifying a first type of audio media item but not identifying individual first type audio media items, causing a first audio media item of the first type of audio media item to be played in response to the receipt of the request to play a first type of audio media item, and instructions for displaying an affordance which, if selected, plays a second audio media item of the first type of audio media item, the second audio media item of the first type of audio media item being different from the first audio media item of the first type of audio media item. Other embodiments of this aspect include corresponding computer systems, devices, and computer programs recorded in one or more computer storage devices, each configured to perform the operations of the Method.

[0040] According to some embodiments, an electronic device is described. This electronic device includes a display device and means for receiving a request to play a first type of audio media item, the request being means for identifying a first type of audio media item but not identifying individual first type audio media items, and in response to receiving a request to play a first type of audio media item, the electronic device plays a first audio media item from the first type of audio media items and displays an affordance which, when selected, plays a second audio media item from the first type of audio media items, wherein the second audio media item from the first type of audio media items is different from the first audio media item from the first type of audio media items.

[0041] The executable instructions that perform these functions are contained in a non-temporary computer-readable storage medium or other computer program product configured to be executed by one or more processors, at the discretion of the user.

[0042] Therefore, devices are provided with faster and more efficient methods and interfaces for managing and controlling external devices from electronic devices, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can complement or replace other methods for managing and controlling external devices from electronic devices. [Brief explanation of the drawing]

[0043] To better understand the various embodiments described, the following “Modes for Carrying Out the Invention” should be referenced in conjunction with the following drawings, and similar reference numbers throughout the following drawings refer to the corresponding parts.

[0044] [Figure 1A] This is a block diagram showing a portable multifunctional device having a touch-sensitive display according to several embodiments.

[0045] [Figure 1B] This is a block diagram showing exemplary components for event handling according to several embodiments.

[0046] [Figure 1C] This block diagram shows a system and environment for implementing a digital assistant in several embodiments.

[0047] [Figure 2] This shows a portable multifunctional device having a touchscreen according to several embodiments.

[0048] [Figure 3] This is a block diagram of an exemplary multifunctional device comprising a display and a touch-sensitive surface according to several embodiments.

[0049] [Figure 4A] This shows an exemplary user interface for an application menu on a portable multifunction device according to several embodiments.

[0050] [Figure 4B] This document illustrates an exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display, according to several embodiments.

[0051] [Figure 5A] Several embodiments of personal electronic devices are shown.

[0052] [Figure 5B] This is a collection of personal electronic devices in several embodiments.

[0053] [Figure 5C] The following are exemplary components of a personal electronic device having a touch-sensitive display and an intensity sensor, according to some embodiments. [Figure 5D] The following are exemplary components of a personal electronic device having a touch-sensitive display and an intensity sensor, according to some embodiments.

[0054] [Figure 5E] This document shows exemplary components and user interfaces of a personal electronic device according to some embodiments. [Figure 5F] This document shows exemplary components and user interfaces of a personal electronic device according to some embodiments. [Figure 5G] This document shows exemplary components and user interfaces of a personal electronic device according to some embodiments. [Figure 5H] This document shows exemplary components and user interfaces of a personal electronic device according to some embodiments.

[0055] [Figure 5I] Several embodiments of electronic devices are shown.

[0056] [Figure 5J] This is a block diagram showing an electronic device in several embodiments.

[0057] [Figure 6A] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6B] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6C] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6D] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6E] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6F] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6G] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6H] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6I] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6J] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6K] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6L] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6M] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6N] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6O] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6P]This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6Q] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 6R] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments.

[0058] [Figure 7A] This flowchart illustrates a method for managing controllable external devices according to several embodiments. [Figure 7B] This flowchart illustrates a method for managing controllable external devices according to several embodiments.

[0059] [Figure 8A] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8B] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8C] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8D] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8E] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8F] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8G] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8H]This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8I] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments. [Figure 8J] This document illustrates exemplary user interfaces for managing controllable external devices, based on several embodiments.

[0060] [Figure 9A] This flowchart illustrates a method for managing controllable external devices according to several embodiments. [Figure 9B] This flowchart illustrates a method for managing controllable external devices according to several embodiments.

[0061] [Figure 10A] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10B] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10C] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10D] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10E] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10F] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10G] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10H] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10I] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10J] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10K] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10L] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10M] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10N] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10O] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10P] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10Q] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10R] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10S] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10T] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10U] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10V] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10W] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10X] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10Y] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10Z] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10AA] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10AB] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments. [Figure 10AC] This document illustrates exemplary user interfaces for managing grouped, controllable external devices, according to several embodiments.

[0062] [Figure 11A] This flowchart illustrates a method for managing grouped, controllable external devices according to several embodiments. [Figure 11B]This flowchart illustrates a method for managing grouped, controllable external devices according to several embodiments.

[0063] [Figure 12A] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12B] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12C] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12D] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12E] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12F] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12G] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12H] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12I] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12J] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12K] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12L] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12M]This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12N] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12O] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12P] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments. [Figure 12Q] This document presents exemplary user interfaces for managing a thermostat, based on several embodiments.

[0064] [Figure 13A] This is a flowchart illustrating a method for managing a thermostat according to several embodiments. [Figure 13B] This is a flowchart illustrating a method for managing a thermostat according to several embodiments.

[0065] [Figure 14A] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14B] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14C] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14D] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14E] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14F]The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14G] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14H] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14I] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14J] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14K] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14L] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14M] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14N] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14O] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14P] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14Q]The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14R] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14S] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments. [Figure 14T] The following are exemplary user interfaces for managing a group of controllable external devices, including speakers, according to several embodiments.

[0066] [Figure 15] This flowchart illustrates a method for managing a group of controllable external devices, including speakers, according to several embodiments.

[0067] [Figure 16A] This document illustrates exemplary user interfaces for managing speakers, based on several embodiments. [Figure 16B] This document illustrates exemplary user interfaces for managing speakers, based on several embodiments. [Figure 16C] This document illustrates exemplary user interfaces for managing speakers, based on several embodiments. [Figure 16D] This document illustrates exemplary user interfaces for managing speakers, based on several embodiments. [Figure 16E] This document illustrates exemplary user interfaces for managing speakers, based on several embodiments. [Figure 16F] This document illustrates exemplary user interfaces for managing speakers, based on several embodiments. [Figure 16G] This document illustrates exemplary user interfaces for managing speakers, based on several embodiments.

[0068] [Figure 17] This is a flowchart illustrating a method for managing speakers according to several embodiments.

[0069] [Figure 18A] This document illustrates exemplary user interfaces for requesting the playback of a specific type of sound, based on several embodiments. [Figure 18B] This document illustrates exemplary user interfaces for requesting the playback of a specific type of sound, based on several embodiments. [Figure 18C] This document illustrates exemplary user interfaces for requesting the playback of a specific type of sound, based on several embodiments. [Figure 18D] This document illustrates exemplary user interfaces for requesting the playback of a specific type of sound, based on several embodiments. [Figure 18E] This document illustrates exemplary user interfaces for requesting the playback of a specific type of sound, based on several embodiments. [Figure 18F] This document illustrates exemplary user interfaces for requesting the playback of a specific type of sound, based on several embodiments. [Figure 18G] This document illustrates exemplary user interfaces for requesting the playback of a specific type of sound, based on several embodiments.

[0070] [Figure 19] This flowchart illustrates how electronic devices can be used to reproduce a specific type of sound, according to several embodiments. [Modes for carrying out the invention]

[0071] The following description includes exemplary methods, parameters, etc. However, it should be noted that such descriptions are not intended to limit the scope of this disclosure, but rather are provided as descriptions of exemplary embodiments.

[0072] There is a need for an electronic device that provides an efficient method and interface for managing controllable external devices. In some embodiments, the electronic device provides access to designated (e.g., favorite) controllable devices on a home screen for managing a system of controllable devices. In some embodiments, the user can roughly control a device by switching between two states (e.g., on / off) using an input with one type of characteristic (e.g., tap), and obtain more granular control of the device using an input with different characteristics (e.g., long press). In some embodiments, a user-defined set of devices (e.g., a scene) is controlled in response to user-defined conditions. Various other techniques are also described. Such techniques can reduce the cognitive burden on the user managing controllable external devices, thereby increasing productivity. Furthermore, such techniques can reduce processor and battery power that would normally be wasted on redundant user input.

[0073] The following Figures 1A-1C, 2, 3, 4A-4B, and 5A-5I provide a description of exemplary devices for implementing techniques for managing controllable external devices.

[0074] Figures 6A to 6R and 8A to 8J show exemplary user interfaces for managing controllable external devices. Figures 7A to 7B and 9A to 9B are flowcharts illustrating methods for managing controllable external devices according to several embodiments. The user interfaces in Figures 6A to 6R are used to illustrate processes described later, including the processes in Figures 7 to 7B. The user interfaces in Figures 8A to 8J are used to illustrate processes described later, including the processes in Figures 9A to 9B.

[0075] Figures 10A to 10AC show exemplary user interfaces for managing grouped controllable external devices. Figures 11A to 11B are flowcharts illustrating methods for managing grouped controllable external devices according to several embodiments. The user interfaces in Figures 8A to 8D are used to illustrate a later process, including the process shown in Figures 11A to 11B.

[0076] Figures 12A to 12Q show exemplary user interfaces for managing a thermostat. Figures 13A to 13B are flowcharts illustrating methods for managing a thermostat according to several embodiments. The user interfaces in Figures 12A to 12Q are used to illustrate the processes described later, including the processes in Figures 13A to 13B.

[0077] Figures 14A to 14T show exemplary user interfaces for managing groups of controllable external devices and speakers. Figure 15 is a flowchart illustrating methods for a user interface for managing groups of controllable external devices and speakers according to several embodiments. The user interfaces in Figures 14A to 14T are used to illustrate a process described later, including the process in Figure 15.

[0078] Figures 16A to 16G show exemplary user interfaces for managing speakers. Figure 17 is a flowchart illustrating how to manage speakers according to several embodiments. The user interfaces in Figures 16A to 16G are used to illustrate the processes described later, including the process in Figure 17.

[0079] Figures 18A to 18G show exemplary user interfaces for requesting the playback of a specific type of sound, according to several embodiments. Figure 19 is a flowchart showing how to request the playback of a specific type of sound, according to several embodiments. The user interfaces in Figures 18A to 18G are used to illustrate a process described later, including the process in Figure 19.

[0080] In the following description, terms such as “first,” “second,” etc., are used to describe various elements, but these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the various embodiments described, the first touch may be called the second touch, and similarly, the second touch may be called the first touch. Both the first touch and the second touch are touches, but they are not the same touch.

[0081] The terminology used in the descriptions of the various embodiments described herein is intended solely to describe specific embodiments and is not intended to be limiting. In the descriptions of the various embodiments and the accompanying claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless otherwise explicitly stated in the context. Furthermore, it should be understood that, as used herein, the term “and / or” refers to and includes any and all possible combinations of one or more of the enumerated items relating to the description. It will be further understood that, as used herein, the terms “includes,” “comprises,” and / or “comprising” specify the presence of the described features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0082] The phrase "if" can be interpreted, at will, in response to the context, as meaning "when" or "upon," or "in response to determining" or "in response to detecting." Similarly, the phrases "if it is determined" or "if [a stated condition or event] is detected" can be interpreted, at will, in response to the context, as meaning "upon determining" or "in response to determining," or "[the stated condition or event] is detected" or "[the stated condition or event] is detected."

[0083] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, which also includes other functions such as PDA functionality and / or music player functionality. Exemplary embodiments of portable multifunction devices include, but are not limited to, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc., Cupertino, California. Optionally, other portable electronic devices such as laptop or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads) are also used. It should also be understood that in some embodiments, the device is not a portable communication device but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0084] The following discussion describes electronic devices including displays and touch-sensitive surfaces. However, it should be understood that electronic devices optionally include one or more other physical user interface devices such as physical keyboards, mice, and / or joysticks.

[0085] The device typically supports a variety of applications, including drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0086] Various applications running on this device optionally utilize at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as the corresponding information displayed on the device, are optionally adjusted and / or modified on an application-by-application basis and / or within each application. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally supports a variety of applications with intuitive and transparent user interfaces for the user.

[0087] Here, we turn our attention to embodiments of portable devices equipped with touch-sensitive displays. Figure 1A is a block diagram of a portable multifunction device 100 having a touch-sensitive display system 112 according to several embodiments. The touch-sensitive display 112 may be conveniently referred to as a “touchscreen” and may be known or referred to as a “touch-sensitive display system”. Device 100 includes memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact intensity sensors 165 (e.g., touch-sensitive surfaces such as the touch-sensitive display system 112 of Device 100) that detect the intensity of contact on Device 100. Device 100 optionally includes one or more tactile output generators 167 that generate tactile outputs on Device 100 (for example, on touch-sensitive surfaces such as the touch-sensitive display system 112 of Device 100 or the touchpad 355 of Device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0088] In this specification and in the claims, the term “strength” of contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of contact on the touch-sensitive surface (e.g., finger contact), or a proxy for the force or pressure of contact on the touch-sensitive surface. The strength of contact has a range of values, including at least four distinct values, and more typically, including several hundred (e.g., at least 256) distinct values. The strength of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated force of contact. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or change in the contact area detected on the touch-sensing surface, the capacitance and / or change in the touch-sensing surface adjacent to the contact, and / or the resistance and / or change in the touch-sensing surface adjacent to the contact may optionally be used as a substitute for the force or pressure of the contact on the touch-sensing surface. In some implementations, the substitute measurement for the force or pressure of the contact is used directly to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some implementations, the substitute measurement for the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold is exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). By using the intensity of contact as an attribute of user input, it becomes possible to enable user access to additional device functions that would normally be inaccessible to the user on a reduced-size device where the area for displaying affordances (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical control unit such as a knob or button) is limited.

[0089] As used herein and in the claims, the term “tactile output” means a physical displacement of the device relative to its previous position, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by the user through the user’s sense of touch. For example, in a situation where the device or a component of the device is in contact with the touch-sensitive surface of the user (e.g., the user’s fingers, palm, or other part of their hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical properties of the device or a component of the device. For example, the movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) may be optionally interpreted by the user as a “down-click” or “up-click” of a physical actuator button. In some cases, the user may feel a tactile sensation such as a “down-click” or “up-click” even when there is no movement of a physical actuator button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user’s action. As another example, movement of a touch-sensitive surface may be interpreted or perceived by the user as "roughness" of that surface, even if there is no change in the smoothness of the touch-sensitive surface. Such user interpretations of touch depend on the user's personal sensory perception, but there are many touch sensory perceptions common to the majority of users. Therefore, when a tactile output is described as corresponding to a user's specific sensory perception (e.g., "up-click," "down-click," "roughness"), unless otherwise stated, the generated tactile output corresponds to the physical displacement of the device or its components that produce the described sensory perception of a typical (or average) user.

[0090] It should be understood that device 100 is merely an example of a portable multifunction device, and that device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of those components. The various components shown in Figure 1A are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing circuits and / or application-specific integrated circuits.

[0091] Memory 102 optionally includes high-speed random-access memory and optionally includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0092] The peripheral interface 118 can be used to connect the device's input and output peripherals to the CPU 120 and memory 102. One or more processors 120 operate or execute various software programs and / or instruction sets stored in memory 102 to perform various functions for device 100 and process data. In some embodiments, the peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0093] The RF (radio frequency) circuit 108 transmits and receives RF signals, also known as electromagnetic signals. The RF circuit 108 converts electrical signals to electromagnetic signals, or electromagnetic signals to electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, which include, but are not limited to, antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, CODEC chipsets, subscriber identity module (SIM) cards, and memory. The RF circuit 108 optionally communicates wirelessly with networks such as the Internet, also known as the World Wide Web (WWW), intranets, and / or wireless networks such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs), as well as with other devices. The RF circuit 108 optionally includes a well-known circuit for detecting a near-field communication (NFC) field using a short-range communication radio or the like. The wireless communication optionally includes GSM (Global System for Mobile Communications) (registered trademark), Enhanced Data GSM Environment (EDGE), HSDPA (high-speed downlink packet access), HSUPA (high-speed uplink packet access), and EV-DO (Evolution,Data-Only), HSPA, HSPA+, DC-HSPDA (Dual-Cell HSPA), LTE (Long Term Evolution), Near Field Communication (NFC), W-CDMA (Wideband Code Division Multiple Access), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth (Registered Trademark), Bluetooth Low Energy (BTLE), Wi-Fi (Wireless Fidelity) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, Email Protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), Instant Messaging (e.g., Extensible Messaging and Presence Protocol (XMPP)), SIMPLE (Session Initiation) Using any multiple communication standards, protocols, and technologies, including, but not limited to, any other suitable communication protocols, including the Protocol for Instant Messaging and Presence Leveraging Extensions, IMPS (Instant Messaging and Presence Service), and / or SMS (Short Message Service), or any communication protocols not yet developed as of the filing date of this specification.

[0094] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral interface 118, converts this audio data into an electrical signal, and transmits this electrical signal to the speaker 111. The speaker 111 converts the electrical signal into human audible sound waves. The audio circuit 110 also receives the electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data and transmits this audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or RF circuit 108 by the peripheral interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., 212 in Figure 2). The headset jack provides an interface between the audio circuit 110 and detachable audio input / output peripherals such as output-only headphones or headsets that have both output (e.g., headphones for one or both ears) and input (e.g., a microphone).

[0095] The I / O subsystem 106 connects input / output peripherals on device 100, such as the touchscreen 112 and other input control devices 116, to the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, a depth camera controller 169, an intensity sensor controller 159, a haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. One or more input controllers 160 receive electrical signals from / transmit electrical signals to other input control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controllers 160 are optionally connected to (or not connected to) one of the following: a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. One or more buttons (e.g., 208 in Figure 2) optionally include up and down buttons for volume control of speaker 111 and / or microphone 113. One or more buttons optionally include push buttons (e.g., 206 in Figure 2).

[0096] As described in U.S. Patent Application No. 11 / 322,549, “Unlocking a Device by Performing Gestures on an Unlock Image,” filed December 23, 2005, U.S. Patent No. 7,657,849, incorporated herein by reference in its entirety, a quick press of a push button optionally unlocks the touchscreen 112, or optionally initiates a process to unlock the device using gestures on the touchscreen. A longer press of a push button (e.g., 206) optionally turns power on or off the device 100. The functionality of one or more of the buttons is optionally customizable by the user. The touchscreen 112 is used to implement virtual or soft buttons and one or more soft keyboards.

[0097] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or transmits electrical signals to the touchscreen 112. The touchscreen 112 displays a visual output to the user. This visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively, “graphics”). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0098] The touchscreen 112 has a touch-sensing surface, sensor, or set of sensors that accept user input based on touch and / or tactile contact. The touchscreen 112 and the display controller 156 (along with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of contact) on the touchscreen 112 and translate the detected contact into interaction with user interface objects displayed on the touchscreen 112 (e.g., one or more soft keys, icons, web pages, or images). In an exemplary embodiment, the point of contact between the touchscreen 112 and the user corresponds to the user's finger.

[0099] The touchscreen 112 optionally uses LCD (liquid crystal display) technology, LPD (polymer light-emitting display) technology, or LED (light-emitting diode) technology, but other display technologies may also be used in other embodiments. The touchscreen 112 and the display controller 156 optionally, but not limited to, use any of several currently known or future-developed touch sensing technologies, including capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements that determine one or more points of contact with the touchscreen 112, to detect contact and any movement or interruption thereof. In exemplary embodiments, projected mutual capacitive sensing technology is used, such as that found in the iPhone® and iPod Touch® from Apple Inc. of Cupertino, California.

[0100] The touch-sensitive displays in some embodiments of the touchscreen 112 are, optionally, similar to the multi-touch-sensitive touchpads described in U.S. Patent No. 6,323,846 (Westerman et al.), No. 6,570,557 (Westerman et al.), and / or No. 6,677,932 (Westerman), and / or U.S. Patent Publication No. 2002 / 0015024(A1), which are each incorporated herein by reference as a whole. However, the touchscreen 112 displays visual output from device 100, whereas the touch-sensitive touchpad does not provide visual output.

[0101] The touch-sensitive displays in several embodiments of the touchscreen 112 include: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, "Multipoint Touch Surface Controller"; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, "Multipoint Touchscreen"; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, "Gestures For Touch Sensitive Input Devices"; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, "Gestures For Touch Sensitive Input Devices"; and (5) U.S. Patent Application No. 11 / 038,590, filed January 18, 2005, "Mode-Based Graphical User Interfaces For Touch Sensitive Input These are described in the following applications: (6) U.S. Patent Application No. 11 / 228,758, filed September 16, 2005, "Virtual Input Device Placement On A Touch Screen User Interface"; (7) U.S. Patent Application No. 11 / 228,700, filed September 16, 2005, "Operation Of A Computer With A Touch Screen Interface"; (8) U.S. Patent Application No. 11 / 228,737, filed September 16, 2005, "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard"; and (9) U.S. Patent Application No. 11 / 367,749, filed March 3, 2006, "Multi-Functional Hand-Held Device". All of these applications are incorporated herein by reference as a whole.

[0102] The touchscreen 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. The user optionally touches the touchscreen 112 using any suitable object or attachment, such as a stylus or finger. In some embodiments, the user interface is designed to primarily handle finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​the finger on the touchscreen. In some embodiments, the device translates coarse finger input into a precise pointer / cursor position or command to perform an action desired by the user.

[0103] In some embodiments, in addition to the touchscreen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensing area of ​​the device that, unlike the touchscreen, does not display a visual output. The touchpad is optionally a touch-sensing surface separate from the touchscreen 112 or an extension of the touch-sensing surface formed by the touchscreen.

[0104] Device 100 also includes a power system 162 that supplies power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., a battery, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)), and any other components associated with generating, managing, and distributing power within the portable device.

[0105] The device 100 also optionally includes one or more optical sensors 164. Figure 1A shows optical sensors coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensors 164 optionally include a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. The optical sensors 164 receive light from the environment projected through one or more lenses and convert that light into data representing an image. The optical sensors 164 work in conjunction with an imaging module 143 (also called a camera module) to optionally capture still images or video. In some embodiments, the optical sensors are located on the back of the device 100, opposite to the touchscreen display 112 on the front of the device, so that the touchscreen display can be effectively used as a viewfinder for acquiring still images and / or video images. In some embodiments, the light sensor is located on the front of the device, so that the user's image is optionally acquired for video conferencing, and the user views other video conferencing participants on the touchscreen display. In some embodiments, the position of the light sensor 164 can be changed by the user (for example, by rotating the lens and sensor within the device housing), so that a single light sensor 164 is used for both video conferencing and acquiring still and / or video images, together with the touchscreen display.

[0106] Device 100 also optionally includes one or more depth camera sensors 175. Figure 1A shows a depth camera sensor coupled to a depth camera controller 169 in the I / O subsystem 106. The depth camera sensor 175 receives data from the environment to create a three-dimensional model of an object in the scene (e.g., a face) from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with an imaging module 143 (also called a camera module), the depth camera sensor 175 is optionally used to determine depth maps of various parts of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is positioned on the front of Device 100 so that an image of the user with depth information is optionally acquired for video conferencing while the user is viewing other video conference participants on a touchscreen display, and so that a selfie image with depth map data is captured. In some embodiments, the depth camera sensor 175 is positioned on the back of the device, or on both the back and front of Device 100. In some embodiments, the position of the depth camera sensor 175 can be changed by the user (for example, by rotating the lens and sensor within the device housing), so that the depth camera sensor 175, together with the touchscreen display, can be used for video conferencing as well as for acquiring still and / or video images.

[0107] Device 100 also optionally includes one or more contact strength sensors 165. Figure 1A shows a contact strength sensor coupled to a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, pressure-power sensors, optical force sensors, capacitive touch-sensing surfaces, or other strength sensors (e.g., sensors used to measure the force (or pressure) of contact on a touch-sensing surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information, or a proxy for pressure information) from the environment. In some embodiments, at least one contact strength sensor is positioned juxtaposed with or adjacent to a touch-sensing surface (e.g., a touch-sensing display system 112). In some embodiments, at least one contact strength sensor is located on the back of Device 100, opposite the touchscreen display 112 located on the front of Device 100.

[0108] The device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows a proximity sensor 166 coupled to a peripheral interface 118. Alternatively, the proximity sensor 166 is optionally coupled to an input controller 160 in the I / O subsystem 106. The proximity sensor 166 may optionally function as described in U.S. Patent Applications 11 / 241,839, “Proximity Detector In Handheld Device,” 11 / 240,788, “Proximity Detector In Handheld Device,” 11 / 620,702, “Using Ambient Light Sensor To Augment Proximity Sensor Output,” 11 / 586,862, “Automated Response To And Sensing Of User Activity In Portable Devices,” and 11 / 638,251, “Methods And Systems For Automatic Configuration Of Peripherals,” which are incorporated herein by reference as a whole. In some embodiments, when the multifunction device is positioned near the user's ear (for example, when the user is making a phone call), the proximity sensor turns off and disables the touchscreen 112.

[0109] Device 100 also optionally includes one or more tactile output generators 167. Figure 1A shows a tactile output generator coupled to a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices such as a speaker or other audio component, and / or electromechanical devices that convert energy into linear motion, such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts an electrical signal into a tactile output on the device). The contact intensity sensor 165 receives a tactile feedback generation command from the tactile feedback module 133 and generates a tactile output on device 100 that can be sensed by the user of device 100. In some embodiments, at least one tactile output generator is located on or near a touch-sensitive surface (e.g., a touch-sensitive display system 112) and optionally generates a tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward from the surface of device 100) or horizontally (e.g., forward and backward in the same plane as the surface of device 100). In some embodiments, at least one tactile output generator sensor is located on the back of device 100, opposite the touchscreen display 112 located on the front of device 100.

[0110] The device 100 also optionally includes one or more accelerometers 168. Figure 1A shows an accelerometer 168 coupled to a peripheral interface 118. Alternatively, the accelerometer 168 is optionally coupled to an input controller 160 in the I / O subsystem 106. The accelerometer 168 optionally functions as described in U.S. Patent Publication No. 20050190059, “Acceleration-based Theft Detection System for Portable Electronic Devices,” and U.S. Patent Publication No. 20060017692, “Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer,” both of which are incorporated herein by reference as a whole. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on an analysis of data received from one or more accelerometers. In addition to the accelerometer 168, device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation of device 100 (e.g., longitudinal or transverse).

[0111] In some embodiments, the software components stored in memory 102 include an operating system 126, a communications module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a character input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application (or instruction set) 136. Furthermore, in some embodiments, memory 102 (Figure 1A) or 370 (Figure 3) stores a device / global internal state 157, as shown in Figures 1A and 3. The device / global internal state 157 includes one or more of the following: an active application state indicating which application is active, if there is an application currently active; a display state indicating which applications, views, or other information occupy different areas of the touchscreen display 112; a sensor state including information obtained from various sensors and input control devices 116 of the device; and position information relating to the device's position and / or orientation.

[0112] An operating system 126 (for example, an embedded operating system such as Darwin®, RTXC®, LINUX®, UNIX®, OS X®, iOS®, WINDOWS®, or VxWorks®) includes various software components and / or drivers that control and manage normal system tasks (e.g., memory management, memory device control, power management, etc.) and facilitates communication between various hardware components and software components.

[0113] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuit 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire®, etc.) are adapted to connect to other devices directly or indirectly via a network (e.g., the Internet, Wi-Fi, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors that are the same as and / or similar to the 30-pin connector used on iPod® (a trademark of Apple Inc.) devices.

[0114] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger being lowered), determining the intensity of the contact (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more events of a finger being dragged), and determining whether contact has stopped (e.g., detecting a finger being lifted or an interruption of contact). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point, represented by a series of contact data, optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions can be optionally applied to a single contact (e.g., a single finger contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0115] In some embodiments, the contact / motion module 130 uses one or more sets of intensity thresholds to determine whether an action has been performed by the user (for example, whether the user has "clicked" on an icon). In some embodiments, at least one subset of the intensity thresholds is determined according to a software parameter (for example, the intensity thresholds can be adjusted without changing the physical hardware of device 100, rather than being determined by the activation threshold of a particular physical actuator). For example, the mouse "click" threshold for a trackpad or touchscreen display can be set to one of a range of default thresholds without changing the trackpad or touchscreen display hardware. In addition, in some implementations, the user of the device is provided with software settings to adjust one or more of the sets of intensity thresholds (for example, by adjusting individual intensity thresholds and / or multiple intensity thresholds at once using a system-level click "intensity" parameter).

[0116] The contact / motion module 130 optionally detects gesture input from the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motion, timing, and / or intensity of the detected contact). Therefore, gestures are optionally detected by detecting a specific contact pattern. For example, detecting a finger tap gesture involves detecting a finger down event, followed by a finger up (lift-off) event at the same location (or substantially the same location) as the finger down event (e.g., the location of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface involves detecting a finger down event, followed by one or more finger drag events, and then a finger up (lift-off) event.

[0117] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other display, including components for changing the visual effects of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). In this specification, the term “graphics” includes, but is not limited to, any object that can be displayed to the user, including characters, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0118] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from an application or the like, as needed, along with coordinate data and other graphic characteristic data, specifying the graphics to be displayed, and then generates screen image data to be output to the display controller 156.

[0119] The haptic feedback module 133 includes various software components for generating commands used by the haptic output generator 167 to generate haptic outputs at one or more locations on the device 100 in response to user interaction with the device 100.

[0120] The text input module 134 is optionally a component of the graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other applications that require text input).

[0121] The GPS module 135 determines the device's location and provides this information for use in various applications (for example, to the telephone 138 for location-based dialing, to the camera 143 as image / video metadata, and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0122] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof: ● Contact module 137 (sometimes called the address book or contact list), ●Telephone module 138, ●Video conferencing module 139, ● Email client module 140, ● Instant messaging (IM) module 141, ●Training support module 142, ● Camera module 143 for still images and / or video images, ●Image management module 144, ●Video player module, ● Music player module, ● Browser module 147, ● Calendar module 148, ● A widget module 149 that optionally includes one or more of the following: weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets obtained by the user, as well as user-created widgets 149-6. ●Widget creation module 150 for creating user-created widget 149-6, ●Search module 151, ●Video and music player module 152, which integrates a video player module and a music player module. ●Memo Module 153, ●Map module 154, and / or, ● Online video module 155.

[0123] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, Java®-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0124] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the contact module 137 is used to manage an address book or contact list (stored, for example, in the application internal state 192 of the contact module 137 in memory 102 or memory 370), including optionally adding names to the address book, removing names from the address book, associating names with phone numbers, email addresses, physical addresses, or other information, associating images with names, categorizing and sorting names, and providing phone numbers or email addresses to initiate and / or facilitate communication via telephone 138, video conferencing module 139, email 140, or IM 141.

[0125] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and character input module 134, the telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in the contact module 137, modify entered telephone numbers, dial each telephone number, conduct a conversation, and disconnect or hang up when the conversation is finished. As previously mentioned, wireless communication optionally uses one of several communication standards, protocols, and technologies.

[0126] In conjunction with the RF circuit 108, audio circuit 110, speaker 111, microphone 113, touchscreen 112, display controller 156, light sensor 164, light sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, the video conferencing module 139 includes executable commands for starting, running, and ending video conferences between the user and one or more other participants in accordance with user commands.

[0127] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the email client module 140 includes executable commands for creating, sending, receiving, and managing emails in response to user commands. In conjunction with the image management module 144, the email client module 140 makes it very easy to create and send emails containing still or video images captured by the camera module 143.

[0128] The instant messaging module 141, in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and character input module 134, includes executable commands for inputting character sequences corresponding to instant messages, modifying previously entered characters, sending each instant message (for example, using the SMS (Short Message Service) or MMS (Multimedia Message Service) protocol for telephone-based instant messaging, or XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments, such as those supported by MMS and / or EMS (Enhanced Messaging Service). In this specification, “instant messaging” refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0129] The training support module 142 works in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module to create training (e.g., with time, distance, and / or calorie burn targets), communicate with training sensors (sports devices), receive training sensor data, calibrate sensors used to monitor training, select and play music for training, and includes executable commands for displaying, storing, and transmitting training data.

[0130] In conjunction with the touchscreen 112, display controller 156, light sensor 164, light sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable commands for capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0131] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable commands for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still and / or video images.

[0132] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the browser module 147 includes executable commands for browsing the Internet according to user commands, including searching, linking, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0133] The calendar module 148 works in conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147 to include executable commands for creating, displaying, modifying, and storing calendars and data associated with the calendar (e.g., calendar items, to-do lists, etc.) in accordance with user instructions.

[0134] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, the widget module 149 is a mini-application that can be optionally downloaded and used by the user (e.g., weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5), or a mini-application created by the user (e.g., user-created widget 149-6). In some embodiments, the widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widget).

[0135] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, the widget creation module 150 is used by the user to optionally create widgets (for example, to turn a user-specified portion of a web page into a widget).

[0136] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching for characters, music, sounds, images, videos, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0137] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the video and music player module 152 includes executable commands that allow the user to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable commands for displaying, presenting, or otherwise playing video (on the touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0138] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the memo module 153 includes executable commands for creating and managing memos, to-do lists, etc., according to user commands.

[0139] In conjunction with the RF circuit 108, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, the map module 154 is optionally used to receive, display, modify, and store maps and map-related data (e.g., driving directions, data on shops and other points of interest in or near a particular location, and other location-based data) in accordance with user commands.

[0140] In conjunction with the touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions that enable the user to access, browse, receive (e.g., by streaming and / or downloading), play (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and perform other management of online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Further descriptions of online video applications can be found in U.S. Provisional Patent Application No. 60 / 936,562, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed June 20, 2007, and U.S. Patent Application No. 11 / 968,067, “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” filed December 31, 2007, the contents of which are incorporated herein by reference in their entirety.

[0141] Each of the modules and applications identified above corresponds to a set of executable instructions that perform one or more of the functions described above and the methods described in this application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) do not need to be implemented as separate software programs, procedures, or modules; therefore, in various embodiments, various subsets of these modules may be optionally combined or otherwise reconfigured. For example, a video player module may optionally be combined with a music player module to form a single module (e.g., the video and music player module 152 in Figure 1A). In some embodiments, memory 102 may optionally store a subset of the modules and data structures identified above. Furthermore, memory 102 may optionally store additional modules and data structures not described above.

[0142] In some embodiments, device 100 is a device in which the operation of a predetermined set of functions in the device is performed solely through a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for device 100 to operate, the number of physical input control devices (push buttons, dials, etc.) on device 100 is optionally reduced.

[0143] A default set of functions, which are performed only through the touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to the main menu, home menu, or root menu. In such embodiments, a “menu button” is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0144] Figure 1B is a block diagram showing exemplary components for event processing according to several embodiments. In some embodiments, memory 102 (Figure 1A) or 370 (Figure 3) includes an event sorter 170 (e.g., within the operating system 126) and each application 136-1 (e.g., any of the aforementioned applications 137-151, 155, 380-390).

[0145] The event sorter 170 receives event information and determines the application 136-1 to which the event information will be distributed, and the application view 191 of application 136-1. The event sorter 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates the current application view displayed on the touch-sensitive display 112 when the application is active or running. In some embodiments, the device / global internal state 157 is used by the event sorter 170 to determine which application is currently active, and the application internal state 192 is used by the event sorter 170 to determine the application view 191 to which the event information will be distributed.

[0146] In some embodiments, the application internal state 192 includes additional information such as resume information to be used when the application 136-1 resumes execution, user interface state information that indicates or is ready to display information displayed by the application 136-1, a state queue that enables the user to return to a previous state or view of the application 136-1, and one or more redo / undo queues of previous actions performed by the user.

[0147] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (for example, user touch on the touch-sensitive display 112 as part of a multi-touch gesture). The peripheral interface 118 transmits information received from the I / O subsystem 106, or from sensors such as the proximity sensor 166, accelerometer 168, and / or microphone 113 (via the audio circuit 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display 112 or touch-sensitive surface.

[0148] In some embodiments, the event monitor 171 sends requests to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 transmits event information. In other embodiments, the peripheral interface 118 transmits event information only when there is a significant event (e.g., reception of input exceeding a predetermined noise threshold and / or exceeding a predetermined duration).

[0149] In some embodiments, the event sorter 170 also includes a hit view determination module 172 and / or an active event recognition determination module 173.

[0150] The hit view determination module 172 provides a software procedure for determining where in one or more views a sub-event occurred when the touch-sensitive display 112 displays two or more views. A view consists of control devices and other elements that the user can see on the display.

[0151] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of each application) in which a touch is detected optionally corresponds to a program level within the application's program hierarchy or view hierarchy. For example, the lowest-level view in which a touch is detected optionally refers to a hit view, and the set of events recognized as appropriate input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0152] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views arranged in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchy whose sub-events should be processed. In most situations, the hit view is the lowest-level view where the initiating sub-event (e.g., the first sub-event in a sub-event sequence that forms an event or potential event) occurs. Once a hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source that identified it as the hit view.

[0153] The active event recognition determination module 173 determines which view in the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognition determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognition determination module 173 determines that all views, including the physical location of the sub-event, are actively involved views, and therefore all actively involved views should receive a particular sequence of sub-events. In other embodiments, even if the touch sub-event is entirely confined to an area associated with one particular view, higher-level views in the hierarchy still remain actively involved views.

[0154] The event dispatcher module 174 dispatches event information to an event recognition unit (e.g., an event recognition unit 180). In embodiments including an active event recognition unit determination module 173, the event dispatcher module 174 distributes the event information to the event recognition unit determined by the active event recognition unit determination module 173. In some embodiments, the event dispatcher module 174 stores the event information acquired by each event receiving unit 182 in an event queue.

[0155] In some embodiments, the operating system 126 includes an event sorter 170. Alternatively, application 136-1 includes an event sorter 170. In yet another embodiment, the event sorter 170 is a standalone module or part of another module stored in memory 102, such as a contact / motion module 130.

[0156] In some embodiments, application 136-1 includes a plurality of event processing units 190 and one or more application views 191, each containing instructions for handling touch events occurring within each view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognition units 180. Typically, each application view 191 includes a plurality of event recognition units 180. In other embodiments, one or more of the event recognition units 180 are part of a separate module, such as a user interface kit or a higher-level object from which application 136-1 inherits methods and other attributes. In some embodiments, each event processing unit 190 includes one or more of the following: a data update unit 176, an object update unit 177, a GUI update unit 178, and / or event data 179 received from an event sorter 170. The event processing unit 190 optionally uses or calls the data update unit 176, the object update unit 177, or the GUI update unit 178 to update the application's internal state 192. Alternatively, one or more of the application views 191 include one or more respective event processing units 190. In some embodiments, one or more of the data update unit 176, object update unit 177, and GUI update unit 178 are included in each application view 191.

[0157] Each event recognition unit 180 receives event information (e.g., event data 179) from the event sorter 170 and identifies an event from this event information. The event recognition unit 180 includes an event receiving unit 182 and an event comparison unit 184. In some embodiments, the event recognition unit 180 also includes metadata 183 and at least a subset of event distribution commands 188 (optionally including sub-event distribution commands).

[0158] The event receiving unit 182 receives event information from the event sorter 170. The event information includes information about sub-events, such as touches or the movement of touches. Depending on the sub-event, the event information also includes additional information, such as the position of the sub-event. When the sub-event involves the movement of a touch, the event information also optionally includes the speed and direction of the sub-event. In some embodiments, an event includes the rotation of the device from one orientation to another (e.g., from portrait to landscape, or vice versa), and the event information includes corresponding information about the current orientation of the device (also called the device's orientation).

[0159] The event comparison unit 184 compares event information with a predefined definition of an event or sub-event, and based on the comparison, determines an event or sub-event, or determines or updates the state of an event or sub-event. In some embodiments, the event comparison unit 184 includes an event definition 186. The event definition 186 includes definitions of events (e.g., a sequence of default sub-events), such as event 1 (187-1) and event 2 (187-2). In some embodiments, sub-events in event (187) include, for example, touch start, touch end, touch movement, touch cancellation, and multiple touches. In one embodiment, the definition for event 1 (187-1) is a double tap on a displayed object. A double tap includes, for example, a first touch on the displayed object for a predetermined stage (touch start), a first lift-off for a predetermined stage (touch end), a second touch on the displayed object for a predetermined stage (touch start), and a second lift-off for a predetermined stage (touch end). In another embodiment, the definition of event 2(187-2) is a drag on a displayed object. A drag includes, for example, a touch (or contact) on the displayed object to a predetermined stage, movement of the touch across the touch-sensitive display 112, and lift-off of the touch (end of touch). In some embodiments, the event also includes information about one or more associated event processing units 190.

[0160] In some embodiments, the event definition 187 includes an event definition for each user interface object. In some embodiments, the event comparison unit 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view where three user interface objects are displayed on the touch-sensitive display 112, when a touch is detected on the touch-sensitive display 112, the event comparison unit 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with its respective event processing unit 190, the event comparison unit uses the results of the hit test to determine which event processing unit 190 should be activated. For example, the event comparison unit 184 selects the sub-event and the event processing unit associated with the object that triggers the hit test.

[0161] In some embodiments, the definition for each event (187) also includes a delay action that delays the transmission of event information until it is determined whether the sub-event sequence corresponds to the event type of the event recognition unit.

[0162] If each event recognition unit 180 determines that a series of sub-events does not match any of the events in the event definition 186, each event recognition unit 180 enters an event impossible, event failed, or event terminated state and thereafter ignores subsequent sub-events of the touch-based gesture. In this situation, if there are other event recognition units that remain active for the hit view, those event recognition units continue to track and process the sub-events of the ongoing touch-based gesture.

[0163] In some embodiments, each event recognition unit 180 includes metadata 183 having configurable properties, flags, and / or lists that indicate to the actively involved event recognition unit how the event distribution system should perform sub-event distribution. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how the event recognition units interact with each other or how they are made capable of interacting with each other. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how sub-events are distributed to various levels in the view hierarchy or program hierarchy.

[0164] In some embodiments, each event recognition unit 180 activates an event processing unit 190 associated with an event when one or more specific sub-events of an event are recognized. In some embodiments, each event recognition unit 180 delivers event information associated with the event to the event processing unit 190. Activating the event processing unit 190 is separate from sending (and delaying the sending of) sub-events to the respective hit view. In some embodiments, the event recognition unit 180 sets a flag associated with the recognized event, and the event processing unit 190 associated with that flag captures the flag and executes a predetermined process.

[0165] In some embodiments, the event distribution command 188 includes a sub-event distribution command that distributes event information about a sub-event without activating an event processing unit. Instead, the sub-event distribution command distributes event information to an event processing unit associated with a set of sub-events, or to a view that is actively involved. The event processing unit associated with the set of sub-events or the view that is actively involved receives the event information and performs predetermined processing.

[0166] In some embodiments, the data update unit 176 creates and updates data used in application 136-1. For example, the data update unit 176 updates telephone numbers used in contact module 137 or stores video files used in video player module. In some embodiments, the object update unit 177 creates and updates objects used in application 136-1. For example, the object update unit 177 creates new user interface objects or updates the position of user interface objects. The GUI update unit 178 updates the GUI. For example, the GUI update unit 178 prepares display information and sends it to graphics module 132 for display on touch-sensitive display.

[0167] In some embodiments, the event processing unit 190 includes or has access to a data update unit 176, an object update unit 177, and a GUI update unit 178. In some embodiments, the data update unit 176, the object update unit 177, and the GUI update unit 178 are included in a single module of their respective applications 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0168] The foregoing description regarding the handling of user touch events on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 using input devices, but it should be understood that not all of this begins on the touchscreen. For example, mouse movement and mouse button presses, touch movements such as taps, drags, and scrolls on a touchpad, pen stylus input, device movement, verbal commands, detected eye movements, biometric input, and / or any combination thereof may be optionally used as inputs corresponding to sub-events that define the events to be recognized.

[0169] Figure 1C shows block diagrams of System 2100 in various embodiments. In some embodiments, System 2100 implements a digital assistant. The terms “digital assistant,” “virtual assistant,” “intelligent automated assistant,” or “automated digital assistant” refer to any information processing system that infers user intent by interpreting natural language input in the form of utterances and / or text, and performs actions based on that inferred user intent. For example, to perform an inferred user intent, the system performs one or more of the following: identifying a task flow having steps and parameters designed to realize the inferred user intent; inputting specific requirements from the inferred user intent into that task flow; executing that task flow by calling a program, method, service, or API, etc.; and generating an output response to the user in the form of an audible (e.g., utterance) and / or visual form.

[0170] Specifically, a digital assistant can accept user requests, at least partially, in the form of natural language commands, requests, statements, descriptions, and / or inquiries. Typically, a user request asks the digital assistant to either provide information or perform a task. A satisfactory response to this user request includes providing the requested information, performing the requested task, or a combination of the two. For example, a user might ask the digital assistant, "Where am I right now?" Based on the user's current location, the digital assistant might respond, "You are in Central Park near the West Gate." The user might also request the performance of a task, for example, "Please invite my friends to my girlfriend's birthday party next week." In response, the digital assistant could affirm the request by saying, "Yes, right away," and then, on behalf of the user, send appropriate calendar invitations to each of the user's friends listed in the user's electronic address book. During the performance of a requested task, a digital assistant may interact with the user in a continuous conversation involving multiple information exchanges over a prolonged period. Numerous other ways exist to interact with a digital assistant to request information or the performance of various tasks. In addition to providing verbal responses and taking programmed actions, digital assistants may also provide responses in other visual or audio formats, such as text, alarms, music, videos, and animations.

[0171] As shown in Figure 1C, in some embodiments, the digital assistant is implemented according to a client-server model. The digital assistant includes a client-side portion 2102 (hereinafter, "DA client 2102") that runs on the user device 104, and a server-side portion 2106 (hereinafter, "DA server 2106") that runs on the server system 2108. The DA client 2102 communicates with the DA server 2106 through one or more networks 2110. The DA client 2102 provides client-side functions such as user-responsive input and output processing, and communication with the DA server 2106. The DA server 2106 provides server-side functions to any number of DA clients 2102, each residing on a different user device 2104.

[0172] In some embodiments, the DA server 2106 includes a client-responsive I / O interface 2112, one or more processing modules 2114, data and models 2116, and an I / O interface 2118 to external services. The client-responsive I / O interface 2112 facilitates client-responsive input and output processing by the DA server 2106. One or more processing modules 2114 use the data and models 2116 to process voice input and determine user intent based on natural language input. Furthermore, one or more processing modules 2114 perform task execution based on the inferred user intent. In some embodiments, the DA server 2106 communicates with an external service 120 via a network(s) 2110 to complete a task or retrieve information. The I / O interface 2118 to external services facilitates such communication.

[0173] The user device 2104 can be any suitable electronic device. In some embodiments, the user device 2104 is a portable multifunction device (e.g., device 100 described above with reference to Figure 1A), a multifunction device, or another electronic device (e.g., devices 600, 800, 1000). A portable multifunction device is, for example, a mobile phone that also includes other functions such as a PDA and / or music player function. Specific examples of portable multifunction devices include the Apple Watch®, iPhone®, iPod Touch®, and iPad® devices by Apple Inc. (Cupertino, California). Other examples of portable multifunction devices include, but are not limited to, earphones / headphones, speakers, and laptop computers or tablet computers. Furthermore, in some embodiments, the user device 2104 is a non-portable multifunction device. Specifically, the user device 2104 is a desktop computer, a game console, a speaker, a television, or a television set-top box. In some embodiments, the user device 2104 includes a touch-sensitive surface (e.g., a touchscreen display and / or touchpad). Furthermore, the user device 2104 optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick. Various embodiments of electronic devices, such as multifunction devices, are described in further detail below.

[0174] Examples of communication networks (one or more) 2110 include local area networks (LANs) and wide area networks (WANs), such as the Internet. Communication networks (one or more) 2110 are implemented using any known network protocol, including a variety of wired or wireless protocols such as Ethernet, USB (Universal Serial Bus), FireWire, GSM (Global System for Mobile Communications), Enhanced Data GSM Environment (EDGE), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wi-Fi, Voice over Internet Protocol (VoIP), Wi-MAX, or any other suitable communication protocol.

[0175] The server system 2108 is implemented on one or more standalone data processing devices or on a distributed computer network. In some embodiments, the server system 2108 also employs various virtual devices and / or services from third-party service providers (e.g., third-party cloud service providers) to provide the basic computing and / or infrastructure resources of the server system 2108.

[0176] In some embodiments, user device 2104 communicates with DA server 2106 via second user device 2122, which is similar to or identical to user device 2104. User device 2104 is configured to be communicatively coupled to second user device 2122 via a direct communication connection such as Bluetooth, NFC, or BTLE, or via a wired or wireless network such as a local Wi-Fi network. In some embodiments, second user device 2122 is configured to act as a proxy between user device 2104 and DA server 2106. For example, the DA client 2102 of user device 2104 is configured to send information (e.g., user requests received by user device 2104) to DA server 2106 via second user device 2122. DA server 2106 processes the information and returns the relevant data (e.g., data content responding to the user request) to user device 2104 via second user device 2122.

[0177] In some embodiments, user device 2104 is configured to reduce the amount of information transmitted from user device 2104 by communicating with a second user device 2122 with abbreviated requests for data. The second user device 2122 is configured to determine any supplementary information to be added to the abbreviated requests and generate a complete request for transmission to the DA server 2106. This system architecture can favorably allow a user device 2104 with limited communication capabilities and / or limited battery power (e.g., a portable watch or similar small electronic device) to access services provided by the DA server 2106 by using a second user device 2122 with higher communication capabilities and / or battery power (e.g., a mobile phone, laptop computer, tablet computer, etc.) as a proxy to the DA server 2106. Although only two user devices 2104 and 2122 are shown in Figure 1C, it should be understood that system 2100 includes any number and types of user devices configured to communicate with the DA server system 2106 in this proxy configuration in some embodiments.

[0178] The digital assistant shown in Figure 1C includes both a client-side portion (e.g., DA client 2102) and a server-side portion (e.g., DA server 2106), although in some examples, the digital assistant's functions are implemented as independent applications installed on the user device. Furthermore, the allocation of functions between the client and server portions of the digital assistant may vary depending on the implementation. For example, in some embodiments, the DA client is a thin client that provides only user-responsive input and output processing functions, delegating all other functions of the digital assistant to the backend server.

[0179] Figure 2 shows a portable multifunction device 100 having a touchscreen 112 according to several embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment, and in other embodiments described later, the user can select one or more of the graphics by performing gestures on the graphics using, for example, one or more fingers 202 (not shown in the figure to an exact scale) or one or more styluses 203 (not shown in the figure to an exact scale). In some embodiments, the selection of one or more graphics occurs when the user interrupts contact with that one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, up, and / or down), and / or rolls of the finger in contact with the device 100 (right to left, left to right, up, and / or down). In some implementations or situations, accidental contact with a graphic does not result in the selection of that graphic. For example, if the gesture corresponding to selection is a tap, a swipe gesture, which involves sweeping over an application icon, does not arbitrarily select the corresponding application.

[0180] Device 100 also optionally includes one or more physical buttons, such as a "Home" button or a menu button 204. As previously mentioned, the menu button 204 is optionally used to navigate to any application 136 within a set of applications running on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touchscreen 112.

[0181] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push button 206 for turning the device on / off and locking the device, volume control buttons 208, a subscriber identification module (SIM) card slot 210, a headset jack 212, and an external docking / charging port 124. The push button 206 is optionally used to turn the device on / off by pressing down and holding the button down for a predetermined period of time, to lock the device by pressing down and releasing the button before a predetermined time has elapsed, and / or to unlock the device or initiate an unlocking process. In alternative embodiments, device 100 also accepts verbal input via a microphone 113 to activate or deactivate certain functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more tactile output generators 167 for generating tactile output to the user of device 100.

[0182] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to several embodiments. Device 300 does not need to be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia playback device, navigation device, educational device (such as a children's learning toy), game system, or control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 that interconnect these components. The communication buses 320 optionally include circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Device 300 typically includes an input / output (I / O) interface 330, which includes a display 340, which is a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350, a touchpad 355, a tactile output generator 357 that generates tactile output on device 300 (for example, similar to the tactile output generator 167 described above with reference to Figure 1A), and a sensor 359 (for example, an optical, acceleration, proximity, touch-sensing, and / or contact intensity sensor similar to the contact intensity sensor 165 described above with reference to Figure 1A). The memory 370 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and optionally includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU 310. In some embodiments, the memory 370 stores programs, modules, and program-like data structures, modules, and data structures, or subsets thereof, that are stored in the memory 102 of the portable multifunction device 100 (Figure 1A).Furthermore, memory 370 optionally stores additional programs, modules, and data structures that are not present in memory 102 of the portable multifunction device 100. For example, memory 370 of device 300 optionally stores the drawing module 380, presentation module 382, ​​word processing module 384, website creation module 386, disk authoring module 388, and / or spreadsheet module 390, whereas memory 102 of the portable multifunction device 100 (Figure 1A) optionally does not store these modules.

[0183] Each of the elements identified above in Figure 3 is optionally stored in one or more of the memory devices described above. Each of the modules identified above corresponds to an instruction set that performs the function described above. The modules or programs (e.g., instruction sets) identified above do not need to be implemented as separate software programs, procedures, or modules, and therefore, in various embodiments, various subsets of these modules are optionally combined or otherwise reconfigured. In some embodiments, memory 370 optionally stores subsets of the modules and data structures identified above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0184] Next, we turn our attention to an optional embodiment of a user interface that may be implemented, for example, on a portable multi-functional device 100.

[0185] Figure 4A shows an exemplary user interface of an application menu on a portable multifunction device 100 according to several embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, the user interface 400 includes the following elements, or subsets or supersets thereof. ●Signal strength indicator 402 for wireless communication such as cellular signals and Wi-Fi signals. ●Time 404, ●Bluetooth indicator 405, ●Battery status indicator 406, ●Tray 408 contains icons for frequently used applications, such as the following: ○Optionally including an indicator 414 of the number of missed calls or voicemail messages, an icon 416 for the telephone module 138 labeled "Telephone", ○Optionally including an indicator 410 of the number of unread emails, an icon 418 of the email client module 140 labeled "Mail", ○ Icon 420 of browser module 147, labeled "Browser", and ○ Icon 422 for the video and music player module 152, also known as the iPod (trademark of Apple Inc.) module 152, which is labeled "iPod", and ● Icons of other applications, such as the following: ○ Icon 424 of IM module 141, labeled "Message", ○ Icon 426 of calendar module 148, labeled "Calendar", ○ Icon 428 of image management module 144, labeled "Photo" ○ Icon 430 of camera module 143, labeled "Camera" ○ Icon 432 of online video module 155, labeled "online video" ○ Icon 434 of stock widget 149-2, labeled "Stock Price" ○ Icon 436 of map module 154, labeled "Map" ○ Icon 438 of weather widget 149-1, labeled "Weather" ○ Icon 440 of the alarm clock widget 149-4, labeled "Clock" ○ Icon 442 of training support module 142, labeled "Training Support" ○ Icon 444 of memo module 153, labeled as "Memo", and ○ An icon 446 labeled "Settings," which provides access to settings for device 100 and its various applications 136, for a settings application or module.

[0186] It should be noted that the icon labels shown in Figure 4A are for illustrative purposes only. For example, the icon 422 for the video and music player module 152 is labeled "Music" or "Music Player," and other labels are optionally used for various application icons. In some embodiments, the label for each application icon includes the name of the application to which that application icon corresponds. In some embodiments, the label for a particular application icon is different from the name of the application to which that particular application icon corresponds.

[0187] Figure 4B shows an exemplary user interface on a device (e.g., device 300 in Figure 3) having a touch-sensitive surface 451 (e.g., tablet or touchpad 355 in Figure 3) separate from the display 450 (e.g., touchscreen display 112). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of contact on the touch-sensitive surface 451, and / or one or more tactile output generators 357 for generating tactile output to the user of device 300.

[0188] Some of the following examples are given by referring to input on a touchscreen display 112 (a combination of a touch-sensing surface and a display), but in some embodiments, the device detects input on a touch-sensing surface separate from the display shown in Figure 4B. In some embodiments, the touch-sensing surface (e.g., 451 in Figure 4B) has a primary axis (e.g., 452 in Figure 4B) corresponding to a primary axis (e.g., 453 in Figure 4B) on the display (e.g., 450). According to these embodiments, the device detects contact with the touch-sensing surface 451 (e.g., 460 and 462 in Figure 4B) at locations corresponding to each location on the display (e.g., 460 corresponds to 468 and 462 corresponds to 470 in Figure 4B). In this way, user input (e.g., touches 460 and 462, and their movements) detected by the device on a touch-sensitive surface (e.g., 451 in Figure 4B) is used by the device to operate the user interface on the display of the multifunction device (e.g., 450 in Figure 4B) when the touch-sensitive surface is separate from the display. It should be understood that a similar method may be optionally used for other user interfaces described herein.

[0189] In addition, while the following examples are given primarily with reference to finger input (e.g., finger touch, finger tap gesture, finger swipe gesture), it should be understood that in some embodiments, one or more of the finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may be optionally replaced by a mouse click (e.g., instead of touch) followed by cursor movement along the swipe path (e.g., instead of touch movement). As another example, a tap gesture may be optionally replaced with a mouse click while the cursor is positioned over the tap gesture location (e.g., instead of detecting touch and then ceasing touch detection). Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice may be optionally used simultaneously, or mouse and finger touch may be optionally used simultaneously.

[0190] FIG. 5A shows an exemplary personal electronic device 500. The device 500 includes a body 502. In some embodiments, the device 500 can include some or all of the features described with respect to devices 100 and 300 (e.g., FIGS. 1A-4B). In some embodiments, the device 500 has a touch-sensitive display screen 504, hereinafter referred to as the touch screen 504. The device 500 alternatively, or in addition to the touch screen 504, has a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, the touch screen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors that detect the intensity of an applied contact (e.g., a touch). One or more intensity sensors of the touch screen 504 (or touch-sensitive surface) can provide output data representing the intensity of the touch. The user interface of the device 500 can respond to touches based on the intensity of the touch, which means that touches of different intensities can invoke different user interface operations on the device 500.

[0191] Exemplary techniques for detecting and processing touch intensity can be found in, for example, International Patent Application No. PCT / US2013 / 040061, filed on May 8, 2013, published as International Patent No. WO / 2013 / 169849, "Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application", and International Patent Application No. PCT / US2013 / 069483, filed on November 11, 2013, published as International Patent No. WO / 2014 / 105276, "Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships", each of which is incorporated herein by reference in its entirety.

[0192] In some embodiments, device 500 has one or more input mechanisms 506 and 508. The input mechanisms 506 and 508, if included, can be physical mechanisms. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms, if included, can enable device 500 to be attached to, for example, hats, eyewear, earrings, necklaces, shirts, jackets, bracelets, watch bands, chains, pants, belts, shoes, wallets, backpacks, etc. These attachment mechanisms enable the user to wear device 500.

[0193] Figure 5B shows an exemplary personal electronic device 500. In some embodiments, the device 500 may include some or all of the components described with respect to Figures 1A, 1B, and 3. The device 500 has a bus 512 that operably connects an I / O section 514 to one or more computer processors 516 and memory 518. The I / O section 514 may be connected to a display 504, which may have a touch-sensing component 522 and optionally a strength sensor 524 (e.g., a contact strength sensor). In addition, the I / O section 514 may be connected to a communication unit 530 that receives application and operating system data using Wi-Fi, Bluetooth, near-field communication (NFC), cellular, and / or other wireless communication techniques. The device 500 may include input mechanisms 506 and / or 508. The input mechanism 506 may optionally be, for example, a rotatable input device or a pressable and rotatable input device. In some examples, the input mechanism 508 may optionally be a button.

[0194] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes a variety of sensors such as a GPS sensor 532, an accelerometer 534, a direction sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or a combination thereof, all of which can be operably connected to the I / O section 514.

[0195] The memory 518 of the personal electronic device 500 may include one or more non-temporary computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, can cause those computer processors to implement, for example, the techniques described below. The computer-readable storage media can be any medium that can tangibly contain or store computer-executable instructions for use by or connected to an instruction execution system, apparatus, or device. In some examples, the storage medium is a temporary computer-readable storage medium. In some examples, the storage medium is a non-temporary computer-readable storage medium. The non-temporary computer-readable storage medium may include, but is not limited to, magnetic, optical, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, CDs, DVDs, or optical disks based on Blu-ray® technology, as well as resident solid-state memory such as flash and solid-state drives. The personal electronic device 500 may include, but is not limited to, the components and configurations shown in Figure 5B, and may include other or additional components in multiple configurations.

[0196] In this specification, the term "affordance" refers to user-interactive graphical user interface objects that are optionally displayed on the display screens of devices 100, 300, 500, and / or 600 (Figures 1A, 3, and 5A-5B). For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) each optionally constitute affordances.

[0197] As used herein, the term “focus selector” refers to an input element that indicates the current part of the user interface that the user is interacting with. In some implementations, including a cursor or other location marker, the cursor acts as a “focus selector,” and therefore, when input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 in Figure 3 or touch-sensitive surface 451 in Figure 4B) while the cursor is positioned over a particular user interface element (e.g., a button, window, slider, or other user interface element), the particular user interface element is adjusted according to the detected input. In some implementations, including a touchscreen display that enables direct interaction with user interface elements on the touchscreen display (e.g., touch-sensitive display system 112 in Figure 1A or touchscreen 112 in Figure 4A), detected contact on the touchscreen acts as a “focus selector,” and therefore, when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus is moved from one area of ​​the user interface to another without corresponding cursor movement or touch movement on the touchscreen display (e.g., by using the tab key or arrow keys to move focus from one button to another), and in these implementations, the focus selector moves in accordance with the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is generally a user interface element (or touch on the touchscreen display) (e.g., controlled by the user to communicate the user's intended interaction through the user interface by indicating the user interface element to the device with which the user intends to interact).For example, the position of a focus selector (e.g., cursor, touch, or selection box) over a corresponding button while pressure input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen) indicates that the user intends to activate that corresponding button (rather than other user interface elements displayed on the device's display).

[0198] As used herein and in the claims, the term “characteristic intensity” of a contact refers to the characteristics of that contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a set of intensity samples collected over a predetermined period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) associated with a predetermined event (e.g., after detection of contact, before detection of lift-off of contact, before or after detection of the start of movement of contact, before detection of the end of contact, before or after detection of an increase in contact intensity, and / or before or after detection of a decrease in contact intensity). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the top 10 percentile value of the contact intensity, the maximum half value of the contact intensity, the maximum 90 percent value of the contact intensity, etc. In some embodiments, the duration of contact is used when determining characteristic intensity (for example, when characteristic intensity is the average intensity of contact over time). In some embodiments, characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an action has been performed by the user. For example, a set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, contact with a characteristic intensity not exceeding the first threshold results in a first action, contact with a characteristic intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a characteristic intensity exceeding the second threshold results in a third action. In some embodiments, the comparison between characteristic intensity and one or more thresholds is not used to determine whether a first action should be performed or a second action should be performed, but rather to determine whether one or more actions should be performed at all (for example, whether each action should be performed or whether each action should be refrained from).

[0199] Figure 5C shows that multiple intensity sensors 524A-524D detect multiple contacts 552A-552E on a touch-sensitive display screen 504. Figure 5C also includes an intensity diagram showing the current intensity measurements of intensity sensors 524A-524D against intensity units. In this example, the intensity measurements of intensity sensors 524A and 524D are 9 intensity units each, and the intensity measurements of intensity sensors 524B and 524C are 7 intensity units each. In some implementations, the aggregate intensity is the sum of the intensity measurements of the multiple intensity sensors 524A-524D, which in this example is 32 intensity units. In some embodiments, each contact is assigned a portion of the aggregate intensity. Figure 5D shows that the aggregate intensity is assigned to contacts 552A-552E based on their distance from the center of force 554. In this example, contacts 552A, 552B, and 552E are each assigned a contact intensity of 8 intensity units of the aggregate intensity, and contacts 552C and 552D are each assigned a contact intensity of 4 intensity units of the aggregate intensity. More generally, in some implementations, each contact j is assigned an intensity Ij, which is a portion of the aggregate intensity A, according to a predetermined mathematical function Ij = A·(Dj / ΣDi), where Dj is the distance from the center of force to each contact j, and ΣDi is the sum of the distances from the center of force to each of the contacts (e.g., from i=1 to the end). The operations described with reference to Figures 5C-5D can be carried out using electronic devices similar to or identical to devices 100, 300, 500, or 600. In some embodiments, the characteristic intensity of a contact is based on one or more intensities of the contact. In some embodiments, an intensity sensor is used to determine a single characteristic intensity (e.g., a single characteristic intensity of a single contact). Please note that the intensity diagrams are not part of the display user interface, but are included in Figures 5C-5D to assist the reader.

[0200] In some embodiments, a portion of the gesture is identified for the purpose of determining characteristic intensity. For example, a touch-sensitive surface optionally receives a series of swipe contacts transitioning from a start location to an end location, at which point the intensity of the contact increases. In this example, the characteristic intensity of the contact at the end location is optionally based only on a portion of the series of swipe contacts (e.g., only the portion of the swipe contact at the end location) rather than the entire swipe contact. In some embodiments, optionally, a smoothing algorithm is applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some situations, these smoothing algorithms eliminate small increases or decreases in the intensity of the swipe contact for the purpose of determining characteristic intensity.

[0201] The intensity of contact on the touch-sensitive surface is optionally characterized to one or more intensity thresholds, such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to the intensity at which the device performs an action typically associated with clicking a physical mouse button or trackpad. In some embodiments, the deep press intensity threshold corresponds to the intensity at which the device performs an action different from the action typically associated with clicking a physical mouse button or trackpad. In some embodiments, when a contact with a characteristic intensity below the light press intensity threshold (e.g., above the nominal contact detection intensity threshold below which contact is no longer detected) is detected, the device moves the focus selector in accordance with the movement of the contact on the touch-sensitive surface without performing an action associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise specified, these intensity thresholds are consistent across different sets of user interface diagrams.

[0202] An increase in the characteristic intensity of contact from an intensity below a light pressure intensity threshold to an intensity between the light and deep pressure intensity thresholds is sometimes referred to as a "light pressure" input. An increase in the characteristic intensity of contact from an intensity below a deep pressure intensity threshold to an intensity above a deep pressure intensity threshold is sometimes referred to as a "deep pressure" input. An increase in the characteristic intensity of contact from an intensity below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light pressure intensity threshold is sometimes referred to as the detection of contact on the touch surface. A decrease in the characteristic intensity of contact from an intensity above a contact detection intensity threshold to an intensity below a contact detection intensity threshold is sometimes referred to the detection of contact lift-off from the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0203] In some embodiments described herein, one or more actions are performed in response to the detection of a gesture including each pressing input, or in response to the detection of each pressing input performed by each (or more) contact, and each pressing input is detected at least in part on the detection of an increase in the intensity of the contact (or more) above a pressing input intensity threshold. In some embodiments, each action is performed in response to the detection of an increase in the intensity of each contact above a pressing input intensity threshold (e.g., a "downstroke" of each pressing input). In some embodiments, a pressing input includes an increase in the intensity of each contact above a pressing input intensity threshold, followed by a decrease in the intensity of the contact below the pressing input intensity threshold, and each action is performed in response to the detection of a decrease in the intensity of each contact below the pressing input threshold (e.g., an "upstroke" of each pressing input).

[0204] Figures 5E to 5H show the light pressure intensity threshold (for example, "IT") in Figure 5E. L From an intensity below ) to the deep pressing intensity threshold in Figure 5H (for example, "IT DThe device indicates the detection of a gesture including a pressing input corresponding to an increase in the intensity of contact 562 to an intensity exceeding a deep pressing intensity threshold (e.g., "IT"). The gesture performed by contact 562 is detected on the touch-sensitive surface 560, and on the display user interface 570, which includes application icons 572A to 572D displayed within a predetermined area 574, a cursor 576 is displayed over the application icon 572B corresponding to app 2. In some embodiments, the gesture is detected on the touch-sensitive display 504. An intensity sensor detects the intensity of contact on the touch-sensitive surface 560. The device indicates that the intensity of contact 562 exceeds a deep pressing intensity threshold (e.g., "IT"). D It is determined that the pressure has reached a peak (e.g., "IT") above the threshold. Contact 562 is maintained on the touch-sensing surface 560. In response to the detection of a gesture, a deep pressure intensity threshold (e.g., "IT") is set during the gesture. D According to a contact 562 having an intensity exceeding '', a scaled representation 578A-578C (e.g., thumbnail) of the recently opened document is displayed for app 2, as shown in Figures 5F-5H. In some embodiments, this intensity, compared to one or more intensity thresholds, is the characteristic intensity of the contact. Note that the intensity diagram for contact 562 is not part of the display user interface but is included in Figures 5E-5H to assist the reader.

[0205] In some embodiments, the display of expressions 578A to 578C includes animation. For example, as shown in Figure 5F, expression 578A is initially displayed close to the application icon 572B. As the animation progresses, as shown in Figure 5G, expression 578A moves upward and expression 578B is displayed close to the application icon 572B. Then, as shown in Figure 5H, expression 578A moves upward and expression 578B moves upward toward expression 578A and expression 578C is displayed close to the application icon 572B. Expressions 578A to 578C form an array above the icon 572B. In some embodiments, as shown in Figures 5F to 5G, the animation progresses according to the intensity of the contact 562, where the intensity of the contact 562 reaches a deep pressing intensity threshold (e.g., "IT"). DAs it increases toward (), expressions 578A-578C appear and move upward. In some embodiments, the intensity on which the animation progresses is based is the characteristic intensity of the contact. The operations described with reference to Figures 5E-5H can be performed using electronic devices similar to or identical to devices 100, 300, 500, or 600.

[0206] In some embodiments, the device employs intensity hysteresis to avoid accidental inputs, which may be referred to as “jitter,” and the device defines or selects a hysteresis intensity threshold that has a predetermined relationship with a press input intensity threshold (for example, the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of each contact above the press input intensity threshold, followed by a decrease in the intensity of the contact below the hysteresis intensity threshold corresponding to the press input intensity threshold, and each action is performed in response to the detection of a subsequent decrease in the intensity of each contact below the hysteresis intensity threshold (for example, an “upstroke” of each press input). Similarly, in some embodiments, a press input is detected only when the device detects an increase in contact intensity from an intensity below a hysteresis intensity threshold to an intensity above a press input intensity threshold, and optionally a decrease in contact intensity to an intensity below the hysteresis intensity, with each action performed in response to the detection of a press input (e.g., an increase in contact intensity or a decrease in contact intensity in response to a situation).

[0207] For the sake of clarity, the description of an action performed in response to a press input associated with a press input intensity threshold, or a gesture including a press input, is optionally triggered in response to the detection of any of the following: an increase in contact intensity above the press input intensity threshold, an increase in contact intensity from below the hysteresis intensity threshold to above the press input intensity threshold, a decrease in contact intensity below the press input intensity threshold, and / or a decrease in contact intensity below the hysteresis intensity threshold corresponding to the press input intensity threshold. Furthermore, in an example where an action is described to be performed in response to the detection of a decrease in contact intensity below the press input intensity threshold, the action may optionally be performed in response to the detection of a decrease in contact intensity below a hysteresis intensity threshold corresponding to the press input intensity threshold and lower than that threshold.

[0208] Figure 5I shows an exemplary electronic device 580. Device 580 includes a body 580A. In some embodiments, device 580 may include some or all of the features described with respect to devices 100, 300, and 500 (e.g., Figures 1A-5B). In some embodiments, device 580 has one or more speakers 580B (concealed within the body 580A), one or more microphones 580C, one or more touch-sensitive surfaces 580D, and one or more displays 580E. Alternatively, or in addition to the displays and touch-sensitive surfaces 580D, the device may have a touch-sensitive display (also called a touchscreen). As with devices 100, 300, and 500, in some embodiments, the touch-sensitive surface 580D (or touchscreen) optionally includes one or more intensity sensors that detect the intensity of the applied contact (e.g., touch). One or more intensity sensors on the touch-sensitive surface 580D (or touchscreen) may provide output data representing the intensity of the touch. The user interface of device 580 can respond to touches based on touch intensity, meaning that touches of different intensity can invoke different user interface behaviors on device 580. In some embodiments, one or more displays 580E are one or more light-emitting diodes (LEDs). For example, the display may be a single LED, an LED cluster (e.g., red, green, and blue LEDs), multiple individual LEDs, multiple individual LED clusters, or other configurations of one or more LEDs. For example, the display 580E may be an array of nine separate LED clusters configured in a circular shape (e.g., a ring). In some embodiments, one or more displays are composed of any one or more of different types of light-emitting elements.

[0209] Figure 5J shows an exemplary personal electronic device 580. In some embodiments, the device 580 may include some or all of the components described with respect to Figures 1A, 1B, 3, and 5A-5B. The device 580 has a bus 592 that operably connects an I / O section 594 to one or more computer processors 596 and memory 598. The I / O section 594 may be connected to a display 582, which may have a touch-sensitive component 584 and optionally a strength sensor 585 (e.g., a contact strength sensor). In some embodiments, the touch-sensitive component 584 is a component separate from the display 582. In addition, the I / O section 594 may be connected to a communication unit 590 to receive application and operating system data using Wi-Fi, Bluetooth, near-field communication (NFC), cellular, and / or other wireless communication techniques. The device 580 may include an input mechanism 588. In some examples, the input mechanism 588 is optionally a button. In some examples, the input mechanism 588 is optionally a microphone. The input mechanism 588 is optionally multiple microphones (e.g., a microphone array).

[0210] The electronic device 580 includes a speaker 586 for outputting audio. Device 580 may include an audio circuit (e.g., within I / O section 594) that receives audio data, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 586. The speaker 586 converts the electrical signal into human audible sound waves. The audio circuit (e.g., within I / O section 594) also receives the electrical signal converted from the sound waves by a microphone (e.g., input mechanism 588). The audio circuit (e.g., within I / O section 594) converts the electrical signal into audio data. The audio data is optionally retrieved by I / O section 594 from memory 598 and / or RF circuitry (e.g., within communication unit 590) and / or transmitted to memory 598 and / or RF circuitry (e.g., within communication unit 590).

[0211] The memory 598 of the personal electronic device 580 can include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 596, can cause, for example, the computer processor to execute the techniques described below, including processes 1500, 1700, and 1900 (FIGS. 15, 17, and 19). A computer-readable storage media can be any media that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage media is a transitory computer-readable storage media. In some examples, the storage media is a non-transitory computer-readable storage media. Non-transitory computer-readable storage media can include, but is not limited to, magnetic, optical, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical disks based on CD, DVD, or Blu-ray technology, and resident solid-state memories such as flash, solid-state drives, and the like. The personal electronic device 580 is not limited to the components and configurations of FIG. 5J and can include other or additional components in multiple configurations.

[0212] Next, attention is directed to embodiments of a user interface ("UI") and related processes implemented on an electronic device such as the portable multifunctional device 100, device 300, device 500, or device 600.

[0213] FIGS. 6A - 6R illustrate exemplary user interfaces for controlling accessories (e.g., lighting, power tap, thermostat) from an electronic device, according to some embodiments. The user interfaces in those figures are used to illustrate the processes described below, including the processes in FIGS. 7A - 7B.

[0214] Figure 6A shows a home interface 602 on the electronic device 600, which includes affordances for accessories (e.g., controllable external devices such as fan lights, chandeliers, thermostats, speakers, power strips, and power outlets) that are enabled to be controlled by device 600. The home interface 602 includes an affordance 640 for adding a new accessory or scene (e.g., a set of tasks associated with the conditions for a task to run), a state section describing the current state of the selected accessory, a scene section containing affordances corresponding to a set of automated tasks, and a favorites section 604 which includes accessory affordances or tiles (e.g., 604A-604F) corresponding to each accessory (e.g., a fan with lights, a chandelier, a thermostat, speakers, a power strip, and a power outlet) that has been added to device 600 (e.g., added to an application of device 600). The favorites section 604 includes accessory affordances for accessories paired with device 600 during the setup process, which allows device 600 to send commands to configure the accessories. If device 600 receives instructions to include an accessory in its favorites during setup, it will display the accessory affordances (e.g., 604A-604F) in the favorites section 604. The accessory setup process is further described with reference to Figures 10A-10B.

[0215] During setup, device 600 determines whether an accessory is of a first or second type based on information received from the accessory. Based on the type of accessory, device 600 generates a tile corresponding to the accessory. When the tile is tapped, a function is performed if the accessory is of a first type, or a user interface is displayed if the accessory is of a second type. An accessory is of a first type if it has a single function (e.g., can turn a chandelier on or off), or if it supports multiple functions (e.g., a ceiling light with a fan has a fan function and a lighting function) and has a primary function (e.g., a fan light has a designated fan function as the primary function of the accessory). In some embodiments, an accessory (e.g., a chandelier) is of a first type if its primary function supports a binary operating state, and that function causes the accessory to operate in one of two possible states (e.g., on or off). If an accessory has multiple functions (e.g., a thermostat with heating, cooling, and fan functions) but is not associated with a primary function, the accessory is of the second type. In some embodiments, if the primary function associated with the accessory has three or more operating states (e.g., a thermostat has the primary function of setting the temperature, but also has heating, cooling, and fan functions), the accessory is of the second type. Based on the determination of the accessory type, device 600 generates accessory affordances (e.g., 604A-604F) corresponding to the accessory and displays them on the home interface 602. When device 600 detects that an accessory affordance for a first type accessory has been selected, device 600 sends a command to perform a single function or primary function associated with the accessory. For example, tapping the chandelier tile switches the chandelier on and off.If device 600 determines that an accessory has a primary function, it sends a command to the accessory by selecting an accessory affordance, and the accessory performs the primary function without performing any other functions. When device 600 detects that an accessory affordance has been selected for a second type of accessory, device 600 displays an accessory interface that includes control affordances (e.g., 608A-608C in Figure 6F). This is for configuring the accessory to perform one or more of the multiple functions associated with it. For example, tapping a thermostat or speaker tile displays the configuration interface instead of performing a specific function.

[0216] Figures 6A and 6B show that the chandelier lights up when the chandelier accessory affordance 604B is selected. The chandelier accessory affordance 604B is also updated to show the updated state of the chandelier. In Figure 6A, device 600 displays the chandelier accessory affordance 604B corresponding to the chandelier, which is a first type of accessory having a single function of turning the chandelier on / off. The chandelier accessory affordance 604B includes a graphical representation of the chandelier 604B1 indicating that the chandelier is currently off, and a textual current state 604B2 indicating that the chandelier is in the off state. Device 600 detects an input (e.g., a tap) 650A in the home interface 602 corresponding to the selection of the chandelier accessory affordance 604B. In response to detecting input 650A, device 600 sends a command to the chandelier to change the chandelier from its previous off state to the on state.

[0217] As shown in Figure 6B, in response to detecting input 650A, device 600 displays an updated chandelier accessory affordance 604B. This includes an updated graphical representation 604B1 indicating that the chandelier bulbs are on, and an updated current state 604B2 indicating that the chandelier is in the on state. In some embodiments, the graphical representation 604B1 includes an animation showing the state of the accessory. In some embodiments, in response to device 600 detecting a long press (or deep press) on the chandelier accessory affordance 604B, device 600 displays a chandelier interface that includes a control affordance for switching the lighting on and off, as further illustrated in Figure 8B.

[0218] Figures 6C to 6F show that when the fan-lighting tile 604A is selected, an animation is displayed to open the fan-lighting accessory interface 610A for controlling the illuminated ceiling fan. In Figure 6C, device 600 displays the fan-lighting accessory affordance 604A corresponding to an illuminated ceiling fan, which is a second type of accessory having multiple functions (e.g., turning the fan on / off and turning the lights on / off) corresponding to the fan components and the lighting components. The fan-lighting accessory affordance 604A is a grouped accessory affordance that corresponds to both the fan and the lighting. Thus, both the fan and the lighting are configurable via the fan-lighting accessory interface 610A, which is displayed in device 600 when the fan-lighting accessory affordance 604A is selected. The fan-lighting accessory affordance 604A includes a graphical representation 604A1 showing the fan components and the lighting components, and text 604A2 showing the current state of the fan and the lighting, both of which are in the off state. The graphical representation 604A1 includes a representation of each component associated with the accessory. In some embodiments, the graphical representation 604A1 includes representations of each component having a function different from the primary function. In some embodiments, if the primary function is turning a ceiling fan on / off, both the ceiling fan and the lighting are displayed in the graphical representation 604A1. This is because the lighting has a function different from the primary function of turning the fan on / off (e.g., turning the lights on / off). In some embodiments, if the primary function is the same for all components, such as a power strip with power outlets, the graphical representation 604A1 displays only the power strip without displaying individual power outlets. Similarly, the current state 604A2 includes the state of each component associated with an accessory having a distinct function. In some embodiments, the current state includes the state of the corresponding component displayed in the graphical representation 604A1. If an individual component is not displayed in the graphical representation 604A1, the current state 604A2 of that individual component is also not displayed.

[0219] In Figure 6C, device 600 detects input (e.g., tap) 650B in the home interface 602, corresponding to the selection of fan-lighting accessory affordance 604A. In Figure 6C, ceiling fan lighting is a second type of accessory, and the accessory does not have a primary function associated with it. As a result, when fan-lighting accessory affordance 610A is selected, the fan-lighting accessory interface 610A is displayed. In response to the detection of input 650B, device 600 displays the fan-lighting accessory interface 610A by displaying an animation that converts fan-lighting accessory affordance 604A to the fan-lighting accessory interface 610A.

[0220] Figures 6D to 6F show an animation of the conversion of the fan-lighting accessory affordance 604A to the fan-lighting accessory interface 610A. Figure 6D shows that as the size of the fan-lighting accessory affordance 604A increases, the size of the corresponding graphical representation 604A1 and the current state 604A2 also increases proportionally. The fan-lighting accessory affordance 604A expands from its current position. As the fan-lighting accessory affordance 604A expands, device 600 determines whether a portion of the fan-lighting accessory interface 610A would be displayed off-screen if it continued to expand at its current position. As shown in Figure 6E, in accordance with the determination that a portion of the fan-lighting accessory interface 610A would be displayed off-screen, device 600 aligns or quickly moves the fan-lighting accessory interface 610A to the left edge of the display 603 so that a portion of the fan-lighting accessory interface 610A is not displayed off-screen. In some embodiments, based on the determination that a portion of the fan-lighting accessory interface 610 is not displayed off-screen, device 600 displays the fan-lighting accessory interface 610A at its current position without aligning the fan-lighting accessory affordance 604A to the left or right boundary of the display 603.

[0221] Figure 6F shows the fan-lighting accessory affordance 610A, which has controls for changing additional characteristics of the ceiling fan and lighting (e.g., lighting color). Figure 6F shows the fan-lighting accessory affordance 610A fully enlarged, which includes a proportionally enlarged graphical representation 610A1 of the fan and lighting, as well as larger text indicating the current state of the fan and lighting 610A2. The graphical representation 610A1 of the fan-lighting accessory 610A corresponds to the graphical representation 604A1 of the fan-lighting accessory affordance 604A. Similarly, the current state 610A1 of the fan-lighting accessory interface 610A corresponds to the textual state 604A2 of the fan-lighting accessory affordance 604A. The fan-lighting accessory interface 610A includes a control affordance 608A for adjusting the fan speed and a control affordance 608B for adjusting the brightness of the lighting. The fan-lighting accessory interface 610A also includes a color selection control 608C for changing the color of the lighting. As will be further described with respect to Figures 8A to 8J, the control affordances 608A to 608C are displayed in a first area of ​​the fan-lighting accessory interface 610A, while the setting section 620 is displayed in a second area of ​​the fan-lighting accessory interface 610A. The fan-lighting accessory interface 610A further includes a setting shortcut affordance 612. The setting shortcut affordance 612 is displayed in the lower right corner of the displayed or visible portion of the first area of ​​the fan-lighting accessory interface 610. In response to device 600 detecting that the setting shortcut affordance 612 has been selected, device 600 quickly moves the display to the setting section 620 at the top of the display 603 and scrolls the remaining controls to the first area. In some embodiments, the setting shortcut affordance 612 is displayed below all controls in the first area of ​​the fan-lighting accessory interface 610A.

[0222] Figure 6F shows an exemplary layout of controls on the accessory interface 610A. Device 600 determines the layout of controls on the accessory interface based on the number of control and sensor affordances included in the accessory, as well as the hierarchy of controls (e.g., 608A-608C). When the accessory interface 610A is opened, the primary controls corresponding to the primary functions (e.g., 608A-608B) are displayed. Sub-controls not associated with primary functions (e.g., 608C) are displayed below the primary controls. Sub-controls may not be displayed initially if they are located within the overflow area of ​​the accessory interface 610A. As described below, Figures 8B-8C further illustrate the arrangement of controls in the first area of ​​the accessory interface (e.g., 810B).

[0223] Device 600 determines the layout of control affordances 608A to 608C within the fan-lighting accessory interface 610A based on the number of control affordances and sensor affordances displayed. Device 600 determines that the accessory's function includes two functions corresponding to the fan and lighting components. If Device 600 detects that the ceiling fan and lighting accessory has two or fewer control affordances corresponding to the main functions of each accessory (such as turning the fan on / off and turning the lights on / off), Device 600 displays control 608A corresponding to the fan function and control 608B corresponding to the lighting function side by side within the fan-lighting accessory interface 610A, as shown in Figure 6F. If there are three or more control affordances for the accessory, Device 600 displays the control affordances 808A to 808C in a vertical arrangement, as shown in the chandelier accessory interface 810B in Figure 8B. In some embodiments, the fan-lighting accessory interface includes additional control affordances that can be displayed in an overflow section that appears when the accessory interface is scrolled down, as further described and illustrated with reference to Figures 8B–8D.

[0224] In some embodiments, the ceiling fan and lighting accessory has a primary fan function that turns the fan on / off, among several functions supported by the ceiling fan and lighting accessory. Because the ceiling fan and lighting accessory has a primary function, device 600 determines that the ceiling fan and lighting accessory is a first type of accessory. Therefore, in response to device 600 receiving an input (e.g., a tap) corresponding to selecting the fan-lighting accessory 604A, device 600 sends a command to perform the primary fan function (e.g., turn on the fan) instead of displaying the fan-lighting accessory interface 610A. Device 600 does not send any commands to perform any other secondary functions of the ceiling fan and lighting accessory (e.g., the lights are not turned on). In some embodiments, the primary function cannot be set to a secondary function (e.g., the primary function cannot be changed to turn the lights on / off). In some embodiments, the user can reassign the primary function to another function (e.g., a secondary function), for example, so that selecting the fan-lighting accessory affordance results in device 600 sending a command to turn on the lights. Since ceiling fans and lighting accessories are first-type accessories, device 600 displays the fan-lighting interface 610A in response to the accessory affordance 610A being pressed and held (or pressed deeply).

[0225] Figure 6F also illustrates changing the fan speed and lighting settings via controls within the fan-lighting interface 610A. In Figure 6F, device 600 detects input 650C corresponding to the selection of control affordance 608A, which corresponds to setting the fan to 50% of its maximum speed. In Figure 6G, in response to the detection of input 650C, device 600 updates the graphical representation 610A1 to indicate that the fan is on, and updates the current state 610A2 to reflect that the fan is set to 50% speed and the lighting remains off. In some embodiments, in response to the detection of input 650C, device 600 updates the graphical representation 610A1 within the fan-lighting accessory interface 610A to include an animation of a rotating fan indicating that the fan is on.

[0226] In Figure 6G, device 600 detects input (e.g., a swipe) 650D1 corresponding to the selection of control 608B, which corresponds to turning the light on to 75% of its maximum brightness. In Figure 6H, in response to the detection of input 650D1, device 600 updates the graphical representation 610A1 to indicate that the light is on and updates the current state 610A2 to indicate that the light is on to 75% brightness, while the fan is set to 50% speed. Also in Figure 6G, device 600 detects input 650D2 (e.g., a tap), which corresponds to changing the color of the light bulb and selecting a third color. In response to the detection of input 650D1, and as shown in Figure 6H, device 600 updates the graphical representation 610A1 to include an image of a colored light bulb that is on and matches the selected light color, as will be further explained with respect to Figures 8C-8D.

[0227] Figure 6H illustrates how the user can access the settings section 620 by tapping a settings shortcut affordance 612, instead of scrolling to the bottom of the screen to access the settings section 620. In Figure 6H, device 600 detects an input (e.g., a tap) 650E corresponding to the selection of a settings shortcut affordance 612. In Figure 6I, in response to the detection of input 650E, device 600 displays the settings section 620 corresponding to the ceiling lighting and fan settings on the fan-lighting accessory interface 610. In some embodiments, as further described and illustrated with reference to Figures 8A-8J, device 600 displays the settings section 620 in response to the detection of an input that results in device 600 scrolling the fan-lighting accessory interface 610A to a second area (e.g., an overflow area), which is below the first area displaying controls 608A-608D. The configuration section 620 includes a graphical representation 610A1 and a current state 610A2 that shows the current state of the ceiling fan and lighting accessories. The configuration section 620 also includes a group section that includes the “group with other accessories” affordance 614A and the accessory affordance 614B. The “group with other accessories” affordance 614A corresponds to grouping the ceiling fan in the bedroom with another accessory of the same type (for example, another fan in the living room).

[0228] Figures 6I–6K illustrate the separation of a fan-lighting accessory group into two separate accessories, each having distinct accessory affordances (e.g., 614G and 614H) and distinct accessory interfaces (e.g., 610G and 610H). The fan-lighting accessory group is separated from the accessory settings from the fan-lighting accessory interface 610A by selecting “Display as separate tile affordances” 616, as shown in Figure 6J. The fan and lighting each retain their respective corresponding settings after the accessory group has been separated into two distinct accessory affordances. In Figure 6I, device 600 detects input (e.g., tap) 650F, corresponding to the selection of accessory affordance 614B. In Figure 6J, in response to the detection of input 650F, device 600 displays accessory interface 622, which includes the respective affordances 618A–618B for each component in the group (e.g., fan and lighting), and the separation affordance 616 for separating the lighting from the fan.

[0229] In Figure 6J, device 600 detects an input (e.g., a tap) 650G corresponding to the selection of the separation affordance 616. In Figure 6K, in response to the detection of input 650G, device 600 displays the home interface 602 with separate accessory affordances, the fan accessory affordance 604G and the lighting accessory affordance 604H, while stopping the display of the fan-lighting accessory affordance 604A because grouped accessories are separated. The fan accessory affordance 604G includes a graphical representation of the fan 604G1, which includes an animation of the rotating fan and a current state 604G2 indicating that the fan is on and operating at 50%. Similarly, the accessory affordance 604H includes a graphical representation of the lighting 604H1, which includes an on-state light bulb and a current state 604H2 indicating that the lighting is on and operating at 75%. Lighting accessory affordance 604H corresponds only to the lighting components of ceiling fans and lighting accessories. A lighting accessory is a first type of accessory that has a single function for turning lights on / off that operate in a binary state. Therefore, when device 600 receives an input (e.g., tap) that corresponds to the selection of lighting accessory affordance 604H, it sends a command to turn the lights on / off.

[0230] Figures 6L-6R show that after an accessory group is divided, each component of the accessory group can be configured individually via its corresponding accessory affordance. For example, lighting from a ceiling fan with lighting can be configured via lighting accessory affordance 604H. When selected, lighting accessory affordance 604H is converted to lighting accessory interface 610H, which displays the control of the lighting. Similarly, when lighting accessory interface 610H is closed, lighting accessory interface 610H is converted back to lighting accessory affordance 604H.

[0231] In Figure 6L, device 600 detects input 650H corresponding to a long press (or deep press) of the lighting accessory affordance 604H. In Figures 6M to 6O, in response to the detection of input 650H, device 600 displays an animation in which the lighting accessory affordance 604H is converted to the lighting accessory interface 610H. Figure 6M shows that as the size of the lighting accessory affordance 604H expands, the size of the corresponding graphical representation 604H1 and the characters corresponding to the current state 604H2 also expand proportionally. Device 600 displays the animation of the expanding lighting accessory affordance 604H based on the current position of the lighting accessory affordance 604H, which is centered on the home interface 602. As shown in Figure 6N, according to the determination that no part of the lighting accessory interface 610H would be displayed off-screen if displayed at the current position, device 600 displays the lighting accessory interface 610 at the current position without aligning the lighting accessory affordance to the left or right boundary of the screen.

[0232] Figure 60 shows a fully enlarged lighting accessory interface 610H, located at the center of a display device 603 where the lighting accessory affordance 604H is positioned. This interface includes a proportionally enlarged graphical representation 610H1 of the lighting and larger characters indicating the current state of the lighting 610H2. The lighting accessory interface 610H includes a control affordance 608A for adjusting the brightness of the lighting and a color control affordance 608C corresponding to the color of the lighting. Device 600 displays additional control affordances associated with the lighting below the color control affordance 608C in the overflow area of ​​the lighting accessory interface 610H, as further described with respect to Figure 8C.

[0233] In Figure 6O, device 600 detects input 650I (e.g., a swipe) corresponding to the selection of control affordance 608A to adjust the brightness to 90%. In Figure 6O, in response to the detection of input 650I, device 600 updates the graphical representation 610H1 to indicate that the lighting is on and updates the current state 610H2 to indicate that the lighting is set to 90% brightness. In some embodiments, in response to the detection of input 650I, device 600 updates the graphical representation 610A1 in the lighting accessory interface 610H to include animation.

[0234] In Figure 6O, device 600 detects input (e.g., tap) 650J corresponding to the selection of the "×" affordance 630 and closes the fan accessory interface 610H. As shown in Figures 6P-6R, in response to the detection of input 650J, device 600 displays an animation of the lighting accessory interface 610H transitioning to and returning to the lighting accessory affordance 604H in the home interface 602. Figures 6P-6Q show that as the size of the lighting accessory interface 610H decreases, the corresponding graphical representation 60GA1 and the characters corresponding to the current state 604H2 also decrease in size proportionally. As the size of the lighting accessory interface 610H decreases, device 600 stops displaying the control affordance 608A and sensor affordances 608C-608D in the lighting accessory interface 610H. In Figure 6R, device 600 displays the lighting accessory affordance 604H on the home interface 602 with an updated graphical representation 604H1 and an updated current state 604H2 in which the lighting is set to 90% brightness. In some embodiments, as shown in Figure 6L, the graphical representation 604H1 and the current state 604H2 remain the same as before the lighting brightness changed.

[0235] Figures 7A-7B are flowcharts illustrating a method 700 for controlling an external device using an electronic device, according to several embodiments. The method 700 is performed on a device (e.g., 100, 300, 500, or 600) equipped with a display device (e.g., 603). Some operations of the method 700 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0236] As described below, Method 700 provides an intuitive way to manage controllable external devices from a home interface (e.g., 602) that provides access to affordances (e.g., 604A-604H) corresponding to controllable external devices. Enabling users to manage controllable external devices from the home interface reduces the cognitive burden on the user because all controllable external devices are accessible from the home interface. Thus, users do not need to open separate applications to manage the configuration of each controllable external device, thereby creating a more efficient human-machine interface. In the case of battery-powered computing devices, managing controllable external devices from a unified home interface reduces the number of inputs required to manage each controllable external device, allowing users to efficiently conserve power on the device and extend the battery charging interval.

[0237] In block 702, a device (e.g., 600) (e.g., a smartphone, a smart home control device) having a display device (e.g., 603) (e.g., a touch-sensitive display device) and a communication device (e.g., a wireless antenna) displays a user interface (e.g., 602) (e.g., a homepage showing tiles for each accessory) on the display device, which includes first accessory affordances (e.g., 604A-604G) (e.g., selectable graphical tiles) associated with a first controllable external device (e.g., a device having at least one controllable function (e.g., a smart light bulb, a smart thermostat, a smart speaker) (e.g., a homepage showing tiles for each accessory).

[0238] In some embodiments, the first accessory affordance (e.g., 604A) includes a first graphical element (e.g., 604A1) (e.g., an icon representing a first controllable external device (e.g., an image of a light bulb)) and a first textual element (e.g., 604A2) (e.g., a name representing the first controllable external device including its current state (e.g., the light bulb is "on") (e.g., "light bulb")). In some embodiments, according to the determination that the first controllable external device is associated with a second controllable component (e.g., the fan of a ceiling fan with lighting), the second controllable component associated with the first function (e.g., turning on the fan) includes a graphical representation (e.g., 604A, 604G) of the second controllable component (e.g., a tile including the fan icon). In some embodiments, according to the determination that a first controllable external device is associated with a third controllable component (e.g., lighting in a ceiling fan with lights), the third controllable component associated with a second function (e.g., turning on the lights) includes a graphical representation of the third controllable component (e.g., 604A, 604H). In some embodiments, according to the determination that a first controllable device is associated with a second and a third controllable component, the graphical representation of the second and third controllable components is included (e.g., tiles include icons or images of each subcomponent in an accessory such as a fan and lighting, or a power strip with several power outlets) (e.g., 604A, 604E). Displaying a graphical representation (e.g., 604A1) and a textual status (e.g., 604A2) of the components (e.g., fan and lighting) within a first accessory affordance (e.g., 604A) provides the user with visual information about which controllable external devices and components are part of an accessory group (if there are multiple components) and the current state of the accessory. By allowing users to see which controllable external devices are controlled by accessory affordances, the cognitive burden is reduced.Therefore, the user does not need to provide additional inputs (e.g., long presses) to manage the configuration of each controllable external device and its corresponding components, thereby creating a more efficient human-machine interface. In the case of battery-powered computing devices, reducing the number of inputs required to manage each controllable external device allows the user to efficiently conserve power and extend the battery charging interval. Furthermore, the user can visually confirm whether the selected function has been executed or the settings have been configured.

[0239] In some embodiments, according to the determination that a first controllable external device is associated with a second controllable component and a third controllable component, the first accessory affordance includes a graphical representation of the second controllable component and a graphical representation of the third controllable component (e.g., 604A).

[0240] In some embodiments, in block 704, a first controllable external device (e.g., a fan with lighting) is associated with a first function (e.g., a primary or main function such as turning on the fan) and a second function (e.g., a secondary function such as turning on the lighting).

[0241] In block 706, the device (e.g., 600) detects a first input (e.g., a tap input) (e.g., 650A, 650B) corresponding to the selection of a first accessory affordance (e.g., 604A). In some embodiments, in block 708, the first accessory affordance has a first appearance before the first input is detected.

[0242] In block 710, in response to the detection of a first input (e.g., 650A), and in block 714, in accordance with the determination that a set of command transmission criteria has been met (e.g., the accessory has a single primary function), the device (e.g., 600) sends a command to a first controllable external device via a communication device to perform a first function (e.g., turn on the device's lighting if it is currently off, or turn it off if it is currently on). In some embodiments, in block 712, in response to the detection of the first input, and in conjunction with sending a command to the first controllable external device to perform the first function (e.g., simultaneously), the first accessory affordance (e.g., 604B) is updated to a second appearance (e.g., 604B1 in Figure 6B) based on the first function (e.g., the first function is to change the state of the lighting which is part of the first controllable external device, and the second appearance includes an image of the lighting in a state that matches the changed lighting state). By updating the first accessory affordance to a second appearance that reflects the updated state of the accessory, the user is provided with visual feedback that a command has been executed. The cognitive burden is reduced by allowing the user to see the updated controllable external devices through the accessory affordance. Therefore, the user does not need to provide additional inputs (e.g., long presses) to manage the configuration of each controllable external device and its corresponding components, thereby creating a more efficient human-machine interface. For battery-powered computing devices, reducing the number of inputs required to manage each controllable external device allows the user to efficiently conserve power and extend the battery charging interval. Furthermore, the user can visually confirm whether the selected function has been executed or whether the settings have been configured.

[0243] In some embodiments, in block 716, the set of command transmission criteria includes a first criterion that is satisfied when a first function associated with a first controllable external device (e.g., a primary function such as turning on a fan) is the primary function of the first controllable external device (e.g., an accessory having multiple functions such as a fan with lighting) (e.g., a designated primary function if the accessory supports multiple functions, the only function associated with the device). In some embodiments, the set of command transmission criteria includes a second criterion that is satisfied when a first function associated with a first controllable external device (e.g., the accessory has a single function with a binary state, i.e., it can only be turned on or off) is associated with a first controllable component (e.g., the fan of a ceiling fan with lighting) having a binary operating state (e.g., off or on, not three or more states). In some embodiments, the first function is a transition from the current operating state of the first controllable component of the first controllable external device to another operating state of the first controllable component of the first controllable external device (e.g., when the component is on, the function is off, and vice versa). In some embodiments, the fan of the illuminated ceiling fan is turned on or off by tapping the accessory tile.

[0244] In some embodiments, block 718 transmits a command to the first controllable external device via the communication device to perform a first function, without transmitting a command to the first controllable external device via the communication device, in accordance with the determination that a set of command transmission criteria (e.g., whether an accessory is associated with a primary function) is satisfied (e.g., the accessory has a single primary function). In response to the detection of a first input (e.g., 650A) and in connection with transmitting a command to the first controllable external device to perform a first function (e.g., simultaneously), the first accessory affordance (e.g., 604B) is updated to a second appearance (e.g., 604B1) based on the first function (e.g., the first function is to change the state of a light that is part of the first controllable external device, and the second appearance includes an image of the light in a state that matches the changed light state). In some embodiments, the second appearance includes an animation (e.g., a repeating / looping animation) based on the first function (e.g., a fan is turned on, and the tile animation shows the fan rotating). A more efficient user interface is provided without displaying additional accessory interfaces by having the device send a command to a first controllable external device to perform a first function when the transmission criteria are met. Since the first controllable external device is mapped to the primary function, the primary function is performed without requiring additional user interaction. This reduces the cognitive burden on the user and conserves battery power because the primary function is performed by the first controllable external device with a single input.

[0245] In block 720, in accordance with the determination that a set of command transmission criteria is not met (for example, an accessory has multiple primary functions, such as a thermostat that can change the temperature and switch between on and off), a first accessory interface (e.g., 610A in Figure 6F, 1210C in Figure 12B) is displayed on the display device (e.g., without sending a command to a controllable external device). The first accessory interface in block 722 includes a first control affordance (e.g., 608A-608C in Figure 6F, 1208 in Figure 12B) (e.g., a dimmer for lighting, a temperature dial for a thermostat) which, when selected, sends a command to a controllable external device to perform a first function, and a second control affordance (e.g., a change in the color of the lighting, a change in the mode of a thermostat such as AC, heater, fan, etc.) which, when selected, sends a command to an external accessory device to perform a second function different from the first function. A more efficient user interface is provided to the user by displaying the accessory interface, which allows the user to manage the settings of the first controllable external device when the device is not mapped to a primary function. Displaying the accessory interface because the controllable external device is not mapped to a primary function provides a more efficient user interface, as it allows the user to manage the control of the controllable external device. This saves battery power because the accessory interface is not displayed every time the corresponding accessory affordance is selected.

[0246] In some embodiments, the first accessory interface (e.g., 610A) includes a first control affordance (e.g., 608A) and a second control affordance (608B, 608C). In some embodiments, displaying the first accessory interface includes displaying the first control affordance (e.g., 608A) and the second control affordance (e.g., 608B) side by side (e.g., in a horizontal arrangement) according to a first and second function that satisfy a first set of layout criteria (e.g., the first set of layout criteria includes criteria that are satisfied when the first and second functions are classified at the same level within the function hierarchy). In some embodiments, the first control affordance (e.g., 608A) is displayed above the second control affordance (e.g., 608C) in an upper-lower arrangement (e.g., in a vertical arrangement) according to a second set of layout criteria that satisfy a second set of layout criteria (e.g., the first set of layout criteria includes criteria that are satisfied when the first and second functions are classified at different levels within the function hierarchy).

[0247] In some embodiments, the first accessory interface (e.g., 610A1) includes a second graphical element (610A1) corresponding to (e.g., matching) a first graphical element (e.g., 604A1), and a second text element (e.g., 610B2) corresponding to a first text element (e.g., 604A2). In some embodiments, displaying the first accessory interface further includes displaying an animation of the first graphical element visually transitioning to the second graphical element (e.g., an animation of the first graphical element expanding to become the second graphical element), and displaying an animation of the first text element visually transitioning to the second text element.

[0248] In some embodiments, the first accessory interface is displayed at a first position (e.g., 610A in Figure 6E) where it is aligned with the boundary, based on the determination that a portion of the first accessory interface (e.g., 610A in Figure 6E) is displayed at a position that extends beyond the boundary of the display device (e.g., a portion of the accessory's user interface) (e.g., a card associated with the accessory is displayed outside the screen boundary of the device) (e.g., the first accessory interface is displayed at a first position (e.g., 610A in Figure 6E) where it is aligned with the boundary (e.g., the accessory's user interface is quickly moved to align with the screen boundary or edge). In some embodiments, the first accessory interface is displayed at a second position (e.g., 610A in Figure 6N) based on the position of the first accessory affordance (e.g., 604H), based on the determination that a portion of the first accessory interface (e.g., 610A in Figure 6N) is not displayed at a position that extends beyond the boundary of the display device (e.g., the user interface is within the screen boundary). By allowing the device to determine the appropriate layout of each accessory interface based on the number of controls and sensors supported by the controllable external device, a more intuitive user interface is achieved, reducing the cognitive burden on the user. The layout of controls and sensors helps reduce the number of inputs required to configure the controllable external device, thereby saving battery power.

[0249] In some embodiments, while the first accessory interface (e.g., 610H in Figure 6O) is being displayed (in some embodiments, while the first accessory affordance is not being displayed), a second input (e.g., 650J) is detected that corresponds to a request to stop displaying the first accessory interface. In response to the detection of the second input, stopping the display of the first accessory interface (in some embodiments, in addition to redisplaying the first accessory affordance) includes displaying an animation of a second graphical element (e.g., 610H1) that visually transitions to a first graphical element (e.g., 604H1) (e.g., an animation of the second graphical element shrinking to become the first graphical element) and displaying an animation of a second character element (e.g., 610H2) that visually transitions to a first character element (e.g., 604H2).

[0250] In some embodiments, the first controllable external device is associated with a second controllable component (e.g., the fan of a ceiling fan with lighting) and a third controllable component (e.g., the lighting of a ceiling fan with lighting), and while displaying the first accessory interface (e.g., 610A) (e.g., displaying the card for the ceiling fan with lighting), it detects a first set of one or more inputs (e.g., 650F-650G in Figures 6I-6J and 1050J-1050I in Figures 10K-10L) (e.g., a request to redisplay the user interface), including inputs (e.g., 616 in Figure 6J, 1020 and 1022 in Figure 10L) corresponding to control reconfiguration affordances (e.g., for removing subordinate components from a group) (e.g., for ungrouping the fan and lighting into two separate tiles). In response to detecting a first set of one or more inputs, a second instance of the user interface (e.g., 602 in Figure 6K) is displayed, which includes a second accessory affordance (e.g., 604G) (e.g., an affordance that, when selected, performs a function associated with the second controllable component) (e.g., a tile corresponding to a fan) associated with a second controllable component of the first controllable external device and not associated with a third controllable component of the first controllable external device, and a third accessory affordance (e.g., 604H) (e.g., an affordance that, when selected, performs a function associated with the third controllable component) (e.g., a tile corresponding to lighting) associated with a third controllable component of the first controllable external device and not associated with the second controllable component of the first controllable device. In some embodiments, the second instance of the user interface does not include the first accessory affordance (e.g., 604A). In some embodiments, the grouped ceiling fan tile with lighting is no longer displayed, and instead, two separate tiles (e.g., a fan tile and a lighting tile) are displayed.Updating accessory affordances in the home interface based on whether accessories are grouped together or separated provides users with a more intuitive user interface for managing controllable external devices. Grouped accessories are represented by a single accessory affordance, reducing the user's cognitive burden. Therefore, by selecting a single accessory affordance, users can manage all controls for the accessories in one place, rather than having to access the controls for each component within the group or the controllable external device separately. This also saves power because it reduces the number of inputs required to configure the grouped accessories. Similarly, if grouped accessories are separated, they are displayed as individual accessory affordances, providing users with a more intuitive user interface. By allowing users to choose to separate the components of a controllable external device and providing separate user interfaces for configuring each component, the components the user is interacting with become clearer, reducing the user's cognitive burden.

[0251] It should be noted that the process details described above with respect to Method 700 (for example, Figures 7A-7B) are also applicable in a similar manner to the methods described above / below. For example, Method 700 optionally includes one or more features of the various methods described above with respect to Methods 900, 1100, and 1300. For example, the controllable external device described in Method 900 can be the controllable external device of Method 700 and can be configured and accessed using the interface described in Method 700. For brevity, these details will not be repeated below.

[0252] Figures 8A to 8J show exemplary user interfaces for configuring accessories (e.g., lighting, power strips, thermostats) according to several embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes in Figures 9A and 9B.

[0253] Figures 8A and 8B illustrate how the control of the chandelier can be accessed by long-pressing the chandelier accessory affordance 604A. In Figure 8A, device 600 displays the chandelier accessory affordance 604A corresponding to the chandelier. The chandelier accessory affordance 604A includes a graphical representation of the chandelier 604A1 indicating that it is currently on, and a textual current state 604A2 indicating that the chandelier is on. Device 600 detects input 850A corresponding to a long press (or deep press) of the chandelier accessory affordance 604A in the home interface 602. In Figure 8B, device 600 displays the chandelier accessory interface 810B in response to the detection of input 650A corresponding to a long press of the chandelier accessory affordance 604B. In some embodiments, if the accessory is a second type of accessory that does not have a primary function, device 600 displays an accessory interface (e.g., chandelier accessory interface 810B) in response to device 600 detecting an input (e.g., a tap) corresponding to the selection of an accessory affordance.

[0254] Figures 8B–8D show the layout of controls within the chandelier accessory interface 810B, consisting of a first area 812A that is displayed when the chandelier accessory interface 810B is first displayed. The second area 812B (overflow area) contains accessory settings. The settings section 822 is displayed when the user scrolls beyond the first area 812A. The user can scroll between areas using a swipe gesture. In Figures 8B–8D, device 600 is displaying the chandelier accessory interface 810B, which includes the displayed first area 812A and the undisplayed second area 812B. When device 600 displays the chandelier accessory interface 810B in response to detected input 850A, the first portion of the first area 812A is displayed on display 603. The first region 812A includes a graphical representation 810B1, the current state 810B2, and control affordances 808A-808C. The first region 812A includes an overflow region that is not initially displayed. The overflow region of the first region 812A includes additional control affordances and sensors, such as affordance 808C, which are not initially displayed in the first region. Device 600 displays the overflow region of the first region 812A in response to detecting an input corresponding to scrolling down within the first region 812A.

[0255] The control affordance 808A corresponds to the control for changing the brightness of the chandelier. In Figure 8B, the chandelier is set to 90% brightness. The control affordance 808B corresponds to the various different colors that can be set for the chandelier. In Figure 8B, six different colors are initially displayed in the first part of the first area 812A of the chandelier accessory interface 810B. Additional colors are displayed in the overflow area of ​​the first area 812A and remain visible until the chandelier accessory interface 810B is scrolled. In Figure 8B, the chandelier is set to color 808B1. The graphical representation 810B1 reflects the selected lighting color indicated by the control 808B.

[0256] In Figure 8B, device 600 detects input 850B corresponding to a swipe gesture to scroll down the chandelier accessory interface 810B. In Figure 8C, in response to detecting input 850B, device 600 displays the overflow region of the first region 812A while maintaining the display of control affordances 808A and 808B. The overflow region of the first region 812A includes a third row of chandelier colors for control 808B and a timer control affordance 808C that specifies the time limit until the chandelier is automatically turned off. While displaying the overflow region of the first region 812A, device 600 maintains the display of the graphical representation 810B1 and the current state 810B2. The display of control affordance 808A becomes partial as the first region 812A is scrolled to reveal control affordance 808C. As the user scrolls through the chandelier accessory interface 810B and scrolls in to view the first area 812, the settings shortcut affordance 816 appears at the bottom of the screen. This provides the user with quick access to the settings section 822. Thus, when device 600 scrolls through the first area 812A of the chandelier accessory affordance 810B, device 600 maintains the display of the settings shortcut affordance 816 in the lower corner of the first area of ​​the chandelier accessory affordance 810B. The settings shortcut affordance 816 is not displayed in the second area of ​​the chandelier accessory affordance 810B. Since the settings shortcut affordance 816 provides quick access to the settings section 822 in the second area 812B of the chandelier accessory interface 810B, the user does not need to scroll through the first area 812A to reach the settings section.

[0257] Figures 8C and 8D show that when a setting is changed within the chandelier accessory interface 810B, the graphical representation 810B1 and the textual state 810B2 are updated. In Figure 8C, device 600 detects an input (e.g., a tap) 850C corresponding to a second color selection 808B2. In Figure 8D, in response to the detection of input 850C, device 600 updates the display of the control affordance 808A and the graphical representation 810B1 to reflect the selected lighting color. Device 600 determines that the lower part of the first area 816 of the chandelier accessory interface 810B has been displayed. In response to this determination, device 600 displays the setting shortcut affordance 816. In response to detecting the selection of the setting shortcut affordance, device 600 displays the second area of ​​the chandelier accessory interface 810B. In some embodiments, the setting shortcut affordance 816 is displayed at the bottom of the first area 812A. In some embodiments, the setting shortcut affordance 816 is displayed at the bottom of the display 603, regardless of which part of the first area 812A is being displayed.

[0258] Figures 8D-8F illustrate access to a second region 812B by using a swipe gesture within the chandelier accessory interface 810B. In Figure 8D, device 600 detects input 850D corresponding to a swipe gesture to scroll further down the chandelier accessory interface 810B. In Figure 8E, in response to detecting input 850D, device 600 displays a portion of the second region 812B of the chandelier accessory interface 810B while the user's finger is still in contact with the display 603. Device 600 determines whether the displayed portion of the second region 812B exceeds a threshold amount 824. In Figure 8E, following the determination that the displayed portion of the second region 812B does not exceed the threshold amount 824, device 600 displays a bounce animation (e.g., a rubber band effect) when the user lifts their finger from the swipe gesture. As a result of the bounce animation, Figure 8D shows that device 600 quickly returns to the previously displayed portion of the first region 812A of the chandelier accessory interface 810B without displaying the second region 812B. Figures 8F to 8G show that, following the determination that the portion of the second region 812B is displayed above a threshold amount 824 (as a result of the quick swipe gesture), the screen of device 600 quickly moves to the second region 812B of the chandelier accessory interface 810B, displaying the second region 812B at the top of display 603.

[0259] Figure 8G shows the second area 812B (configuration section 822) of the chandelier accessory interface 810B. The second area 812B includes the same graphical representation 810B1 and current state 810B2 as those displayed in the first area 812A. The second area 812B includes a configuration section 822 for configuring the chandelier settings. In response to the device 600 screen quickly moving to display the second area 812B of the chandelier accessory interface 810B, the device 600 outputs haptic and audio outputs 820 indicating the transition from the display of the first area 812A to the second area 812B. In some embodiments, the device 600 outputs a haptic output without an audio output. In some embodiments, the device 600 outputs an audio output without a haptic output.

[0260] In Figure 8G, device 500 detects input (e.g., tap) 850E corresponding to the selection of the "×" affordance 830 and closes the chandelier accessory interface 810B. In Figure 8H, in response to detecting input 850E, device 600 displays an updated chandelier accessory affordance 604A, which includes a graphical representation 604A1 indicating that the lighting color has changed to color 808B2, and a current state 604A2 indicating that the lighting is on and at 90% brightness. In some embodiments, in response to detecting input 850E, device 600 outputs a tactile output and an audio output when the chandelier accessory interface 810B is closed. In some embodiments, device 600 outputs a tactile output without an audio output. In some embodiments, device 600 outputs an audio output without a tactile output.

[0261] Figures 8H to 8J show several layouts of the controls within the fan interface 810I. As described above with respect to Figure 6F, device 600 determines the layout depending on the number of controls and sensors associated with the accessory. In Figure 8H, device 600 detects input 850F, which corresponds to a long press (or deep press) of the living room fan accessory affordance 604I. In response to detecting input 850F, device 600 displays the living room fan accessory interface 610I. Figures 8I to 8J show two layouts of the living room fan accessory interface 610I. Device 600 determines the layout of the control and sensor affordances within the fan accessory interface 610I based on the number of control and sensor affordances displayed, as generally described with respect to Figure 6F. In Figure 8I, device 600 determines whether the living room fan has two or fewer sensor affordances (e.g., 816A to 816B). In accordance with the determination that the fan has two sensor affordances 808A to 816B, device 600 displays the layout within the fan accessory interface 610I where the sensor affordances 816A to 816B are displayed above the control 818. In Figure 8J, device 600 determines that the living room fan has three sensor affordances 816A to 816C. In accordance with the determination that the fan has three or more sensor affordances, device 600 displays the layout within the fan accessory interface 610I where the sensor affordances 816A to 816C are displayed below the control 818.

[0262] Figures 9A-9B are flowcharts illustrating a method 900 for controlling an external device using an electronic device, according to several embodiments. The method 900 is implemented in a device (e.g., 100, 300, 500, or 600) equipped with a display device (e.g., 603). Some operations of the method 900 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0263] As described below, Method 900 provides an intuitive way to manage controllable external devices from a home interface (e.g., 602) that provides access to affordances corresponding to controllable external devices. By enabling users to manage controllable external devices within the home from the home interface, all controllable external devices become accessible from the home interface, thus reducing the cognitive burden on the user. Consequently, users do not need to open separate applications to manage the configuration of each controllable external device, thereby creating a more efficient human-machine interface. In the case of battery-powered computing devices, enabling users to manage controllable external devices from the home interface reduces the number of inputs required to open multiple applications, allowing users to efficiently conserve power on the device and extend the battery charging interval.

[0264] In block 902, the device (e.g., 600) displays on the display device a user interface (e.g., a homepage showing tiles for each accessory) that includes a first accessory affordance (e.g., 604B) (e.g., selectable graphical tiles) associated with a first controllable external device (e.g., a device having at least one controllable function (e.g., a smart light bulb, a smart thermostat, a smart speaker)).

[0265] In some embodiments, in block 904, the first accessory interface (e.g., 810B) includes a first total number of control affordances (e.g., 808A-808C), all of which are contained in a first area (e.g., 812A) (e.g., all accessory controls are displayed in the first area), while the second area (e.g., 812B) contains no control affordances from the first total number (e.g., no controls are displayed in the settings section displayed in the second area). In some embodiments, the first area of ​​the first accessory interface includes a second area affordance (e.g., 816) (e.g., a settings shortcut) at a third position in the first area (e.g., adjacent to the displayed portion of the first area).

[0266] In block 906, the device (e.g., 600) detects a first input (e.g., 850A) (e.g., a tap on the affordance) corresponding to the selection of a first accessory affordance.

[0267] In block 908, in response to detecting a first input (e.g., 850A) (e.g., a tap), the device (e.g., 600) displays on the display device (e.g., 603) a first area (e.g., 812A) of the first accessory interface (e.g., 810B) (a user interface that displays all controls of an accessory, such as a card associated with the accessory), which includes a first control affordance (e.g., 808A) that, when selected, controls a first function of a first controllable external device (e.g., dimming the lights, changing the color of the lights) (e.g., sending a control signal), without displaying a second area (e.g., 812B) of the first accessory interface (e.g., a settings section). In some embodiments, the first area includes all controls of the accessory, and the second area includes a settings section for the accessory (e.g., 816).

[0268] In some embodiments, in block 910, according to the determination that the first controllable external device is associated with (e.g., including) fewer than a predetermined number of sensors (e.g., temperature, humidity, and / or ambient light sensors) (e.g., three sensors), the first region (812A) of the first accessory interface (e.g., 810I) includes, in a first location (e.g., above the first control affordance) a set of control affordances (e.g., 816A-816C) corresponding to the sensors of the first controllable device.

[0269] In some embodiments, in block 912, according to the determination that the first controllable external device is associated with a predetermined number of sensors (e.g., including), the first region (e.g., 812A) of the first accessory interface (e.g., 810I) includes a set of control affordances (e.g., 816A-816C) corresponding to the sensors of the first controllable device in a second location (e.g., under the first control affordance (e.g., 818)) of the first region (e.g., 812A), which is different from the first location.

[0270] In block 914, while a first area (e.g., 812A) of a first accessory interface (e.g., 810B) is displayed on a display device (e.g., 603), the device detects a second input (e.g., 850B, 850D) (e.g., a swipe). In some embodiments, the second input includes a swipe (e.g., 850B) that scrolls the display to show additional controls on lower cards that are not currently displayed. In some embodiments, a swipe (e.g., 850D) quickly moves the display to a section setting, thereby quickly moving the display to a settings section (e.g., 822), so that controls are no longer displayed.

[0271] In blocks 916 and 918, in response to detecting a second input (e.g., 850D), and in accordance with the determination that the second input (e.g., 850D) causes a portion of the second area (e.g., 812B) to be displayed beyond a threshold amount (e.g., a threshold amount for the content or area of ​​the second area) (e.g., a quarter of the configuration section is displayed on the screen), in block 920, the device (e.g., 600) displays the second area above the first accessory interface (e.g., 810B) (e.g., access The display of the first accessory interface (e.g., 810B) is updated to quickly navigate to the settings section of Sari, while the display of the first area (e.g., 812A) is stopped (e.g., controls in the first area are no longer displayed), and the second area contains the settings for one or more of the first controllable external devices (e.g., the settings are the main settings page for the accessory (e.g., the accessory's settings, including sharing data including favorites, receiving notifications, adding people to access lists) that allows the user to manage the accessory's setup). In some embodiments, in block 922, the device outputs haptic and audio feedback (e.g., 820). Displaying controls and settings in the same accessory interface provides a more efficient user interface for accessing the controls of controllable external devices. By allowing the user to access the controls and settings of controllable external devices from a single interface, the cognitive burden on the user is reduced. Thus, the user does not need to open a separate application to manage the configuration of each controllable external device. Furthermore, since the settings of controllable external devices are generally used less frequently than other controls, they are hidden unless the user intentionally tries to access the settings. Therefore, by using a threshold to determine when to display settings, a more efficient user interface is provided, as settings sections are only displayed when the user intentionally wants to access them. Furthermore, by floating setting shortcut affordances, users can navigate to settings sections without having to scroll to the bottom of the accessories interface.

[0272] In block 924, in response to the detection of a second input, and according to the determination that the second input does not cause a portion of the second area (e.g., 812B) to be displayed beyond a threshold amount (e.g., less than a quarter of the settings section is displayed), the device (e.g., 600) updates the display of the first accessory interface (e.g., 810B) based on the second input (e.g., 850D) (e.g., gradually scrolling the second control (e.g., 808C) through the first area (e.g., 812A) to a page that was not previously displayed, corresponding to the speed of the swipe rather than quickly moving to the settings section), to the second control affordance (e.g., 808B, 808C) in the first area of ​​the first accessory interface (e.g., the second control was not previously displayed) (e.g., changing the color of the lighting, the second control was not displayed before the second input). In some embodiments, scrolling down the page to display the second control hides the second area (e.g., 812B) of the first accessory interface (e.g., 810B), and the first control disappears from view. By not displaying the configuration section when the threshold is not exceeded, a more efficient user interface is provided. Instead, the focus is directed to controlling the controllable external device, reducing the cognitive burden on the user.

[0273] In some embodiments, the device (e.g., 600) detects a fourth input (e.g., 650E in Figure 6H) corresponding to a second area affordance (e.g., 816). In response to detecting the fourth input corresponding to the second area affordance, the device updates the display of the first accessory interface (e.g., 610A) (e.g., by scrolling through other controls within the overflow area of ​​the first area) to display the second area (e.g., 620 in Figure 6H) at the top of the first accessory interface (e.g., quickly moving to the accessory settings section) while the display of the first area is stopped.

[0274] In some embodiments, while a first accessory interface (e.g., 810B) is displayed on a display device (e.g., 603), the device detects a third input (e.g., 830) (e.g., the third input cancels or closes the first accessory interface). In response to detecting the third input, the device stops displaying the first accessory interface and outputs tactile and audio feedback (e.g., 820) indicating that the first accessory interface is no longer displayed (e.g., canceling the feedback to indicate that the card is closed).

[0275] It should be noted that the process details described above with respect to Method 900 (for example, Figures 9A and 9B) are also applicable in a similar manner to the methods described above / below. For example, Method 900 optionally includes one or more characteristics of the various methods described above with respect to Methods 700, 1100, and 1300. For example, the accessory interface of Method 900 is used to configure a controllable external device of Method 700. For brevity, these details will not be repeated below.

[0276] Figures 10A to 10AC show exemplary interfaces for grouped accessories (e.g., power taps) according to several embodiments. The interfaces in these figures are used to illustrate processes described later, including the processes shown in Figures 11A to 11B.

[0277] In Figures 10A-10C, a power strip and a power outlet are added to the home interface 602. The user can add accessories by scanning a code or by entering a code. Once an accessory is added, accessory affordances (e.g., 604E and 604F) are added to the home interface 602. The user can manage the accessory configuration by tapping the accessory affordance (e.g., 604E) corresponding to the accessory (e.g., a power strip). In Figure 10A, device 600 displays the home interface 602, including an affordance 640 for adding a new accessory or scene. Device 600 detects a set of inputs 1050A (e.g., a tap) corresponding to the selection of affordance 640 and selects it to add a new accessory. In Figure 10B, in response to detecting input 1050A, device 600 displays the "Add Accessory" interface 1002 for adding a new accessory that will be controllable by device 600. The accessory addition interface 1002 includes a viewfinder 1004A and instructions for scanning a code (e.g., a barcode, QR code®, or a series of numbers) associated with a new accessory using device 600. The accessory addition interface 1002 also includes affordances 1004B for manually entering a code to add a new accessory.

[0278] In Figure 10C, after successfully adding a new accessory (for example, in response to it), device 600 displays the home interface 602 containing the accessory affordances corresponding to the added accessory. Figure 10C shows the newly added power strip accessory affordance 604F and entrance power outlet accessory affordance 604F in the home interface 602, corresponding to the power strip accessory and the power outlet accessory, respectively. Based on received information about the functions that the accessory can perform, device 600 determines whether it has a corresponding accessory affordance that performs a function when the accessory interface is tapped or opened, allowing the user to manage the accessory. For example, the power outlet accessory affordance 604F causes the power outlet to switch between on and off when the power outlet accessory affordance 604F is tapped. In contrast, the thermostat accessory affordance 604C causes device 600 to open the thermostat interface 1210C in Figure 12B when the thermostat accessory affordance 604C is tapped. During setup, device 600 determines, based on information received from the power outlet, that the power outlet is a first type of accessory because it has a single function that operates in two binary states, switching between on and off. During setup, device 600 determines, based on information received from the power strip, that the power strip is a grouped accessory, also a first type of accessory, because it has a single function that operates in two binary states, switching between on and off. In contrast, as will be discussed later with respect to Figures 12A and 12B, the thermostat is a second type of accessory because the thermostat has no primary function.

[0279] In Figure 10C, device 600 detects input 1050B corresponding to a long press (or deep press) of the power outlet accessory affordance 604F in the home interface 602. In Figure 10D, in response to detecting input 1050B, device 600 displays the power outlet accessory interface 1010F. The power outlet accessory interface 1010F includes a graphical representation 1010F1 and a current state 1010F2 indicating that the power outlet is currently off. The power outlet accessory interface 1010F includes a control affordance 1008 corresponding to switching the power outlet between the on and off states.

[0280] In Figure 10D, device 600 detects input 1050C, corresponding to a swipe gesture that scrolls down the power outlet accessory interface 1010F to access the power outlet configuration section 1018. Since the power outlet is not part of an accessory group, the power outlet configuration section 1018 does not contain any affordances for reconfiguring the power outlet. In Figure 10E, in response to detecting input 1050C, device 600 quickly moves the display of the power outlet accessory interface 1010F to the power outlet configuration section 1018. Since the power outlet is a single accessory component, the power outlet configuration section 1018 does not contain any accessory reconfiguration affordances for reconfiguring the accessory components. For example, the power outlet configuration section 1018 does not include affordance 1014B as shown in Figure 10K, but is associated with adding or removing components to a grouped accessory or set of affordances 1014C as shown in Figure 10R, and is associated with recombining accessories that are physically part of the group (for example, a power outlet is a component of a power strip).

[0281] Figure 10F shows the newly added power tap accessory affordance 604E within the home interface 602. The power tap accessory affordance 604E corresponds to a power tap accessory that includes six individual power outlet components (e.g., sub-components). The power tap affordance 604E represents an accessory group that includes six power outlets. The power tap has a primary function of being turned on / off. The six individual power outlet components of the power tap also have the same primary function of being switched between on and off. Therefore, device 600 generates a single power tap accessory affordance 604E for the power tap and the accessory group that includes the six individual power outlet components. The accessory group can be turned on or off together, or its individual components can be controlled individually. Therefore, in response to device 600 detecting an input (e.g., a tap) corresponding to the selection of the power tap accessory affordance 604E, device 600 sends a command to the power tap to turn the entire accessory group on / off. Power strips and individual power outlets can be individually switched between on and off states by control affordances associated with each power outlet component within the power strip accessory interface 1010E.

[0282] In Figure 10F, device 600 detects input 1050D, which corresponds to a long press (or deep press) of the power tap accessory affordance 604E, and accesses the individual controls corresponding to each power outlet. In Figure 10G, in response to detecting input 1050D, device 600 displays the power tap accessory interface 1010E. The power tap accessory interface 1010E includes controls 1020A to 1020G for managing the state of each power outlet and the power tap as a whole. The power tap accessory interface 1010E includes a graphical representation 1010E1 indicating that the power tap is off, and a current state 1010E2 indicating that all power outlets in the power tap are currently off. The power outlet accessory interface 1010E includes control affordances 1020A to 1020G. Control affordance 1020A sets the power tap, including all six power outlet components, to the ON state if the power tap is currently OFF, and sets the power tap to the OFF state if the power tap is currently ON. Power outlets that are already ON when control affordance 1020A is activated to turn the power tap ON remain ON. Similarly, power outlets that are already OFF when control affordance 1020A is selected to turn the power tap OFF remain OFF. Control affordances 1020B to 1020G correspond to the control affordance corresponding to each individual power outlet in the group, and the control affordance can be set to ON or OFF without affecting the state of other power outlets in the group. The power tap accessory interface 1010E also includes the setting shortcut affordance 1012.

[0283] Figures 10G to 10J show how to turn on individual power outlets in a power tap or power tap accessory group. The power tap accessory affordance 604E of the home interface 602 is updated each time the power tap status is updated.

[0284] In Figure 10G, device 600 detects input (e.g., tap) 1050E, which corresponds to the selection of control affordance 1020A for turning on the power tap. In response to detecting input 1050E, device 600 sends a command or instruction to the power tap accessory to turn on each of the six power outlet components. In Figure 10H, after detecting input 1050E, device 600 updates the display of power tap accessory affordance 604E in the home interface 602. The graphic representation 604E1 of power tap accessory affordance 604E is updated to indicate that the power tap has been turned on, as indicated by a lightning bolt. The current state 604E2 of power tap accessory affordance 604E is updated to indicate that the power tap is in the ON state.

[0285] In Figure 10H, device 600 detects input 1050F, which corresponds to a long press (or deep press) of the power tap accessory affordance 604E. In Figure 10I, in response to detecting input 1050F, device 600 displays the power tap accessory interface 610E. The power tap accessory interface 610E includes a graphical representation 610E1 that matches the updated graphical representation 604E1 in Figure 10H, which indicates that the power tap is ON. The current state 1010E2 is updated to indicate that the power tap is ON and that each of the six power outlets is ON. In response to device 600's determination that the power tap is currently ON, device 600 updates the control affordance 1020A for turning off the power tap. The control affordances 1020B-1020G indicate that each of the power outlets is ON, and tapping one of the control affordances 1020B-1020G turns off the corresponding power outlet.

[0286] In Figure 10I, device 600 detects input (e.g., tap) 1050G corresponding to the selection of control affordance 1020B corresponding to power outlet 1. In Figure 10J, following (and in response to) the detection of input 1050G, device 600 sends a command to the power tap accessory to turn off power outlet 1. Device 600 updates the display of control affordance 1020B to indicate that the power outlet is now off. Device 600 can then detect subsequent taps of control affordance 1020B to turn on power outlet 1. Control affordances 1020C~020G indicate that each of the corresponding power outlets remains on, and tapping the corresponding control affordance will turn off the corresponding power outlet.

[0287] A power strip accessory group can be reconfigured via the configuration section 1018E to remove power outlets, add power outlets, or split the group. Figures 10K-10O illustrate the process of removing a power outlet from a power strip accessory group. In Figure 10J, device 600 detects input (e.g., tap) 1050H corresponding to the selection of configuration shortcut affordance 1012. In Figure 10K, in response to detecting input 1050H, device 600 displays the power strip configuration section 1018E. The power strip configuration section 1018E includes graphical representations 1010E1 and state 1010E2 corresponding to the state of the power outlet accessory, as shown in the power outlet accessory interface 1010E. The power outlet configuration section 1018 includes a “group with other accessories” affordance 1014A for grouping a power outlet with other accessories of the same type, and an accessory affordance 1014B for reconfiguring a power strip accessory (e.g., by removing components from or adding components to a power strip). Accessory affordance 1014B indicates that there are currently six power outlet components in the group.

[0288] In Figure 10K, device 600 detects input (e.g., tap) 1050I corresponding to the selection of accessory affordance 1014B. In Figure 10L, in response to detecting input 1050I, device 600 displays accessory interface 1060. Accessory interface 1060 includes six sets 1018A to 1018F of accessory affordances corresponding to each power outlet component in the power tap accessory group. Accessory interface 1060 includes an "add or remove" affordance 1020 for adding a power outlet that has been removed from the power tap or removing a power outlet from the power tap, and a group dismantling affordance 1022 for dismantling the group by creating a separate accessory affordance for each component in the group (e.g., a power outlet accessory affordance for each of the six power outlets).

[0289] In Figure 10L, device 600 detects input 1050J, which corresponds to the selection of the "Add or Remove" affordance 1020. In Figure 10M, in response to detecting input 1050J, device 600 updates the display of the Add or Remove interface 1060 to enable the selection or deselection of each power outlet accessory affordance 1018A~1018F. Device 600 detects input (e.g., tap) 1050K, which corresponds to the selection of the power outlet accessory affordance 1018C, and removes power outlet 3 from the power tap group. In Figure 10N, in response to detecting input 1050K, device 600 displays a check mark next to the selected power outlet accessory affordance 1018C. In Figure 10N, device 600 detects input (e.g., tap) 1050L, which corresponds to the selection of the Exit button 1032. After detecting input 1050L, device 600 removes power outlet 3 from the power tap accessory group.

[0290] In Figure 10O, in response to detecting input 1050L, device 600 displays the power tap accessory interface 1010E, which has only five control affordances corresponding to power outlets 1020B, 1020C, 1020E, 1020F, and 1020G. Control affordance 1020D is not displayed because the corresponding power outlet 3 has been removed. Adding or removing power outlets from the power tap accessory group does not change the state of the power outlets still in the group (e.g., power outlets 1-2 and 4-6) or the state of the power outlet removed from the group (e.g., power outlet 3). Therefore, as shown in Figure 10J, power outlet 1 remains in the off state and power outlet 3 remains in the on state. This is the same state the corresponding power outlets were in before outlet 3 was removed from the power tap accessory group. The graphical representation 1010E1 does not change, but the current state 1010E2 is updated to show that four power outlets in the power tap accessory group are on. In Figure 10O, device 600 detects input 1050M, which corresponds to the selection of the × button 1030.

[0291] When power outlet 3 is removed from the power tap group, device 600 displays power outlet 3 along with its corresponding power outlet 3 accessory affordance 1004J. In Figure 10P, in response to detecting input 1050M, device 600 displays the new power outlet 3 accessory affordance 1004J added to the home interface 602. The power outlet 3 accessory affordance 1004J includes a graphical representation 1004J1 indicating that the power outlet is on, as indicated by a lightning bolt, and a current state 1004J2 indicating that the power outlet is on. The power tap accessory affordance 604E does not change within the home interface 602.

[0292] In Figure 10P, device 600 detects input 1050N, which corresponds to a long press (or deep press) of the power outlet 3 accessory affordance 1004J. Since device 600 determines that power outlet 3 is a first type of accessory with a single function, the tap detected by affordance 1004J causes device 600 to send a command to turn power outlet 3 on / off, instead of displaying the power outlet 3 accessory interface 1010J. In Figure 10Q, in response to detecting input 1050N, device 600 displays the power outlet 3 accessory interface 1010J. The power outlet 3 accessory interface 1010J includes a graphical representation 1010J1 and a current state 1010J2 indicating that power outlet 3 is in the ON state. The power outlet 3 accessory interface 1010J includes a single control affordance 1008 for turning power outlet 3 on / off.

[0293] In Figure 10Q, device 600 detects input (e.g., tap) 1050O, which corresponds to the selection of the setting shortcut affordance 1012. In Figure 10R, in response to detecting input 1050O, device 600 displays the power outlet 3 setting section 1018J within the power outlet 3 accessory affordance 1010J. The setting section 1010J includes a regrouping affordance 1014C for regrouping power outlet 3 into the power tap accessory to which it was originally grouped. The setting section 1010J also includes an accessory removal affordance 1016 for removing the accessory from the home interface 602, thereby disassociating it from device 600.

[0294] Figures 10R to 10T show that when an accessory component, such as a power outlet on a power strip, is removed, notification 1062 prompts the user to confirm whether to proceed. If the user proceeds with removing the component, the accessory is also removed along with the accessory affordance in the home interface 602. In Figure 10R, device 600 detects input (e.g., power strip) 1050P, which corresponds to the selection of accessory removal affordance 1016. In Figure 10S, in response to detecting input 1050P, device 600 displays a notification indicating that removing power outlet 3 will disassociate the power strip and the other five power outlet components from device 600. Since power outlet 3 is a physical part of the power strip, removing power outlet 3 removes the entire power strip accessory that includes power outlet 3. In Figure 10S, device 600 detects input 1050Q, which corresponds to the removal of power outlet 3.

[0295] In Figure 10T, after (and in response to) the removal of power outlet 3, device 600 updates the display on home interface 602. Since the power tap and power outlet accessories are no longer associated with device 600, the power outlet 3 accessory affordance 1004J and the power tap accessory affordance 604E are removed from home interface 602.

[0296] Figures 10U to 10X show that power outlet 3 is regrouped with a power tap via the regrouping affordance 1014C selected from the power outlet 3 setting section 1018J of the power outlet 3 accessory interface 1010J. In Figure 10U, device 600 displays the home interface 602, including the power tap accessory affordance 602E and the power outlet 3 accessory affordance 1004J, before the power outlet 3 accessory affordance 1004J is removed. Device 600 detects input 1050R, which corresponds to a long press (or deep press) of the power outlet 3 accessory affordance 1004J. In Figure 10V, in response to detecting input 1050R, device 600 displays the power outlet 3 accessory interface 1010J, as described above with respect to Figure 10Q. Device 600 detects input (e.g., tap) 1050S, which corresponds to the selection of the setting shortcut affordance 1012.

[0297] In Figure 10W, in response to detecting input 1050S, device 600 displays the power outlet 3 setting section 1018J as described above with respect to Figure 10R. Device 600 detects input (e.g., tap) 1050T, which corresponds to the selection of regrouping affordance 1014C, reconnecting power outlet 3 to the power tap accessory to which it was originally grouped.

[0298] In Figure 10X, following (for example, in response to) the detection of input 1050T, device 600 displays the power strip accessory interface 1010E2 indicating that power outlet 3 has been re-added to the power accessory group. The power tap accessory interface 1010E2 displays six control affordances corresponding to each of the six power outlets, including power outlet 3 which has just been re-added to the power tap accessory group. Each of the six control affordances indicates that the corresponding power outlet will remain in the same state as it was before power outlet 3 was re-added to the group. For example, power outlet 1 was previously in the off state and remains in the off state. Similarly, power outlet 3 was in the on state before being removed from the group and remains in the on state after being re-added to the group. In Figure 10X, device 600 detects input (e.g., tap) 1040U corresponding to the selection of the setting shortcut affordance 1012. In some embodiments, once power outlet 3 is re-coupled to the power tap, the power outlet 3 accessory affordance 1004J is no longer displayed on the home interface 602.

[0299] Figures 10Y to 10AC show the coupling of power outlet 3 with the outlet of the power strip in the living room. As a result, the power outlet group accessory affordance 1004L is generated in the home interface 602, and the power outlet 3 accessory affordance 1004J and the living room power strip accessory affordance 1004K are no longer displayed. In Figure 10Y, before power outlet 3 is recoupled with the power strip, device 600 displays the power outlet 3 accessory interface 1010J. Device 600 detects the input (e.g., tap) 1050U corresponding to the selection of the “Group with other accessories” affordance 1014A and adds power outlet 3 to a custom accessory group that includes other outlets and power strip accessories.

[0300] In Figure 10Z, in response to detecting input 1050U, device 600 displays a new group interface 1064. Device 600 displays power outlet and power tap affordances 1034A~1034D corresponding to household power outlets and power tap accessories that can be selected to form a new group within the new group interface 1064. For example, bedroom power outlet 3 and power tap accessory affordances 1034A~1034B are displayed under bedroom. Entrance power outlet accessory affordance 1034C is displayed under entrance. Living room power tap accessory affordance 1034D is displayed under living room. In Figure 10Z, device 600 detects inputs (e.g., taps) 1050V~1050W corresponding to the selection of power outlet 3 accessory affordance 1034C and living room power tap accessory affordance 1034D, and identifies the accessories to be grouped. In Figure 10AA, in response to detecting an input of 1050V~1050W, device 600 displays check marks next to the selected accessory power outlet 3 accessory affordance 1034C and power tap accessory affordance 1034D. Device 600 detects an input (e.g., a tap) 1050X corresponding to the selection of the exit button 1032. In some embodiments, a user can group various accessories of different types (e.g., thermostats, lights, and power taps) to generate shortcut access to the group (e.g., a shortcut to a group of tasks triggered by a voice command) which is triggered by an event or voice command (e.g., turn on the accessories in the group when someone is home).

[0301] In Figure 10AB, in response to detecting input 1050X, device 600 displays a new power outlet group accessory interface 1010L, which includes control affordances 1040A to 1040D for the living room power tap power outlets and power outlet 3, which are currently in the new power outlet accessory group. The power outlet group accessory interface 1010L includes a graphical representation 1010L1 indicating that the group contains power outlet accessories, and a current state 1010L2 indicating that there is one power outlet in the accessory group that is turned on. Control affordance 1040A turns on all power outlet accessories in the group. Control affordances 1040B and 1040C correspond to the control to switch between the on / off states of power outlets 1 and 2 associated with the living room power tap. Control affordance 1040D corresponds to the control to switch between the on / off states of power outlet 3. Device 600 detects input (e.g., tap) 1050Y, which corresponds to the selection of the "×" button 1030.

[0302] In Figure 10AC, in response to detecting input 1050Z, device 600 displays a new power outlet group accessory affordance 1004L corresponding to the new power outlet accessory group. Device 600 removes power outlet 3 accessory affordance 1004J and living room power tap accessory affordance 1004K and replaces them with the new power outlet group accessory affordance 1004L.

[0303] Figures 11A-11B are flowcharts illustrating method 1100 for controlling a group of external devices using an electronic device, according to several embodiments. Method 1100 is implemented in devices having a display device (e.g., 100, 300, 500, 600). Some operations of method 1100 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0304] As described later, Method 1100 provides an intuitive method for managing a controllable external device with multiple components. This method reduces the cognitive burden on the user to manage a controllable external device with multiple components, thereby creating a more efficient human-machine interface. By enabling the user to manage a controllable external device with multiple components from the same device interface, the control of the group can be managed from one place, thus reducing the cognitive burden on the user. Therefore, the user does not need to open separate applications to manage the configuration of each controllable external device and its sub-components, thereby creating a more efficient human-machine interface. In the case of battery-powered computing devices, enabling the user to manage controllable external devices and components from the same device interface reduces the number of inputs required to configure the components of the controllable external device, allowing the user to efficiently conserve power in the device and extend the battery charging interval.

[0305] In block 1102, a device (e.g., 600) (e.g., a smartphone, a smart home control device) equipped with a display device (e.g., a touch-sensitive display device) is not configured to control the first controllable external device, while the device detects a first set of one or more inputs (e.g., 1050A) corresponding to a request to configure an electronic device to control the first controllable external device.

[0306] In block 1104, in response to detecting a first set of one or more inputs (e.g., 1050A), the device initiates a process to configure an electronic device to control a first controllable external device, which includes in block 1106 receiving information corresponding to the first controllable external device (e.g., from an accessory device, from a server that stores the characteristics of the device).

[0307] In block 1108, according to a determination based on received information that a first controllable external device is associated with multiple controllable functions (e.g., turning on / off multiple power outlets on a power strip, turning on / off lights, turning on / off fans) associated with multiple subordinate components of the first controllable external device (e.g., power outlets on a power strip, lighting on a ceiling fan), the device (e.g., 600) displays on the display device (e.g., 603) a first accessory affordance (e.g., 604E) (e.g., a tile representing the group) corresponding to a group including the first controllable external device and the multiple subordinate components.

[0308] In block 1110, according to a determination based on received information that the first controllable external device is not associated with multiple sub-components (for example, any multiple sub-components either do not contain an accessory or contain only one sub-component), the device displays on the display device a second accessory affordance (e.g., 1004J) corresponding to the first controllable external device (e.g., a tile representing only the accessory). By displaying grouped accessory affordances for the controllable external device and its components, instead of individual accessory affordances for each component in the group, a more efficient user interface is provided, reducing the cognitive burden on the user. By selecting grouped accessory affordances, a single entry point is provided for accessing all controls in the accessory group. This reduces the number of inputs required to configure the controllable external device and its components, thereby saving battery power.

[0309] In block 1112, while displaying a first or second accessory affordance on a display device (e.g., 603), the device (e.g., 600) detects a second input (e.g., 1050B, 1050D) (e.g., a contact maintained for a predetermined period of time).

[0310] In block 1114, in response to detecting a second input (e.g., 1050D), in block 1116, according to the determination that the second input corresponds to the selection of a first accessory affordance (e.g., 604E) (e.g., grouped accessory tiles), the device (e.g., 600) displays to the display device (e.g., 603) a first accessory interface (e.g., 1010E) (e.g., accessory configuration page (e.g., card)) which includes a first controllable external device and a plurality of control affordances (e.g., 1020A-1020G) (e.g., controlling the turning accessory on / off, changing the color of the lighting, changing the temperature), and a control reconfiguration affordance (e.g., 1020) which, when selected, initiates a process of removing at least a first control affordance (e.g., e.g., 1020D) from the plurality of control affordances (e.g., ungrouping a subordinate component from a group).

[0311] In some embodiments, the first accessory interface includes a group control affordance (e.g., 1020A) which, when selected, sends a command to the first controllable external device that affects the state of a sub-component among a plurality of sub-components (e.g., affects / changes the state of each of the plurality of sub-components (e.g., all sub-components of the first controllable external device)). In some embodiments, the first accessory interface (e.g., 100E) includes an accessory removal affordance (e.g., 1016). In some embodiments, the plurality of control affordances of the first accessory interface include a first control affordance (e.g., 1020B) associated with a first sub-component of a plurality of sub-components (e.g., a first power outlet of a multi-power outlet power strip) and a second control affordance (e.g., 1020D) associated with a second sub-component of a plurality of sub-components (e.g., a second power outlet of a multi-power outlet power strip). In some embodiments, a plurality of control affordances (e.g., 1020A to 1020G) include one or more control affordances corresponding to one or more control affordances corresponding to a first type of subcomponent (e.g., a lighting subcomponent, a power outlet subcomponent, a fan subcomponent) and a second type of subcomponent.

[0312] In block 1118, according to the determination that the second input (e.g., 1050B) corresponds to the selection of a second accessory affordance (e.g., 604F), the device (e.g., 600) displays to the display device (e.g., 603) a second accessory interface (e.g., 1010F) (e.g., accessory configuration page (e.g., card)) which includes control of the first controllable external device (e.g., 1008), without displaying the control reconfiguration affordance (e.g., 1020).

[0313] In some embodiments, a device (e.g., 600) detects a third input (e.g., a tap gesture) corresponding to the selection of a first control affordance (e.g., 1020A). In response to detecting the third input, a command is sent to a first controllable external device that affects the state of multiple sub-components (e.g., turning all power outlets on / off on a power strip). Controlling an accessory group (e.g., a power strip and power outlets) using a single control affordance (e.g., 1020A) provides a more efficient user interface that reduces the cognitive burden on the user. For example, a power strip contains six power outlets. Instead of switching each power outlet on / off, the user can switch all power outlets in the group by tapping a single control affordance. This not only simplifies the control of the power strip but also reduces the number of inputs, thus helping to conserve battery power in electronic devices.

[0314] In some embodiments, while displaying a first accessory interface (e.g., 1010E), the device (e.g., 600) detects a fourth input (e.g., 1050G) (e.g., a tap gesture) corresponding to a first control affordance (e.g., 1020B). In response to detecting the fourth input, the device sends a command to the first controllable external device that affects the state of the first sub-component (e.g., toggling the state of the first power outlet between off and on) without sending a command that affects the state of the second sub-component (e.g., the second power outlet of the power tap remains unaffected). A more efficient user interface is provided that reduces the cognitive burden on the user by displaying an aggregated accessory interface (e.g., 1010E) that allows the user to manage the configuration of each component of a grouped accessory. The user can control a specific component without affecting the state of other components in the group. Thus, the management of accessory groups is simplified because control of the group is accessible from a single user interface. This not only simplifies the management of components within accessory groups but also reduces the number of inputs, thus helping to conserve battery power in electronic devices.

[0315] In some embodiments, the device detects a fifth input (e.g., 1050P) corresponding to an accessory removal affordance (e.g., 1016) (e.g., removing an accessory so that it is no longer controlled by the electronic device). In response to detecting the fifth input corresponding to the accessory removal affordance, the device (e.g., 600) initiates a process including the following alert criteria, which configures the electronic device (e.g., 600) to no longer control the first controllable external device. In accordance with the determination that a set of alert criteria is met, the alert criteria include a criterion that is met if the first controllable external device is a sub-component device of a composite external device that includes at least one other sub-component device that the electronic device is currently configured to control (e.g., display a notification only if the accessory to be removed is a sub-component that is part of a controllable accessory (e.g., removal of a power outlet that is part of a power strip)), and in accordance with the determination that a set of alert criteria is met, completing the process of configuring the electronic device so that it no longer controls the first controllable external device also means displaying (e.g., 1062) that the electronic device is configured so that it no longer controls at least one other sub-component device (e.g., a notification that removing a power outlet means removing the entire power strip from the control of the electronic device). In accordance with the determination that a set of alert criteria is not met (e.g., the accessory to be removed is a standalone accessory such as a light or thermostat), the device refrains from displaying (e.g., removing the accessory without displaying a notification) that completing the process of configuring the electronic device so that it no longer controls the first controllable external device also means configuring the electronic device so that it no longer controls at least one other sub-component device (e.g., removing the accessory without displaying a notification). Displaying a notification that warns the user that removing a component of an accessory group will remove the entire accessory (for example, removing a power outlet will remove the power strip) provides a more efficient user experience and reduces the user's cognitive burden.In many cases, users are unaware that removing a component from a group will remove the entire accessory. Therefore, a notification requesting user confirmation before removing a component saves users time in re-adding accessories they unintentionally removed.

[0316] In some embodiments, while displaying a first accessory interface (e.g., 610A in Figure 6J, 1010E in Figure 10I) (e.g., an accessory user interface or card for an accessory that includes multiple control affordances associated with sub-components), the device (e.g., 600) selects a specific sub-component (e.g., 1018C) to ungroup, a second set of one or more inputs (e.g., 650G in Figure 6J, 1050I to 1050L in Figures 10K to 10N) (e.g., in some examples, the selection of reconfiguration affordances (e.g., 616 in Figure 6J, 1020 and 1022 in Figure 10L) and inputs corresponding to control reconfiguration affordances (e.g., grouped accessory affordances, and the addition or deletion of affordances) The device detects a click of an “Exit” button (e.g., 1032) including (e.g., 650G in Figure 6J, 1050J in Figure 10L). In response to the detection of a second set of one or more inputs, the device stops displaying the first accessory interface (e.g., 610A) and stops displaying the first accessory affordance (e.g., 604A) (e.g., ungroups). In some embodiments, only the selected sub-components (e.g., power outlets) (e.g., 1018C) are removed from the group (e.g., power tap) and have their own tile (e.g., 1050N), while the remaining accessories of the sub-components (e.g., the remaining power outlets in the power tap) remain in the group within the grouped tile (e.g., 604E).

[0317] The device displays a third accessory affordance (e.g., 604G, 1004J) which, when selected, displays a third accessory interface (e.g., 1010J, 810I in Figures 10V-10Y) that includes one or more control affordances (e.g., 1008) corresponding to a first type of sub-component (e.g., fan, power outlet). In some embodiments, a new grouped tile (e.g., 1004L) displays all sub-components of the same type, including a new group of lighting components, such as lighting for a ceiling fan with lighting, separated from the previous group. In some embodiments, the third accessory interface includes a regrouping affordance (e.g., 1014C).

[0318] The device displays a fourth accessory affordance (e.g., 604H) which, when selected, displays a fourth accessory interface (e.g., 610H) containing one or more control affordances (e.g., 608B-608C) corresponding to sub-components of a second type (e.g., lighting). In some embodiments, selecting a control reconfiguration affordance (e.g., 616) provides the option to ungroup all the control affordances (e.g., 608A-608C) that were grouped together in the first accessory interface (e.g., 610A) and create new groups of control affordances (e.g., a new group for lighting: 614A, a new group for power outl...

Claims

1. It is a method, In an electronic device equipped with a display device and a communication device, Displaying a user interface on the display device that includes a first accessory affordance associated with a first controllable external device, To detect a first input corresponding to the selection of the first accessory affordance, In response to the detection of the first input, In accordance with the determination that the set of command transmission criteria is met, a command for performing the first function is transmitted to the first controllable external device via the communication device. The process includes, in accordance with the determination that the set of command transmission criteria is not met, displaying a first accessory interface on the display device, wherein the first accessory interface is A first control affordance, which, when selected, transmits a command to the controllable external device to perform the first function; A method comprising: a second control affordance, which, when selected, transmits a command to the first controllable external device to perform a second function different from the first function.

2. The first controllable external device is associated with the first function and the second function, The method according to claim 1, comprising, in accordance with the determination that the set of command transmission criteria has been met, transmitting a command to the first controllable external device via the communication device to perform the first function, without transmitting a command to the first controllable external device via the communication device to perform the second function.

3. The method according to any one of claims 1 to 2, wherein the set of command transmission criteria includes a first criterion that is satisfied when the first function associated with the first controllable external device is the primary function of the first controllable external device.

4. The set of command transmission criteria includes a second criterion that is satisfied when the first function associated with the first controllable external device is associated with a first controllable component having a binary operating state of the first controllable external device, The method according to any one of claims 1 to 3, wherein the first function is to transition from the current operating state of the first controllable component of the first controllable external device to another operating state of the first controllable component of the first controllable external device.

5. Displaying the first accessory interface on the display device is: In accordance with the determination that a portion of the first accessory interface is displayed at a position that extends beyond the boundary of the display device, the first accessory interface is displayed at a first position where it is aligned with the boundary, The method according to any one of claims 1 to 4, further comprising displaying the first accessory interface at a second position based on the position of the first accessory affordance, in accordance with the determination that the portion of the first accessory interface is not displayed in a position beyond the boundary of the display device.

6. The first accessory affordance includes a first graphical element and a first text element. The first accessory interface includes a second graphical element corresponding to the first graphical element and a second character element corresponding to the first character element, Displaying the first accessory interface is Displaying an animation of the first graphical element visually transitioning to the second graphical element, The method further includes displaying an animation of the first character element visually transitioning to the second character element, and the method is While the first accessory interface is being displayed, a second input corresponding to a request to stop displaying the first accessory interface is detected, The further includes stopping the display of the first accessory interface in response to detecting the second input, wherein the stopping is The second graphical element displays an animation in which it visually transitions to the first graphical element, The method according to any one of claims 1 to 5, further comprising displaying an animation in which the second character element visually transitions to the first character element.

7. The first accessory affordance described above is, In accordance with the determination that the first controllable external device is associated with the second controllable component, the second controllable component associated with the first function includes a graphical representation of the second controllable component. In accordance with the determination that the first controllable external device is associated with the third controllable component, the third controllable component associated with the second function includes a graphical representation of the third controllable component. The method according to any one of claims 1 to 6, comprising the graphical display of the second controllable component and the graphical display of the third controllable component, in accordance with the determination that the first controllable external device is associated with the second controllable component and the third controllable component.

8. The first controllable external device is associated with the second controllable component and the third controllable component, and the method is While the first accessory interface is displayed, a first set of one or more inputs is detected, including an input corresponding to a control reconfiguration affordance. The further includes, in response to detecting the first set of one or more inputs, displaying a second instance of the user interface, the second instance of the user interface is A second accessory affordance associated with the second controllable component of the first controllable external device, and not associated with the third controllable component of the first controllable external device, The method according to claim 7, further comprising: a third accessory affordance associated with the third controllable component of the first controllable external device, and not associated with the second controllable component of the first controllable external device.

9. The first accessory affordance has a first appearance before detecting the first input, and the method The method according to any one of claims 1 to 8, further comprising updating the first accessory affordance to a second appearance based on the first function, in response to the detection of the first input and in conjunction with transmitting the command to the first controllable external device for performing the first function.

10. The method according to claim 9, wherein the second appearance includes an animation based on the first function.

11. The first accessory interface includes the first control affordance and the second control affordance, and displaying the first accessory interface is In response to the first function and the second function satisfying a first set of layout criteria, the first control affordance and the second control affordance are displayed side by side, The method according to any one of claims 1 to 10, comprising displaying the first control affordance above the second control affordance in a vertical arrangement in response that the first function and the second function satisfy a second set of layout criteria.

12. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device comprising a display device and a communication device, wherein the one or more programs include instructions causing the method according to any one of claims 1 to 11 to be performed.

13. Display device and, Communication devices and One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 1 to 11.

14. Display device and, Communication devices and An electronic device comprising means for carrying out the method described in any one of claims 1 to 11.

15. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device equipped with a display device and a communication device, wherein the one or more programs are instructions, A user interface including a first accessory affordance associated with a first controllable external device is displayed on the display device. To detect the first input corresponding to the selection of the first accessory affordance, In response to the detection of the first input, In accordance with the determination that the set of command transmission criteria has been met, the first controllable external device is instructed via the communication device to send a command to perform the first function. In accordance with the determination that the set of command transmission criteria is not met, the display device is made to display the first accessory interface, and the first accessory interface contains: A first control affordance, which, when selected, transmits a command to the controllable external device to perform the first function; A non-temporary computer-readable storage medium includes an instruction comprising a second control affordance, which, when selected, transmits a command to the controllable external device to perform a second function different from the first function.

16. Display device and, Communication devices and One or more processors, A memory that stores one or more programs configured to be executed by one or more processors, An electronic device comprising, wherein one or more programs are instructions, A user interface including a first accessory affordance associated with a first controllable external device is displayed on the display device. To detect the first input corresponding to the selection of the first accessory affordance, In response to the detection of the first input, In accordance with the determination that the set of command transmission criteria has been met, the first controllable external device is instructed via the communication device to send a command to perform the first function. In accordance with the determination that the set of command transmission criteria is not met, the display device is made to display the first accessory interface, and the first accessory interface contains: A first control affordance, which, when selected, transmits a command to the controllable external device to perform the first function; An electronic device including an instruction comprising a second control affordance, which, when selected, transmits a command to the controllable external device to perform a second function different from the first function.

17. It is an electronic device, Display device and, Communication devices and Means for displaying a user interface on a display device that includes a first accessory affordance associated with a first controllable external device, Means for detecting a first input corresponding to the selection of the first accessory affordance, and means, In response to the detection of the first input, In accordance with the determination that the set of command transmission criteria has been met, a command to perform the first function is transmitted to the first controllable external device via the communication device. In accordance with the determination that the set of command transmission criteria is not met, the first accessory interface is displayed on the display device, and the first accessory interface contains: A first control affordance, which, when selected, transmits a command to the controllable external device to perform the first function; An electronic device comprising means including a second control affordance, which, when selected, transmits a command to the controllable external device to perform a second function different from the first function.

18. It is a method, In an electronic device equipped with a display device, Displaying a user interface on the display device that includes a first accessory affordance associated with a first controllable external device, To detect a first input corresponding to the selection of the first accessory affordance, In response to detecting the first input, the display device displays a first region of the first accessory interface, which includes a first control affordance that, when selected, controls a first function of the first controllable external device without displaying a second region of the first accessory interface. While the first area of ​​the first accessory interface is displayed on the display device, a second input is detected. In response to detecting the second input, In accordance with the determination that the second input causes a portion of the second area to display beyond a threshold amount, while the display of the first area is stopped, the user interface display is updated to display the second area, which includes one or more settings for the first controllable external device, above the first accessory interface. A method comprising updating the display of the first accessory interface based on the second input to the second control affordance in the first area of ​​the first accessory interface, without displaying the second area of ​​the first accessory interface, in accordance with the determination that the second input does not cause a portion of the second area to be displayed beyond the threshold amount.

19. The display device detects a third input while the first accessory interface is being displayed, In response to detecting the third input, To stop the display of the accessory interface described above, The method according to claim 18, further comprising outputting haptic and audio feedback indicating that the first accessory interface is no longer displayed.

20. The method according to any one of claims 18 to 19, further comprising outputting haptic and audio feedback in accordance with the determination that the second input causes a portion of the second region to be displayed above a threshold amount.

21. The first area of ​​the first accessory interface is The first controllable external device includes, in accordance with the determination that it is associated with fewer than a predetermined number of sensors, a set of control affordances corresponding to the sensors of the first controllable device at a first position within the first region, The method according to any one of claims 18 to 20, comprising the set of control affordances corresponding to the sensors of the first controllable external device at a second position different from the first position in the first region, according to the determination that the first controllable external device is associated with a predetermined number or more sensors.

22. The first region of the first accessory interface includes a second region affordance at a third position within the first region, and the method is as follows: To detect a fourth input corresponding to the second domain affordance, The method according to any one of claims 18 to 21, further comprising: updating the display of the user interface to display the second region above the first accessory interface while the display of the first region is stopped, in response to the detection of the fourth input corresponding to the second region affordance.

23. The method according to any one of claims 18 to 22, wherein the first accessory interface includes a first total number of control affordances, all of the first total number of control affordances are included in the first area, and none of the first total number of control affordances are included in the second area.

24. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display device, wherein the one or more programs include instructions for performing the method according to any one of claims 18 to 23.

25. It is an electronic device, Display device and, One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 18 to 23.

26. Display device and, An electronic device comprising means for carrying out the method described in any one of claims 18 to 23.

27. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device equipped with a display device, wherein the one or more programs are instructions, A user interface including a first accessory affordance associated with a first controllable external device is displayed on the display device. To detect the first input corresponding to the selection of the first accessory affordance, In response to detecting the first input, the display device is caused to display a first area of ​​the first accessory interface, which includes a first control affordance, if selected, that controls a first function of the first controllable external device without displaying a second area of ​​the first accessory interface. While the first area of ​​the first accessory interface is displayed on the display device, a second input is detected. In response to detecting the second input, In accordance with the determination that the second input causes a portion of the second area to display beyond a threshold amount, while the display of the first area is stopped, the user interface display is updated to display the second area, which includes one or more settings for the first controllable external device, above the first accessory interface. A non-temporary computer-readable storage medium containing instructions that update the display of the first accessory interface based on the second input to the second control affordance in the first area of ​​the first accessory interface, without displaying the second area of ​​the first accessory interface, in accordance with the determination that the second input does not display a portion of the second area beyond the threshold amount.

28. Display device and, One or more processors, A memory that stores one or more programs configured to be executed by one or more processors, An electronic device comprising, wherein one or more programs are instructions, A user interface including a first accessory affordance associated with a first controllable external device is displayed on the display device. To detect the first input corresponding to the selection of the first accessory affordance, In response to detecting the first input, the display device is caused to display a first area of ​​the first accessory interface, which includes a first control affordance, if selected, that controls a first function of the first controllable external device without displaying a second area of ​​the first accessory interface. While the first area of ​​the first accessory interface is displayed on the display device, a second input is detected. In response to detecting the second input, In accordance with the determination that the second input causes a portion of the second area to display beyond a threshold amount, while the display of the first area is stopped, the user interface display is updated to display the second area, which includes one or more settings for the first controllable external device, above the first accessory interface. An electronic device including a command that updates the display of the first accessory interface based on the second input to the second control affordance in the first area of ​​the first accessory interface, without displaying the second area of ​​the first accessory interface, in accordance with the determination that the second input does not cause a portion of the second area to be displayed beyond the threshold amount.

29. It is an electronic device, Display device and, Means for displaying a user interface on a display device that includes a first accessory affordance associated with a first controllable external device, Means for detecting a first input corresponding to the selection of the first accessory affordance, In response to detecting the first input, the display device is provided with means for displaying a first region of the first accessory interface, which includes a first control affordance, selected, for controlling a first function of the first controllable external device without displaying a second region of the first accessory interface. The display device includes means for detecting a second input while the first area of ​​the first accessory interface is being displayed. In response to detecting the second input, A means for updating the user interface display to show the second area, which includes one or more settings for the first controllable external device, while the display of the first area is stopped, in accordance with the determination that the second input causes a portion of the second area to be displayed beyond a threshold amount, An electronic device comprising: means for updating the display of the first accessory interface based on the second input to the second control affordance in the first region of the first accessory interface, without displaying the second region of the first accessory interface, in accordance with the determination that the second input does not cause a portion of the second region to be displayed beyond the threshold amount.

30. It is a method, In an electronic device equipped with a display device, While the electronic device is not configured to control the first controllable external device, the first set of one or more inputs corresponding to a request to configure the electronic device to control the first controllable external device is detected. The process includes, in response to detecting the first set of one or more inputs, initiating a process for configuring the electronic device to control the first controllable external device, wherein the process Receiving information corresponding to the first controllable external device, Based on the received information, the display device displays a first accessory affordance corresponding to a group including the first controllable external device and the plurality of sub-components, in accordance with the determination that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device. Based on the received information, in accordance with the determination that the first controllable external device is not associated with the plurality of subordinate components, the display device displays a second accessory affordance corresponding to the first controllable external device. The display device detects a second input while displaying the first accessory affordance or the second accessory affordance. In response to detecting the second input, In accordance with the determination that the second input corresponds to the selection of the first accessory affordance, the display device displays a first accessory interface including a plurality of control affordances corresponding to the first controllable external device and the plurality of subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process to remove at least one of the plurality of control affordances from the first accessory interface. A method comprising: determining that the second input corresponds to the selection of the second accessory affordance, and then displaying on the display device a second accessory interface that includes control of the first controllable external device without displaying the control reconfiguration affordance.

31. The method according to claim 30, wherein the first accessory interface includes a group control affordance, which, when selected, transmits a command to the first controllable external device that affects the state of each corresponding sub-component of the plurality of sub-components.

32. To detect a third input corresponding to the selection of the first control affordance, The method according to claim 31, further comprising: in response to the detection of the third input, transmitting a command to the first controllable external device that affects the state of each corresponding sub-component of the plurality of sub-components.

33. The plurality of control affordances of the first accessory interface include a first control affordance associated with a first sub-component among the plurality of sub-components, and a second control affordance associated with a second sub-component among the plurality of sub-components, and the method is While the first accessory interface is displayed, a fourth input corresponding to the first control affordance is detected, The method according to any one of claims 30 to 32, further comprising: in response to the detection of the fourth input, transmitting a command affecting the state of the first subordinate component to the first controllable external device without transmitting a command affecting the state of the second subordinate component to the first controllable external device.

34. The first accessory interface includes an accessory removal affordance, and the method is To detect a fifth input corresponding to the accessory removal affordance, In response to detecting the fifth input corresponding to the accessory removal affordance, the process further includes initiating a process to configure the electronic device so that it no longer controls the first controllable external device, the process is: Completing the process of configuring the electronic device to no longer control the first controllable external device, in accordance with a determination that a set of alert criteria has been met, which includes criteria that are met when the first controllable external device is a sub-component device of a composite external device that includes at least one other sub-component device that the electronic device is currently configured to control, also includes indicating that the electronic device is configured to no longer control the at least one other sub-component device, The method according to any one of claims 30 to 33, wherein completing the process of configuring the electronic device to no longer control the first controllable external device in accordance with the determination that the set of alert criteria is not met also includes ceasing to indicate that the electronic device is configured to no longer control at least one other subordinate component device.

35. The plurality of control affordances include one or more control affordances corresponding to a first type of subcomponent and one or more control affordances corresponding to a second type of subcomponent, and the method is While the first accessory interface is displayed, a second set of one or more inputs is detected, including an input corresponding to the control reconstruction affordance. In response to detecting the second set of one or more inputs, The display of the first accessory interface is stopped, and the display of the first accessory affordance is stopped, A third accessory affordance, which, when selected, displays a third accessory affordance that displays a third accessory interface including one or more control affordances corresponding to the first type of sub-component, The method according to any one of claims 30 to 34, further comprising displaying a fourth accessory affordance, which, when selected, displays a fourth accessory interface including one or more control affordances corresponding to a second type of sub-component.

36. The third accessory interface includes a regrouping affordance, and the method is While the third accessory interface is displayed, a third set of one or more inputs is detected, including an input corresponding to the regrouping affordance. In response to detecting the third set of one or more inputs, To stop the display of the third accessory affordance and the fourth accessory affordance, The method of claim 35, further comprising redisplaying the first accessory affordance, which, when selected, displays the first accessory interface including the plurality of control affordances.

37. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display device, wherein the one or more programs include instructions causing the method according to any one of claims 30 to 36.

38. It is an electronic device, Display device and, One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method described in any one of claims 30 to 36.

39. Display device and, An electronic device comprising means for carrying out the method described in any one of claims 30 to 36.

40. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device equipped with a display device, wherein the one or more programs are instructions While the electronic device is not configured to control the first controllable external device, it detects a first set of one or more inputs corresponding to a request to configure the electronic device to control the first controllable external device. An instruction to initiate a process for configuring the electronic device and controlling the first controllable external device in response to the detection of the first set of one or more inputs, wherein the process includes: To receive information corresponding to the first controllable external device, Based on the received information, the display device is made to display a first accessory affordance corresponding to a group including the first controllable external device and the plurality of sub-components, in accordance with the determination that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device. Based on the received information, the display device is made to display a second accessory affordance corresponding to the first controllable external device, in accordance with the determination that the first controllable external device is not associated with the plurality of subordinate components. The display device is made to detect a second input while displaying the first accessory affordance or the second accessory affordance. In response to detecting the second input, In accordance with the determination that the second input corresponds to the selection of the first accessory affordance, the display device is made to display a first accessory interface including a plurality of control affordances corresponding to the first controllable external device and the plurality of subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process to remove at least one of the plurality of control affordances from the first accessory interface. A non-temporary computer-readable storage medium comprising an instruction that causes the display device to display a second accessory interface, including control of the first controllable external device, without displaying the control reconfiguration affordance, in accordance with the determination that the second input corresponds to the selection of the second accessory affordance.

41. Display device and, One or more processors, An electronic device comprising a memory that stores one or more programs configured to be executed by one or more processors, wherein the one or more programs are instructions, While the electronic device is not configured to control the first controllable external device, it detects a first set of one or more inputs corresponding to a request to configure the electronic device to control the first controllable external device. The instruction includes, in response to detecting the first set of one or more inputs, an instruction to initiate a process for configuring the electronic device and controlling the first controllable external device, the process being: Receiving information corresponding to the first controllable external device, Based on the received information, the display device displays a first accessory affordance corresponding to a group including the first controllable external device and the plurality of sub-components, in accordance with the determination that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device. Based on the received information, in accordance with the determination that the first controllable external device is not associated with the plurality of subordinate components, the display device displays a second accessory affordance corresponding to the first controllable external device. The display device detects a second input while displaying the first accessory affordance or the second accessory affordance. In response to detecting the second input, In accordance with the determination that the second input corresponds to the selection of the first accessory affordance, the display device displays a first accessory interface including a plurality of control affordances corresponding to the first controllable external device and the plurality of subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process to remove at least one of the plurality of control affordances from the first accessory interface. An electronic device comprising: displaying a second accessory interface, which includes control of the first controllable external device, on the display device without displaying the control reconfiguration affordance, in accordance with the determination that the second input corresponds to the selection of the second accessory affordance.

42. It is an electronic device, Display device and, Means for detecting a first set of one or more inputs corresponding to a request to configure an electronic device to control a first controllable external device while the electronic device is not configured to control the first controllable external device, Means for initiating a process for configuring the electronic device to control the first controllable external device in response to detecting the first set of one or more inputs, wherein the process is: Receiving information corresponding to the first controllable external device, Based on the received information, the display device displays a first accessory affordance corresponding to a group including the first controllable external device and the plurality of sub-components, in accordance with the determination that the first controllable external device is associated with a plurality of controllable functions associated with a plurality of sub-components of the first controllable external device. Means including, based on the received information, displaying a second accessory affordance corresponding to the first controllable external device on the display device, in accordance with the determination that the first controllable external device is not associated with the plurality of subordinate components, The display device includes means for detecting a second input while the first accessory affordance or the second accessory affordance is being displayed. A means, in response to the detection of the aforementioned second input, In accordance with the determination that the second input corresponds to the selection of the first accessory affordance, the display device displays a first accessory interface including a plurality of control affordances corresponding to the first controllable external device and the plurality of subordinate components, and a control reconfiguration affordance, which, when selected, initiates a process to remove at least one of the plurality of control affordances from the first accessory interface. An electronic device comprising, in accordance with the determination that the second input corresponds to the selection of the second accessory affordance, means for displaying a second accessory interface, including control of the first controllable external device, on the display device without displaying the control reconfiguration affordance.

43. It is a method, In an electronic device equipped with a display device, The display device displays a temperature mode creation user interface including a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device. While the temperature mode creation user interface is displayed on the display device, a first set of one or more inputs is detected. In response to the aforementioned first set of one or more inputs, Select a temperature range that includes a minimum and maximum temperature that is different from the minimum temperature range, Associating the aforementioned temperature range with the first temperature mode, After associating the temperature range with a first temperature mode, the display device displays a temperature scheduling user interface including a second set of control for creating scheduled operations of the external thermostat device. While the temperature scheduling user interface is displayed on the display device, a second set of one or more inputs is detected. In response to detecting the second set of one or more inputs, A method comprising selecting a first criterion for activating a first scheduled operation, and associating the first temperature mode with the first scheduled operation.

44. The method according to claim 43, wherein the second set of one or more inputs includes an input corresponding to the selection of the first temperature mode from among a plurality of available temperature modes.

45. Selecting the aforementioned temperature range means The user interface for creating the temperature mode displays a graphical representation of the selected temperature range, In accordance with the fact that the first portion of the selected temperature range is within a first predetermined temperature range, the first portion of the graphical display is displayed in a first color corresponding to the first predetermined temperature range. The method according to any one of claims 43 to 44, comprising displaying the first portion of the graphical display in a second color different from the first color, corresponding to the second predetermined temperature range, in accordance with the fact that the first portion of the selected temperature range is within a second predetermined temperature range.

46. Selecting the aforementioned temperature range means Displaying a numerical value for the minimum temperature, In accordance with the fact that the minimum temperature is within the first predetermined temperature range, the numerical value of the minimum temperature is displayed in the first color. The minimum temperature is within the second predetermined temperature range, and the numerical value of the minimum temperature is displayed in the second color, Displaying a numerical representation of the aforementioned maximum temperature, In accordance with the fact that the maximum temperature is within the first predetermined temperature range, the numerical value of the maximum temperature is displayed in the first color. The method according to claim 45, comprising displaying the numerical value of the maximum temperature in the second color according to the fact that the maximum temperature is within the second predetermined temperature range.

47. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display device, wherein the one or more programs include instructions causing the method according to any one of claims 43 to 46.

48. It is an electronic device, Display device and, One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method described in any one of claims 43 to 46.

49. Display device and, An electronic device comprising means for carrying out the method described in any one of claims 43 to 46.

50. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device equipped with a display device, wherein the one or more programs are instructions, The display device is made to display a temperature mode creation user interface which includes a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device. While the temperature mode creation user interface is displayed on the display device, a first set of one or more inputs is detected. In response to the aforementioned first set of one or more inputs, Select a temperature range that includes a minimum and maximum temperature, which is different from the minimum temperature range. The aforementioned temperature range is associated with a first temperature mode, After associating the temperature range with a first temperature mode, the display device is made to display a temperature scheduling user interface including a second set of scheduled operation creation control for the external thermostat device. While the temperature scheduling user interface is displayed on the display device, a second set of one or more inputs is detected. In response to detecting the second set of one or more inputs, Select a first criterion for activating the first scheduled action, A non-temporary computer-readable storage medium including instructions for associating the first temperature mode with the first scheduled operation.

51. Display device and, One or more processors, A memory that stores one or more programs configured to be executed by one or more processors, An electronic device comprising, wherein one or more programs are instructions, The display device is made to display a temperature mode creation user interface which includes a first set of temperature mode creation controls for an external thermostat device controllable by the electronic device. While the temperature mode creation user interface is displayed on the display device, a first set of one or more inputs is detected. In response to the aforementioned first set of one or more inputs, Select a temperature range that includes a minimum and maximum temperature, which is different from the minimum temperature range. The aforementioned temperature range is associated with a first temperature mode, After associating the temperature range with a first temperature mode, the display device is made to display a temperature scheduling user interface including a second set of scheduled operation creation control for the external thermostat device. While the temperature scheduling user interface is displayed on the display device, a second set of one or more inputs is detected. In response to detecting the second set of one or more inputs, An electronic device including instructions for selecting a first criterion for activating a first scheduled operation and associating the first temperature mode with the first scheduled operation.

52. It is an electronic device, Display device and, The display device includes means for displaying a temperature mode creation user interface, which includes a first set of temperature mode creation controls for an external thermostat device controllable by an electronic device. Means for detecting a first set of one or more inputs while the temperature mode creation user interface is displayed on the display device, In response to the aforementioned first set of one or more inputs, Select a temperature range that includes a minimum and maximum temperature that is different from the minimum temperature range. Means for associating the temperature range with a first temperature mode, After associating the temperature range with a first temperature mode, the display device includes means for displaying a temperature scheduling user interface, which includes a second set of scheduling operation control for the external thermostat device. Means for detecting a second set of one or more inputs while the temperature scheduling user interface is displayed on the display device, In response to detecting the second set of one or more inputs, Select a first criterion for activating the first scheduled action, An electronic device comprising means for associating the first temperature mode with the first scheduled operation.

53. It is a method, In an electronic device equipped with a display device, The first accessory affordance corresponding to the first controllable external device is displayed via the aforementioned display device, Detecting one or more inputs for adding the first controllable external device to a first set of controllable external devices, In response to detecting one or more inputs for adding the first controllable external device to the first set of controllable external devices, Updating the first set of controllable external devices to include the first controllable external device, According to the determination that the first controllable external device is of the first type, A second accessory affordance corresponding to the first controllable external device, A configuration affordance, which, when selected, initiates the process of configuring the audio settings of the first controllable external device, and a configuration affordance, are displayed simultaneously. A method comprising, in accordance with the determination that the first controllable external device is not of the first type, displaying the configuration affordance, which, when selected, displays the second accessory affordance corresponding to the first controllable external device without the configuration affordance for initiating the process for configuring the audio settings of the first controllable external device.

54. Initiating the process for configuring the audio settings of the first controllable external device includes displaying audio selection affordances, and the method is To detect the selection of the aforementioned audio selection affordance, The method according to claim 53, further comprising: in response to detecting the selection of the audio selection affordance, initiating a process for selecting the audio item of the first media type to be played on the first controllable external device, which includes displaying one or more options for navigating to an audio item of the first media type.

55. Initiating the process for configuring the audio settings of the first controllable external device includes displaying audio selection affordances, and the method is To detect the selection of the aforementioned audio selection affordance, In response to detecting the selection of the aforementioned audio selection affordance, via the display device, A first audio category option, which, when selected, initiates a process for selecting an audio item of a first media type, The method according to claim 53, further comprising simultaneously displaying a second audio category option, which, when selected, initiates a process for selecting an audio item of a second media type different from the first media type.

56. Initiating the process for configuring the audio settings of the first controllable external device includes displaying audio selection affordances, and the method is After detecting the selection of the aforementioned audio selection affordance, one or more user inputs are detected to select an audio item to be played on the first controllable external device. After detecting one or more user inputs for selecting the audio item to be played on the first controllable external device, via the display device, The aforementioned audio selection affordance and, The method according to any one of claims 53 to 55, further comprising displaying simultaneously a representation of the audio item selected to be played back on the first controllable external device.

57. Initiating the process for configuring the audio settings of the first controllable external device includes displaying audio selection affordances, and the method is After detecting the selection of the aforementioned audio selection affordance, one or more user inputs are detected to select an audio item to be played on the first controllable external device. After detecting one or more user inputs for selecting the audio item to be played on the first controllable external device, via the display device, A repeat affordance, which, when selected, determines whether to repeat the audio item being played on the first controllable external device, The method according to any one of claims 53 to 56, further comprising displaying a shuffle affordance, which, when selected, constitutes an order in which the audio items are played on the first controllable external device.

58. The method according to any one of claims 53 to 57, wherein initiating the process for configuring the audio settings of the first controllable external device includes displaying audio playback options in accordance with the determination that a first audio mode type has been selected.

59. While the aforementioned audio playback options are displayed, the selection of the audio mode is detected, In response to detecting the selection of the aforementioned audio mode, In accordance with the determination that the first audio mode type has been selected, the audio playback options will continue to be displayed, The method according to claim 58, further comprising stopping the display of the audio playback options in accordance with the determination that a second audio mode type different from the first audio mode type has been selected.

60. Detecting one or more inputs for adding a second controllable external device to the first set of controllable external devices, The method according to any one of claims 53 to 59, further comprising updating the first set of controllable external devices to include the second controllable external device in response to detecting one or more inputs for adding the second controllable external device to the first set of controllable external devices.

61. After initiating the process for configuring the audio settings of the first controllable external device, the system detects user input to customize the playback volume of the audio item selected to be played on the first controllable external device, In response to detecting the user input for customizing the playback volume, In accordance with the determination that the first set of controllable external devices includes a single controllable external device of the first type, a single volume control for adjusting the playback volume of the audio item on the first controllable external device is displayed. The method according to any one of claims 53 to 60, further comprising displaying a plurality of volume controls for adjusting the respective playback volume of the audio item in each of the controllable external devices, in accordance with the determination that the first set of controllable external devices includes a plurality of controllable external devices of the first type.

62. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device comprising a display device and an audio input device, wherein the one or more programs include instructions causing the method according to any one of claims 53 to 61.

63. It is an electronic device, Display device and, One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 53 to 61.

64. Display device and, An electronic device comprising means for carrying out the method described in any one of claims 53 to 61.

65. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device equipped with a display device, wherein the one or more programs are The first accessory affordance corresponding to the first controllable external device is displayed via the aforementioned display device. To detect one or more inputs for adding the first controllable external device to a first set of controllable external devices, In response to detecting one or more inputs for adding the first controllable external device to the first set of controllable external devices, Update the first set of controllable external devices to include the first controllable external device, According to the determination that the first controllable external device is of the first type, A second accessory affordance corresponding to the first controllable external device, A configuration affordance, which, when selected, initiates the process of configuring the audio settings for the first controllable external device, is displayed simultaneously. A non-temporary computer-readable storage medium that, in accordance with the determination that the first controllable external device is not of the first type, includes an instruction that causes the system to display the second accessory affordance corresponding to the first controllable external device, without the configuration affordance which, when selected, initiates the process of configuring the audio settings for the first controllable external device.

66. Display device and, One or more processors, A memory that stores one or more programs configured to be executed by one or more processors, An electronic device comprising, the one or more programs, The first accessory affordance corresponding to the first controllable external device is displayed via the aforementioned display device. To detect one or more inputs for adding the first controllable external device to a first set of controllable external devices, In response to detecting one or more inputs for adding the first controllable external device to the first set of controllable external devices, Update the first set of controllable external devices to include the first controllable external device, According to the determination that the first controllable external device is of the first type, A second accessory affordance corresponding to the first controllable external device, A configuration affordance, which, when selected, initiates the process of configuring the audio settings for the first controllable external device, is displayed simultaneously. An electronic device that, in accordance with the determination that the first controllable external device is not of the first type, includes a command that causes the configuration affordance, which, when selected, causes the display of the second accessory affordance corresponding to the first controllable external device without the configuration affordance that initiates the process of configuring the audio settings for the first controllable external device.

67. It is an electronic device, Display device and, Means for displaying a first accessory affordance corresponding to a first controllable external device via a display device, Means for detecting one or more inputs for adding the first controllable external device to a first set of controllable external devices, and means, In response to detecting one or more inputs for adding the first controllable external device to the first set of controllable external devices, Update the first set of controllable external devices to include the first controllable external device, According to the determination that the first controllable external device is of the first type, A second accessory affordance corresponding to the first controllable external device, A configuration affordance, which, when selected, initiates a process for configuring the audio settings of the first controllable external device, is displayed simultaneously. An electronic device comprising: a configuration affordance, in accordance with the determination that the first controllable external device is not of the first type, means for displaying the second accessory affordance corresponding to the first controllable external device, without the configuration affordance, which, when selected, initiates a process for configuring the audio settings of the first controllable external device.

68. It is a method, In an electronic device equipped with a display device, The display device simultaneously displays one or more sets of playback controls and alarm configuration affordances for performing playback functions related to media items. Detecting user input and In response to detecting the aforementioned user input, In accordance with the determination that the detected user input corresponds to the selection of a playback control from the set of one or more playback controls, a process is initiated to perform the playback function on the controllable external device based on the selected playback control. A method comprising replacing the set of one or more playback controls and the simultaneous display of the alarm configuration affordance with a set of time affordances for setting the time of a first alarm, in accordance with the determination that the detected user input corresponds to the selection of the alarm configuration affordance.

69. In accordance with the determination that the detected user input corresponds to the selection of the alarm configuration affordance, The set of time affordances for setting the time of the first alarm, along with a plurality of alarm sound options including a first option and a second option, are displayed simultaneously. To detect a second user input corresponding to one of the aforementioned multiple alarm sound options, In response to detecting the second user input corresponding to one of the aforementioned multiple alarm sound options, In accordance with the determination that the second user input corresponds to the selection of the first option, a process is initiated to select a media item of the first media type for the first alarm. The method according to claim 68, further comprising: initiating a process for selecting a media item of a second media type for the first alarm, in accordance with the determination that the second user input corresponds to the selection of the second option.

70. The method according to claim 69, wherein initiating the process for selecting a media item of a first media type for the first alarm comprises replacing the simultaneous display of the set of time affordances and the plurality of alarm sound options with a display of a plurality of selectable media items of the first media type.

71. The method according to any one of claims 69 to 70, wherein initiating the process for selecting a media item of a second media type for the first alarm comprises replacing the simultaneous display of the set of time affordances and the plurality of alarm sound options with a display of one or more options for navigating to the media item of the second media type.

72. In accordance with the determination that the alarm configuration affordance is of a first type, the set of time affordances for setting the time of the first alarm first includes a visual indication of the current time of the time of the first alarm, The method according to any one of claims 68 to 71, further comprising the following: the set of time affordances for setting the time of the first alarm, in accordance with the determination that the alarm configuration affordance is of a second type different from the first type, firstly includes a visual indication of the time the first alarm was previously set.

73. In accordance with the determination that the detected user input corresponds to the selection of the alarm configuration affordance, The method according to any one of claims 68 to 72, further comprising displaying one or more volume options for setting the volume of the first alarm sounded on the controllable external device, along with the set of time affordances for setting the time of the first alarm.

74. While displaying one or more volume options for setting the volume of the alarm sounded by the controllable external device, the system detects user input for customizing the volume of the first alarm, The method according to claim 73, further comprising: detecting the user input for customizing the volume of the first alarm, and displaying a volume control for adjusting the volume of the first alarm sounded on the controllable external device.

75. In accordance with the determination that the detected user input corresponds to the selection of the alarm configuration affordance, The set of time affordances for setting the time of the first alarm is displayed simultaneously with the snooze affordance for enabling the snooze option on the controllable external device, To detect the selection of the snooze affordance for enabling the snooze option, To detect one or more user inputs for setting the first alarm on the controllable external device, The method according to any one of claims 68 to 74, further comprising: initiating a process to configure the controllable external device to temporarily disable the first alarm that sounds on the controllable external device in response to the detection of one or more inputs for setting the first alarm on the controllable external device, and in response to the detection of an audio input that meets certain criteria, in accordance with the determination that the snooze option is enabled.

76. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display device, wherein the one or more programs include instructions causing the method according to any one of claims 68 to 75.

77. It is an electronic device, Display device and, One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 68 to 75.

78. Display device and, An electronic device comprising means for carrying out the method described in any one of claims 68 to 75.

79. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device equipped with a display device, wherein the one or more programs are instructions, The display device simultaneously displays one or more sets of playback controls and alarm configuration affordances for performing playback functions related to media items. Detect user input, In response to detecting the aforementioned user input, In accordance with the determination that the detected user input corresponds to the selection of a playback control from the set of one or more playback controls, a process for performing the playback function on a controllable external device is initiated based on the selected playback control. A non-temporary computer-readable storage medium including instructions that, in accordance with the determination that the detected user input corresponds to the selection of the alarm configuration affordance, cause the simultaneous display of one or more playback controls and the alarm configuration affordance to be replaced with a set of time affordances for setting the time of a first alarm.

80. Display device and, One or more processors, A memory that stores one or more programs configured to be executed by one or more processors, An electronic device comprising, the one or more programs, The display device simultaneously displays one or more sets of playback controls and alarm configuration affordances for performing playback functions related to media items. Detect user input, In response to detecting the aforementioned user input, In accordance with the determination that the detected user input corresponds to the selection of a playback control from the set of one or more playback controls, a process for performing the playback function on a controllable external device is initiated based on the selected playback control. An electronic device including a command that, in accordance with the determination that the detected user input corresponds to the selection of the alarm configuration affordance, causes the simultaneous display of one or more playback controls and the alarm configuration affordance to be replaced with a set of time affordances for setting the time of a first alarm.

81. It is an electronic device, Display device and, A means for simultaneously displaying one or more sets of playback controls and alarm configuration affordances for performing playback functions related to a media item via a display device, Means for detecting user input, and means, In response to detecting the aforementioned user input, In accordance with the determination that the detected user input corresponds to the selection of a playback control from the set of one or more playback controls, a process is initiated to perform the playback function on the controllable external device based on the selected playback control. An electronic device comprising means for replacing the set of one or more playback controls and the simultaneous display of the alarm configuration affordances with a set of time affordances for setting the time of a first alarm, in accordance with the determination that the detected user input corresponds to the selection of the alarm configuration affordances.

82. It is a method, In an electronic device equipped with a display device, Receiving a playback request for a first type of audio media item, wherein the request identifies the first type of audio media item but does not identify an individual first type of audio media item, In response to receiving the playback request for the first type of audio media item, Playing a first audio media item from the first type of audio media item described above, A method comprising displaying an affordance which, when selected, plays a second audio media item from among the first type of audio media items, wherein the second audio media item from among the first type of audio media items is different from the first audio media item from among the first type of audio media items.

83. The method according to claim 82, further comprising playing the first audio media item from among the first type of audio media items and simultaneously displaying a graphical object that identifies the first audio media item.

84. After playing the first audio media item of the first type, the system receives a second playback request for the first type of audio media item, which identifies the first type of audio media item but does not identify individual audio media items of the first type. In response to receiving the second request to play the first type of audio media item, Playing a third audio media item from the first type of audio media item, A request to play an audio media item of a first type, which does not identify individual audio media items of the first type, and according to a determination that a first set of criteria has been met, a second affordance is displayed, which, when selected, plays a fourth audio media item of the first type, where the fourth audio media item of the first type is different from the third audio media item of the first type. The method according to any one of claims 82 to 83, further comprising discontinuing the display of the second affordance in accordance with the determination that the first set of criteria is not met.

85. After receiving a second predetermined number of requests to play an audio media item of the first type, which does not identify an individual audio media item of the first type, a third request for playback of an audio media item of the first type is received, which identifies an audio media item of the first type but does not identify an individual audio media item of the first type. In response to receiving the aforementioned third request, The method according to any one of claims 82 to 84, further comprising playing a fifth audio media item from among the first type of audio media items selected before receiving the third request.

86. The method according to any one of claims 82 to 85, wherein the first type of audio media item is an audio media item that does not include instrumental or vocal audio elements.

87. The method according to any one of claims 82 to 86, wherein the first type of audio media item is an audio media item comprising a plurality of randomly selected and arranged audio segments.

88. The aforementioned electronic device includes a microphone, The method according to any one of claims 82 to 87, wherein the request to play the first type of audio media item is an audio input detected via the microphone.

89. The electronic device includes a touch-sensing surface, The method according to any one of claims 82 to 87, wherein the requirement for playback of the first type of audio media item is a set of one or more inputs detected via a touch-sensing surface.

90. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device having a display device, wherein the one or more programs include instructions causing the method according to any one of claims 82 to 89.

91. It is an electronic device, Display device and, One or more processors, An electronic device comprising: a memory for storing one or more programs configured to be executed by the one or more processors, wherein the one or more programs include instructions for performing the method according to any one of claims 82 to 89.

92. Display device and, An electronic device comprising means for carrying out the method according to any one of claims 82 to 89.

93. A non-temporary computer-readable storage medium for storing one or more programs configured to be executed by one or more processors of an electronic device equipped with a display device, wherein the one or more programs are instructions, A request for playback of a first type of audio media item, which identifies the first type of audio media item but does not identify individual first type audio media items, is received. In response to receiving the playback request for the first type of audio media item, Play the first audio media item from the first type of audio media item described above. An affordance that, when selected, displays an affordance to play a second audio media item from among the first type of audio media items, wherein the second audio media item from among the first type of audio media items contains instructions that are different from the first audio media item from among the first type of audio media items.

94. Display device and, One or more processors, A memory that stores one or more programs configured to be executed by one or more processors, An electronic device comprising, wherein one or more programs are instructions that cause the device to receive a request to play a first type of audio media item, the request identifying the first type of audio media item but not identifying individual first type audio media items, In response to receiving the playback request for the first type of audio media item, Play the first audio media item from the first type of audio media item described above. An electronic device comprising an affordance that, when selected, causes a command to display an affordance for playing a second audio media item of the first type of audio media item, wherein the second audio media item of the first type of audio media item is different from the first audio media item of the first type of audio media item.

95. Display device and, An electronic device comprising means for receiving a playback request for a first type of audio media item, wherein the request identifies the first type of audio media item but does not identify individual audio media items of the first type, In response to receiving the request to play the first type of audio media item, Play the first audio media item from the first type of audio media item described above. An electronic device comprising means for displaying an affordance which, when selected, plays a second audio media item from among the first type of audio media items, wherein the second audio media item from among the first type of audio media items is different from the first audio media item from among the first type of audio media items.

Images