User interface for audio media controls

Abstract

This disclosure relates to user interfaces for audio media controls. A method includes displaying a first media information affordance representing a first media item in response to detecting an indication that a physical proximity between an electronic device and an external device satisfies a proximity condition. The electronic device receives a first input representing a selection of the first media information affordance. In response to receiving the first input: in accordance with a determination that the first input is of a first type of input, initiating a process to play back the first media item; and in accordance with a determination that the first input is of a second type of input that is different from the first type of input, displaying a second media information affordance representing the first media item.

Description

[0001] Case Separation Statement

[0002] This application is a divisional application of the invention patent application with PCT international application number PCT / US2020 / 035488, international application date May 31, 2020, Chinese national phase application number 202080031753.2, and invention title "User Interface for Audio Media Control".

[0003] Cross-reference to related applications

[0004] This application claims priority to U.S. Patent Application Serial No. 16 / 583,989 entitled "USER INTERFACES FOR AUDIO MEDIA CONTROL", filed September 26, 2019; U.S. Patent Application Serial No. 16 / 584,490 entitled "USER INTERFACES FOR AUDIO MEDIA CONTROL", filed September 26, 2019; U.S. Provisional Patent Application Serial No. 63 / 032,603 ​​entitled "USER INTERFACES FOR AUDIO MEDIA CONTROL", filed May 30, 2020; and U.S. Provisional Patent Application Serial No. 62 / 855,852 entitled "USER INTERFACES FOR AUDIO MEDIA CONTROL", filed May 31, 2019, the contents of each of these patent applications are incorporated herein by reference in their entirety. Technical Field

[0005] This disclosure relates generally to computer user interfaces, and more specifically, to techniques for controlling audio playback. Background Technology

[0006] The number of electronic devices, especially smart devices, is constantly increasing. These devices are becoming more interconnected, more feature-rich, and capable of performing more complex tasks. As a result, users increasingly expect these devices to have well-designed user interfaces. Summary of the Invention

[0007] Some technologies used to control audio playback with electronic devices are often cumbersome and inefficient. For example, some existing technologies use complex and time-consuming user interfaces that may involve multiple keystrokes or button presses. Such technologies require more time than necessary, resulting in wasted user time and device power. This latter consideration is particularly important in battery-powered devices.

[0008] Therefore, the technology of this invention provides electronic devices with faster and more efficient methods and interfaces for controlling audio playback. Such methods and interfaces optionally complement or replace other methods for controlling audio playback. These methods and interfaces reduce the cognitive burden on the user and result in a more efficient human-machine interface. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charging cycles.

[0009] This document discloses an example method. An exemplary method includes, at an electronic device having a display device: upon connection to an external device, detecting an indication that physical proximity between the electronic device and the external device satisfies a proximity condition; in response to detecting the indication that physical proximity satisfies the proximity condition, displaying via the display device a first media information indication representing a first media item, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: based on determining that the first input is a first type of input, transmitting an instruction to the external device to initiate playback of the first media item on the external device; and based on determining that the first input is a second type of input different from the first type of input, displaying via the display device a second media information indication representing the first media item, the second media information indication differing from the first media information indication and including a second set of information about the first media item.

[0010] This document describes an example non-transitory computer-readable storage medium. The exemplary non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for performing the following operations: upon connection to an external device, detecting an indication that physical proximity between the electronic device and the external device satisfies a proximity condition; in response to detecting the indication that physical proximity satisfies the proximity condition, displaying via the display device a first media information indication representing a first media item, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: based on determining that the first input is a first type of input, transmitting instructions to the external device to initiate playback of the first media item on the external device; and based on determining that the first input is a second type of input different from the first type of input, displaying via the display device a second media information indication representing the first media item, the second media information indication differing from the first media information indication and including a second set of information about the first media item.

[0011] This document describes an example transient computer-readable storage medium. The exemplary transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for performing the following operations: upon connection to an external device, detecting an indication that physical proximity between the electronic device and the external device satisfies a proximity condition; in response to detecting the indication that physical proximity satisfies the proximity condition, displaying via the display device a first media information indication representing a first media item, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: based on determining that the first input is a first type of input, transmitting an instruction to the external device to initiate playback of the first media item on the external device; and based on determining that the first input is a second type of input different from the first type of input, displaying via the display device a second media information indication representing the first media item, the second media information indication differing from the first media information indication and including a second set of information about the first media item.

[0012] This document describes an example electronic device. An example electronic device includes a display device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: upon connection to an external device, detecting an indication that physical proximity between the electronic device and the external device satisfies a proximity condition; in response to detecting the indication that physical proximity satisfies the proximity condition, displaying via the display device a first media information indication representing a first media item, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: based on determining that the first input is a first type of input, transmitting instructions to the external device to initiate playback of the first media item on the external device; and based on determining that the first input is a second type of input different from the first type of input, displaying via the display device a second media information indication representing the first media item, the second media information indication being different from the first media information indication and including a second set of information about the first media item.

[0013] An example electronic device includes a display device; means for detecting, when connected to an external device, an indication that the physical proximity between the electronic device and the external device meets a proximity condition; means for displaying, via the display device, a first media information indication representing a first media item, the first media information indication including a first set of information about the first media item, in response to detecting the indication that the physical proximity meets the proximity condition; means for receiving a first input indicating a selection of the first media information indication; and means for, in response to receiving the first input: transmitting an instruction to the external device to initiate playback of the first media item on the external device based on determining that the first input is a first type of input; and for displaying, via the display device, a second media information indication representing the first media item, the second media information indication being different from the first media information indication and including a second set of information about the first media item, based on determining that the first input is a second type of input different from the first type of input.

[0014] An exemplary method includes, at an electronic device having a display device: upon connection to an external device, in response to an indication that physical proximity meets proximity conditions, detecting an indication that physical proximity between the electronic device and the external device meets proximity conditions, displaying via the display device a first media information indication indicating that a first media item is currently being played on the external device, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: initiating playback of the first media item on the electronic device based on determining that the first input is a first type of input; and displaying via the display device a second media information indication indicating that the first media item is different from the first media information indication and includes a second set of information about the first media item.

[0015] An exemplary non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for performing the following operations: upon connection to an external device, in response to an indication that physical proximity meets proximity conditions, detecting an indication that physical proximity between the electronic device and the external device meets proximity conditions, displaying via the display device a first media information indication indicating that a first media item is currently being played on the external device, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: initiating playback of the first media item on the electronic device based on determining that the first input is a first type of input; and displaying via the display device a second media information indication indicating that the first media item is different from the first media information indication and includes a second set of information about the first media item.

[0016] An exemplary transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for performing the following operations: upon connection to an external device, in response to an indication that physical proximity meets proximity conditions, detecting an indication that physical proximity between the electronic device and the external device meets proximity conditions, displaying via the display device a first media information indication indicating that a first media item is currently being played on the external device, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: initiating playback of the first media item on the electronic device based on determining that the first input is a first type of input; and displaying via the display device a second media information indication indicating that the first media item is different from the first media information indication and includes a second set of information about the first media item.

[0017] An example electronic device includes a display device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: upon connection to an external device, in response to an indication that physical proximity meets proximity conditions, detecting an indication that physical proximity between the electronic device and the external device meets proximity conditions, displaying via the display device a first media information indication indicating that a first media item is currently being played on the external device, the first media information indication including a first set of information about the first media item; receiving a first input indicating a selection of the first media information indication; and in response to receiving the first input: initiating playback of the first media item on the electronic device based on determining that the first input is a first type of input; and displaying via the display device a second media information indication indicating that the first media item is different from the first media information indication and includes a second set of information about the first media item.

[0018] An example electronic device includes a display device; means for detecting, when connected to an external device, an indication that the physical proximity between the electronic device and the external device meets a proximity condition; means for displaying, via the display device, a first media information indication indicating that a first media item is currently playing on the external device, the first media information indication including a first set of information about the first media item, in response to detecting the indication that the physical proximity meets the proximity condition; means for receiving a first input indicating a selection of the first media information indication; and means for, in response to receiving the first input: initiating playback of the first media item on the electronic device based on determining that the first input is a first type of input; and displaying, via the display device, a second media information indication indicating that the first media item is different from the first media information indication and includes a second set of information about the first media item, based on determining that the first input is a second type of input different from the first type of input.

[0019] An exemplary method includes, at an electronic device having a display device: when connected to an external device, detecting an indication that the physical proximity between the electronic device and the external device satisfies a first proximity condition; in response to detecting the indication that the physical proximity satisfies the first proximity condition, displaying a first media information enablement representation of a first media item via the display device; while displaying the first media information enablement representation of the first media item, detecting an indication that the physical proximity between the electronic device and the external device satisfies a second proximity condition; and in response to detecting the indication that the physical proximity satisfies the second proximity condition, initiating playback of the first media item.

[0020] An exemplary non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for performing the following operations: when connected to an external device, detecting an indication that the physical proximity between the electronic device and the external device satisfies a first proximity condition; in response to detecting the indication that the physical proximity satisfies the first proximity condition, displaying a first media information enablement representation representing a first media item via the display device; while displaying the first media information enablement representation representing the first media item, detecting an indication that the physical proximity between the electronic device and the external device satisfies a second proximity condition; and in response to detecting the indication that the physical proximity satisfies the second proximity condition, initiating playback of the first media item.

[0021] An exemplary transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device, the one or more programs including instructions for performing the following operations: when connected to an external device, detecting an indication that the physical proximity between the electronic device and the external device satisfies a first proximity condition; in response to detecting the indication that the physical proximity satisfies the first proximity condition, displaying a first media information enablement representation of a first media item via the display device; while displaying the first media information enablement representation of the first media item, detecting an indication that the physical proximity between the electronic device and the external device satisfies a second proximity condition; and initiating playback of the first media item in response to detecting the indication that the physical proximity satisfies the second proximity condition.

[0022] An example electronic device includes a display device; one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: when connected to an external device, detecting an indication that the physical proximity between the electronic device and the external device satisfies a first proximity condition; in response to detecting the indication that the physical proximity satisfies the first proximity condition, displaying a first media information enablement representation representing a first media item via the display device; while displaying the first media information enablement representation representing the first media item, detecting an indication that the physical proximity between the electronic device and the external device satisfies a second proximity condition; and initiating playback of the first media item in response to detecting the indication that the physical proximity satisfies the second proximity condition.

[0023] An example electronic device includes a display device; means for detecting, when connected to an external device, an indication that the physical proximity between the electronic device and the external device satisfies a first proximity condition; means for displaying, via the display device, a first media information enablement indication of a first media item in response to detecting the indication that the physical proximity satisfies the first proximity condition; means for detecting, when displaying the first media information enablement indication of the first media item, an indication that the physical proximity between the electronic device and the external device satisfies a second proximity condition; and means for initiating playback of the first media item in response to detecting the indication that the physical proximity satisfies the second proximity condition.

[0024] An exemplary method includes, at a computer system communicating with a display generation component and one or more input devices: displaying via the display generation component a first user interface including a first plurality of optional user interface objects, wherein the first plurality of optional user interface objects include: a first optional user interface object that, when selected, causes the computer system to modify the state of the computer system; a second optional user interface object, which, when selected, causes a first external accessory device among a plurality of external accessory devices to perform a first function, based on determining that the computer system is in a first usage context; and a third optional user interface object, which, when selected, causes a second external accessory device among a plurality of external accessory devices to perform a second function, based on determining that the computer system is in a second usage context different from the first usage context, wherein the third optional user interface object is not included in the user interface based on determining that the computer system is in the first usage context.

[0025] An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for performing the following operations: displaying, via the display generation component, a first user interface comprising a first plurality of optional user interface objects, wherein the first plurality of optional user interface objects comprises: a first optional user interface object that, when selected, causes the computer system to modify the state of the computer system; a second optional user interface object, when selected, causing a first external accessory device among a plurality of external accessory devices to perform a first function, based on a determination that the computer system is in a first usage context; and a third optional user interface object, different from the second optional user interface object, when selected, causing a second external accessory device among a plurality of external accessory devices to perform a second function, wherein, when determined that the computer system is in the first usage context, the third optional user interface object is not included in the user interface.

[0026] An example transient computer-readable storage medium, an exemplary non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices, the one or more programs including instructions for performing the following operations: displaying via the display generation component a first user interface including a first plurality of optional user interface objects, wherein the first plurality of optional user interface objects include: a first optional user interface object that, when selected, causes the computer system to modify the state of the computer system; a second optional user interface object, when selected, causing a first external accessory device among a plurality of external accessory devices to perform a first function, based on determining that the computer system is in a first usage context; and a third optional user interface object, different from the second optional user interface object, when selected, causing a second external accessory device among a plurality of external accessory devices to perform a second function, wherein, when determined that the computer system is in the first usage context, the third optional user interface object is not included in the user interface.

[0027] An exemplary computer system includes one or more processors, wherein the computer system communicates with a display generation component and one or more input devices; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for performing the following operations: displaying, via the display generation component, a first user interface including a first plurality of optional user interface objects, wherein the first plurality of optional user interface objects include: a first optional user interface object that, when selected, causes the computer system to modify the state of the computer system; a second optional user interface object, when selected, causing a first external accessory device among a plurality of external accessory devices to perform a first function, based on a determination that the computer system is in a first usage context; and a third optional user interface object, different from the second optional user interface object, when selected, causing a second external accessory device among a plurality of external accessory devices to perform a second function, wherein, when determined that the computer system is in the first usage context, the third optional user interface object is not included in the user interface.

[0028] An exemplary computer system includes: means for displaying a first user interface including a first plurality of optional user interface objects via a display generating component, wherein the computer system communicates with the display generating component and one or more input devices, and wherein the first plurality of optional user interface objects include: a first optional user interface object that, when selected, causes the computer system to modify the state of the computer system; a second optional user interface object that, upon determining that the computer system is in a first usage context, causes a first external accessory device among a plurality of external accessory devices to perform a first function; and a third optional user interface object, different from the second optional user interface object, that, upon determining that the computer system is in a second usage context different from the first usage context, causes a second external accessory device among a plurality of external accessory devices to perform a second function, wherein, upon determining that the computer system is in the first usage context, the third optional user interface object is not included in the user interface.

[0029] An exemplary method includes, at a computer system communicating with a display generation component and one or more input devices: receiving data indicating the current media playback state of an external media playback device; and in response to receiving the data and according to a determined set of external media playback device control criteria being met, displaying via the display generation component a user interface for controlling media playback at the external media playback device, the user interface including: indicating, according to the determined data, that the external media playback device is currently performing a media playback operation; a first optional user interface object, which, when selected via one or more input devices, causes the external media playback device to modify the media playback operation; and indicating, according to the determined data, that the external media playback device is not currently performing a media playback operation; a second optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a first predetermined media item; and a third optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a second predetermined media item.

[0030] An exemplary non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with a display generation component and one or more input devices. The one or more programs include instructions for: receiving data indicating the current media playback state of an external media playback device; and, in response to receiving the data and according to a determined set of external media playback device control criteria being met, displaying via the display generation component a user interface for controlling media playback at the external media playback device. The user interface includes: indicating, based on the determined data, that the external media playback device is currently performing a media playback operation; a first optional user interface object, which, when selected via one or more input devices, causes the external media playback device to modify the media playback operation; and indicating, based on the determined data, that the external media playback device is not currently performing a media playback operation; a second optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a first predetermined media item; and a third optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a second predetermined media item.

[0031] An exemplary transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for: receiving data indicating the current media playback status of an external media playback device; and, in response to receiving the data and meeting a determined set of external media playback device control criteria, displaying via the display generation component a user interface for controlling media playback at the external media playback device, the user interface including: indicating, based on the determined data, that the external media playback device is currently... The media playback operation includes: a first optional user interface object, which, when selected via one or more input devices, causes the external media playback device to modify the media playback operation; and a second optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a first predetermined media item; and a third optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a second predetermined media item.

[0032] An exemplary computer system includes one or more processors, wherein the computer system communicates with a display generation component and one or more input devices; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for performing the following operations: receiving data indicating the current media playback state of an external media playback device; and, in response to receiving the data and according to a determined set of external media playback device control criteria being met, displaying via the display generation component a user interface for controlling media playback at the external media playback device, the user interface including: indicating, according to the determined data, that the external media playback device is currently performing a media playback operation; a first optional user interface object, which, when selected via one or more input devices, causes the external media playback device to modify the media playback operation; and indicating, according to the determined data, that the external media playback device is not currently performing a media playback operation; a second optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a first predetermined media item; and a third optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a second predetermined media item.

[0033] An exemplary computer system includes: means for receiving data indicating the current media playback state of an external media playback device, wherein the computer system communicates with a display generation component and one or more input devices; and means for displaying, via the display generation component, a user interface for controlling media playback at the external media playback device in response to receiving the data and according to a determined set of external media playback device control criteria being met, the user interface including: indicating, based on the determined data, that the external media playback device is currently performing a media playback operation; a first optional user interface object, which, when selected via one or more input devices, causes the external media playback device to modify the media playback operation; and indicating, based on the determined data, that the external media playback device is not currently performing a media playback operation; a second optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a first predetermined media item; and a third optional user interface object, which, when selected via one or more input devices, causes the external media playback device to initiate playback of a second predetermined media item.

[0034] An exemplary method includes, at a computer system communicating with a display generation component and one or more input devices, displaying a user interface via the display generation component, the user interface including: a first optional user interface object for controlling a first group of one or more media playback devices; a second optional user interface object for controlling a second group of one or more media playback devices different from the first group of one or more media playback devices; and an instruction to select the first optional user interface object; receiving user input corresponding to the selection of the second optional user interface object when the user interface includes the instruction; and, in response to receiving user input corresponding to the selection of the second optional user interface object and satisfying a determined set of criteria: displaying a third optional user interface object in the user interface for controlling a third group of one or more media playback devices including the first group of one or more media playback devices and the second group of one or more media playback devices; and stopping the display of the first optional user interface object and the second user interface object.

[0035] An exemplary non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for performing the following operations: displaying a user interface via the display generation component, the user interface including: a first optional user interface object for controlling a first group of one or more media playback devices; a second optional user interface object for controlling a second group of one or more media playback devices different from the first group of one or more media playback devices; and an instruction to select the first optional user interface object; receiving user input corresponding to the selection of the second optional user interface object when the user interface includes the instruction; and in response to receiving user input corresponding to the selection of the second optional user interface object and satisfying a determined set of criteria: displaying a third optional user interface object in the user interface for controlling a third group of one or more media playback devices including the first group of one or more media playback devices and the second group of one or more media playback devices; and stopping the display of the first optional user interface object and the second user interface object.

[0036] An exemplary transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices. The one or more programs include instructions for performing the following operations: displaying a user interface via the display generation component, the user interface including: a first optional user interface object for controlling a first group of one or more media playback devices; a second optional user interface object for controlling a second group of one or more media playback devices different from the first group of one or more media playback devices; and an instruction to select the first optional user interface object; receiving user input corresponding to the selection of the second optional user interface object when the user interface includes the instruction; and in response to receiving user input corresponding to the selection of the second optional user interface object and satisfying a determined set of criteria: displaying a third optional user interface object in the user interface for controlling a third group of one or more media playback devices including the first group of one or more media playback devices and the second group of one or more media playback devices; and stopping the display of the first optional user interface object and the second user interface object.

[0037] An exemplary computer system includes one or more processors, wherein the computer system communicates with a display generation component and one or more input devices; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for performing the following operations: displaying a user interface via the display generation component, the user interface including: a first optional user interface object for controlling a first group of one or more media playback devices; a second optional user interface object for controlling a second group of one or more media playback devices different from the first group of one or more media playback devices; and an instruction to select the first optional user interface object; receiving user input corresponding to the selection of the second optional user interface object when the user interface includes the instruction; and in response to receiving user input corresponding to the selection of the second optional user interface object and satisfying a determined set of criteria: displaying a third optional user interface object in the user interface for controlling a third group of one or more media playback devices including the first group of one or more media playback devices and the second group of one or more media playback devices; and stopping the display of the first optional user interface object and the second user interface object.

[0038] An exemplary computer system includes: means for displaying a user interface via a display generation component, the user interface including: a first optional user interface object for controlling a first group of one or more media playback devices; a second optional user interface object for controlling a second group of one or more media playback devices different from the first group of one or more media playback devices; and an indication for selecting the first optional user interface object; means for receiving user input corresponding to the selection of the second optional user interface object when the user interface includes the indication; and means for displaying the following items in the user interface in response to receiving user input corresponding to the selection of the second optional user interface object and upon satisfaction of a determined set of criteria: a third optional user interface object for controlling a third group of one or more media playback devices including the first group of one or more media playback devices and the second group of one or more media playback devices; and stopping the display of the first optional user interface object and the second user interface object.

[0039] An exemplary method includes, at a computer system: while the computer system is providing media to one or more devices in a first group, initiating a first process for providing media to one or more devices in a second group, while continuing to provide media to the one or more devices in the first group, the first process including: receiving from a first external device communicating with the one or more devices in the second group first data requesting a first request to perform the first process to provide media to the one or more devices in the second group; and receiving from a second external device communicating with the one or more devices in the second group second data requesting a second request to provide media to the one or more devices in the second group, wherein the second data is received from a second external device communicating with the one or more devices in the second group or from a first device in the one or more devices in the second group; and after receiving the first data and the second data, providing media to the one or more devices in the second group while continuing to provide media to the one or more devices in the first group.

[0040] An exemplary non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors, the one or more programs including instructions for performing the following operations: while a computer system is providing media to one or more devices in a first group, initiating a first process of providing media to one or more devices in a second group, while continuing to provide media to the one or more devices in the first group, the first process including: receiving first data from a first external device communicating with the one or more devices in the second group, instructing to perform the first process to provide media to the one or more devices in the second group; and receiving second data instructing to perform the first process to provide media to the one or more devices in the second group, wherein the second data is received from a second external device communicating with the one or more devices in the second group or from a first device in the one or more devices in the second group; and after receiving the first data and the second data, providing media to the one or more devices in the second group, while continuing to provide media to the one or more devices in the first group.

[0041] An exemplary transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system, the one or more programs including instructions for performing the following operations: while the computer system is providing media to one or more devices in the first group, initiating a first process of providing media to one or more devices in the second group, while continuing to provide media to the one or more devices in the first group, the first process including: receiving first data from a first external device communicating with the one or more devices in the second group, instructing to perform the first process to provide media to the one or more devices in the second group; and receiving second data instructing to perform the first process to provide media to the one or more devices in the second group, wherein the second data is received from a second external device communicating with the one or more devices in the second group or from a first device in the one or more devices in the second group; and after receiving the first data and the second data, providing media to the one or more devices in the second group, while continuing to provide media to the one or more devices in the first group.

[0042] An exemplary computer system includes one or more processors and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for performing the following operations: while the computer system is providing media to a first group of one or more devices, initiating a first process for providing media to a second group of one or more devices, while continuing to provide media to the first group of one or more devices, the first process including: receiving from a first external device communicating with the second group of one or more devices the first data indicating a first request to perform the first process to provide media to the second group of one or more devices; and receiving second data indicating a second request to perform the first process to provide media to the second group of one or more devices, wherein the second data is received from a second external device communicating with the second group of one or more devices or from the first device in the second group of one or more devices; and after receiving the first data and the second data, providing media to the second group of one or more devices while continuing to provide media to the first group of one or more devices.

[0043] An exemplary computer system includes: means for initiating a first process of providing media to a second group of one or more devices while the computer system is providing media to a first group of one or more devices, and for continuing to provide media to the first group of one or more devices, the means including: means for receiving first data from a first external device communicating with the second group of one or more devices, instructing to perform the first process to provide media to the second group of one or more devices; and means for receiving second data instructing to perform the first process to provide media to the second group of one or more devices, wherein the second data is received from a second external device communicating with the second group of one or more devices or from the first device in the second group of one or more devices; and means for continuing to provide media to the first group of one or more devices while providing media to the second group of one or more devices after receiving the first data and the second data.

[0044] An exemplary method includes, at a computer system communicating with one or more input devices: receiving a first request from a first user to perform an operation; in response to the first request, initiating the execution of the first operation, wherein the first operation is performed based on at least a first performance parameter; after initiating the execution of the first operation, receiving a second request to perform the operation via the one or more input devices; and in response to receiving the second request: initiating execution based on determining that the second request was made by the first user, wherein the second operation is performed based on at least the first performance parameter; and initiating the execution of a third operation different from the first operation based on determining that the second request was made by a second user different from the first user.

[0045] An example non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more communication devices. The one or more programs include instructions for: receiving a first request from a first user to perform an operation; in response to the first request, initiating execution of the first operation, wherein the first operation is performed based on at least a first performance parameter; after initiating execution of the first operation, receiving a second request to perform the operation via the one or more input devices; and in response to receiving the second request: initiating execution based on determining that the second request was made by the first user, wherein the second operation is performed based on at least the first performance parameter; and initiating execution of a third operation different from the first operation based on determining that the second request was made by a second user different from the first user.

[0046] An example transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of an electronic device having a display device and one or more communication devices, the one or more programs including instructions for: receiving a first request from a first user to perform an operation; in response to the first request, initiating execution of the first operation, wherein the first operation is performed based on at least a first performance parameter; after initiating execution of the first operation, receiving a second request to perform the operation via the one or more input devices; and in response to receiving the second request: initiating execution based on determining that the second request was made by the first user, wherein the second operation is performed based on at least the first performance parameter; and initiating execution of a third operation different from the first operation based on determining that the second request was made by a second user different from the first user.

[0047] An exemplary computer system includes one or more processors, wherein the computer system communicates with a display generation component and one or more input devices; and a memory storing one or more programs configured to be executed by the one or more processors. The one or more programs include instructions for performing the following operations: receiving a first request from a first user to perform an operation; in response to the first request, initiating execution of the first operation, wherein the first operation is performed based on at least a first performance parameter; after initiating execution of the first operation, receiving a second request to perform the operation via the one or more input devices; and in response to receiving the second request: initiating execution based on determining that the second request was made by the first user, wherein the second operation is performed based on at least the first performance parameter; and initiating execution of a third operation different from the first operation based on determining that the second request was made by a second user different from the first user.

[0048] An exemplary computer system includes: means for receiving a first request from a first user to perform an operation; means for initiating the execution of the first operation in response to the first request, wherein the first operation is performed based on at least a first performance parameter; means for receiving a second request to perform the operation via the one or more input devices after initiating the execution of the first operation; and means for performing the following operations in response to receiving the second request: initiating execution based on determining that the second request was made by the first user, wherein the second operation is performed based on at least the first performance parameter; and initiating the execution of a third operation different from the first operation based on determining that the second request was made by a second user different from the first user.

[0049] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0050] Therefore, providing devices with faster and more efficient methods and interfaces for controlling audio playback improves the effectiveness, efficiency, and user satisfaction of such devices. These methods and interfaces can complement or replace other methods used to control audio playback. Attached Figure Description

[0051] To better understand the various embodiments described, reference should be made to the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals indicate corresponding parts in all the drawings.

[0052] Figure 1A This is a block diagram illustrating a portable multi-functional device with a touch-sensitive display according to some embodiments.

[0053] Figure 1B This is a block diagram illustrating exemplary components for event handling according to some implementation schemes.

[0054] Figure 2 A portable multi-functional device with a touchscreen is shown according to some embodiments.

[0055] Figure 3 This is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface according to some implementation schemes.

[0056] Figure 4A An exemplary user interface for a menu on an application on a portable multifunction device, according to some implementation schemes, is shown.

[0057] Figure 4B An exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0058] Figure 5A A personal electronic device according to some implementation schemes is shown.

[0059] Figure 5B This is a block diagram illustrating a personal electronic device according to some implementation schemes.

[0060] Figures 5C-5D Exemplary components of a personal electronic device having a touch-sensitive display and an intensity sensor according to some embodiments are shown.

[0061] Figures 5E-5H Exemplary components and user interfaces of a personal electronic device according to some implementation schemes are shown.

[0062] Figure 5I An electronic device according to some implementation schemes is shown.

[0063] Figure 5J This is a block diagram illustrating an electronic device according to some implementation schemes.

[0064] Figures 6A to 6P An exemplary user interface according to some implementation schemes is shown.

[0065] Figure 7 Exemplary methods according to some implementation schemes are shown.

[0066] Figure 8 Exemplary methods according to some implementation schemes are shown.

[0067] Figures 9A to 9D An exemplary user interface according to some implementation schemes is shown.

[0068] Figure 10 Exemplary methods according to some implementation schemes are shown.

[0069] Figures 11A to 11D An exemplary user interface according to some implementation schemes is shown.

[0070] Figure 12 This is a diagram illustrating a set of exemplary devices according to some implementation schemes.

[0071] Figures 13A to 13N An exemplary user interface according to some implementation schemes is shown.

[0072] Figure 14 Exemplary methods according to some implementation schemes are shown.

[0073] Figures 15A to 15L An exemplary user interface according to some implementation schemes is shown.

[0074] Figure 16 Exemplary methods according to some implementation schemes are shown.

[0075] Figures 17A to 17P An exemplary user interface according to some implementation schemes is shown.

[0076] Figure 18 Exemplary methods according to some implementation schemes are shown.

[0077] Figures 19A to 19I An exemplary user interface according to some implementation schemes is shown.

[0078] Figure 20 Exemplary methods according to some implementation schemes are shown.

[0079] Figures 21A to 21F An exemplary user interface according to some implementation schemes is shown.

[0080] Figures 22A to 22F An exemplary user interface according to some implementation schemes is shown.

[0081] Figure 23 Exemplary methods according to some implementation schemes are shown. Detailed Implementation

[0082] The following description illustrates exemplary methods, parameters, etc. However, it should be understood that such description is not intended to limit the scope of this disclosure, but is provided as a description of exemplary embodiments.

[0083] Electronic devices are needed to provide efficient methods and interfaces for controlling audio playback. The techniques described below allow users to transfer audio playback from one device to another by bringing the two devices close together. For example, when playing music on a phone, a user can place the phone close to a wireless speaker to transfer music playback to the wireless speaker, which provides better sound quality compared to the phone itself. When the phone is close to the speaker, a notification is displayed on the phone (e.g., at the top of the phone's display) to inform the user that music can be played on the speaker. The notification can be tapped to initiate music playback on the speaker. Instead of a swipe gesture to initiate playback, a pull-down notification provides an interface with additional information, such as a queue of recent and / or scheduled media items that can be selected for playback on the speaker. Audio media playing on the speaker can be transferred to the phone in a similar manner. When audio is playing on the speaker, bringing the phone close to the speaker causes the phone to display a notification similar to the one described above, except that tapping the notification transfers the audio from the speaker to the phone, and swiping the notification displays additional information about the media being played on the speaker. In some implementations, when displaying a notification, the phone can be brought closer to the speaker to initiate playback without touch input on the phone's display. Such technology can reduce the cognitive load on the user initiating playback of audio media, thereby improving productivity. Furthermore, such technology can reduce processor and battery power otherwise wasted on redundant user input.

[0084] under Figures 1A to 1B , Figure 2 , Figure 3 , Figures 4A to 4B and Figures 5A to 5J A description of an exemplary device for performing management event notifications is provided. Figures 6A to 6P and Figures 11A to 11D An exemplary user interface for controlling audio playback is shown. Figures 7 to 8 This is a flowchart illustrating a method for controlling audio playback according to some implementation schemes. Figures 6A to 6P and Figures 11A to 11D The user interface in the document is used to illustrate the process described below, including Figures 7 to 8 The process is shown. Figures 9A to 9D and Figures 11A to 11D An exemplary user interface for controlling audio playback is shown. Figure 10 This is a flowchart illustrating a method for controlling audio playback according to some implementation schemes. Figures 9A to 9D and Figures 11A to 11D The user interface in the document is used to illustrate the process described below, including Figure 10 The process is shown. Figure 12 This is a diagram illustrating a set of exemplary devices according to some implementation schemes. Figure 12 The diagram in the figure is used to illustrate the process described below, including Figure 14 , 18 The processes shown in 20 and 23. Figures 13A to 13N An exemplary user interface for managing controls is shown according to some implementation schemes. Figure 14 This is a flowchart illustrating a method for managing controls according to some implementation schemes. Figures 13A to 13N The user interface in the document is used to illustrate the process described below, including Figure 14 The process is shown. Figures 15A to 15L An exemplary user interface for controlling audio playback is shown according to some implementation schemes. Figure 16 This is a flowchart illustrating a method for controlling audio playback according to some implementation schemes. Figures 15A to 15L The user interface in the document is used to illustrate the process described below, including Figure 16 The process is shown. Figures 17A to 17P An exemplary user interface for managing controls for controlling audio playback on a set of devices, according to some implementation schemes, is shown. Figure 18 This is a flowchart illustrating a method for managing controls used to control audio playback on a set of devices. Figures 17A to 17P The user interface in the document is used to illustrate the process described below, including Figure 18 The process is shown. Figures 19A to 19I An exemplary user interface for sharing media is shown according to some implementation schemes. Figure 20 This is a flowchart illustrating a method for sharing media. Figures 19A to 19I The user interface in the document is used to illustrate the process described below, including Figure 20 The process is shown. Figures 21A to 21F An exemplary user interface for managing voice input is shown according to some implementation schemes. Figures 22A to 22F An exemplary user interface for managing voice input is shown according to some implementation schemes. Figure 23This is a flowchart illustrating a method for managing voice input according to some implementation schemes. Figures 21A to 21F and Figures 22A to 22F The user interface in the document is used to illustrate the process described below, including Figure 23 The process is shown.

[0085] Although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, a first touch may be named a second touch and similarly, a second touch may be named a first touch, without departing from the scope of the various described embodiments. Both the first touch and the second touch are touches, but they are not the same touch.

[0086] The terminology used in the description of the various embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It will also be understood that the terms “includes”, “including”, “comprises”, and / or “comprising”, when used in this specification, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0087] Depending on the context, the term "if" may optionally be interpreted as meaning "when," "in response to," or "in response to detection." Similarly, depending on the context, the phrases "if it is determined..." or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "in response to determining..." or "in response to detecting [the stated condition or event]."

[0088] This document describes implementations of electronic devices, user interfaces for such devices, and related processes for using such devices. In some implementations, the device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and / or music player functionality. Exemplary implementations of portable multi-functional devices include, but are not limited to, those from Apple Inc. (Cupertino, California). Devices, iPod Equipment, and Device. Optionally, other portable electronic devices may be used, such as laptops or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). In some embodiments, the electronic device is a computer system that communicates with a display generating component (e.g., via wireless or wired communication). The display generating component is configured to provide visual output, such as a display via a CRT monitor, a display via an LED monitor, or a display via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. As used herein, “display” content includes displaying content (e.g., video data rendered or decoded by display controller 156) by transmitting data (e.g., image data or video data) to an integrated or external display generating component via a wired or wireless connection to visually generate content.

[0089] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device may optionally include one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0090] The device typically supports a variety of applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, phone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camcorder applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0091] Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the device's common physical architecture (such as the touch-sensitive surface) optionally utilizes a user interface that is intuitive and clear to the user to support various applications.

[0092] Now let’s turn our attention to implementation schemes for portable devices with touch-sensitive displays. Figure 1AThis is a block diagram illustrating a portable multi-functional device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a “touchscreen” for convenience, and is sometimes referred to as or called a “touch-sensitive display system.” Device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more haptic output generators 167 for generating haptic output on device 100 (e.g., generating haptic output on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0093] As used in this specification and claims, the term "intensity" of contact on a tactile surface refers to the force or pressure (force per unit area) of a contact (e.g., finger contact) on a tactile surface, or to a substitute (alternative) for the force or pressure of a contact on a tactile surface. The intensity of contact has a range of values ​​that includes at least four different values ​​and more typically hundreds of different values ​​(e.g., at least 256). The intensity of contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the tactile surface are optionally used to measure the force at different points on the tactile surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the tactile surface. Alternatively, the size and / or variation of the contact area detected on the touch-sensitive surface, the capacitance and / or variation of the touch-sensitive surface near the contact, and / or the resistance and / or variation of the touch-sensitive surface near the contact may optionally be used as substitutes for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold (e.g., the intensity threshold is described in units corresponding to the substitute measurement) has been exceeded. In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold (e.g., the intensity threshold is a pressure threshold measured in units of pressure) has been exceeded. Using the intensity of the contact as an attribute of user input allows the user to access additional device functions that would otherwise be inaccessible to the user on smaller devices with limited physical space, such smaller devices being used (e.g., on a touch-sensitive display) to display power indications and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0094] As used in this specification and claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device, which is detected by the user using the user's tactile sense. For example, when the device or a component of the device comes into contact with a touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or touchpad) may optionally be interpreted by the user as a "press-click" or "release-click" on a physically actuated button. In some cases, the user will feel a tactile sensation, such as a "press-click" or "release-click," even when a physically actuated button associated with a touch-sensitive surface that has been physically pressed (e.g., displaced) by the user's movement does not move. For example, even when the smoothness of the tactile surface remains unchanged, the movement of the tactile surface can optionally be interpreted or sensed by the user as the "roughness" of the tactile surface. While such interpretations of touch by users will be limited by the individualized sensory perceptions of the user, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or ordinary) user.

[0095] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 may optionally have more or fewer components than those shown, may optionally combine two or more components, or may optionally have different configurations or arrangements of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both, including one or more signal processing and / or application-specific integrated circuits.

[0096] Memory 102 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 122 optionally controls other components of device 100 to access memory 102.

[0097] Peripheral interface 118 can be used to couple the device's input and output peripherals to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data. In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0098] RF (Radio Frequency) circuit 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuit 108 converts electrical signals into electromagnetic signals and vice versa, and communicates with communication networks and other communication devices via these electromagnetic signals. RF circuit 108 optionally includes well-known circuitry for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, Subscriber Identity Module (SIM) cards, memory, etc. RF circuit 108 optionally communicates wirelessly with networks and other devices, such as the Internet (also known as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). RF circuit 108 optionally includes well-known circuitry for detecting near-field communication (NFC) fields, such as via near-field communication radio components. Wireless communication may optionally employ any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed ​​Downlink Packet Access (HSDPA), High-Speed ​​Uplink Packet Access (HSUPA), Evolution, Pure Data (EV-DO), HSPA, HSPA+, Dual-Unit HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), and Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE...). 802.11n and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Messaging Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence with Extended Utility (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol that has not been developed as of the date of this document submission.

[0099] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and device 100. Audio circuitry 110 receives audio data from peripheral interface 118, converts the audio data into electrical signals, and transmits the electrical signals to speaker 111. Speaker 111 converts the electrical signals into sound waves that are audible to humans. Audio circuitry 110 also receives electrical signals converted from sound waves by microphone 113. Audio circuitry 110 converts the electrical signals into audio data and transmits the audio data to peripheral interface 118 for processing. Audio data is optionally retrieved by peripheral interface 118 from and / or transmitted to memory 102 and / or RF circuitry 108. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., ...). Figure 2 (212 in the text). The headset jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset with both output (e.g., a single-ear or dual-ear headphone) and input (e.g., a microphone).

[0100] I / O subsystem 106 couples input / output peripherals on device 100, such as touchscreen 112 and other input control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, depth camera controller 169, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive electrical signals from / send electrical signals to the other input control device 116. The other input control device 116 optionally includes physical buttons (e.g., push-buttons, rocker buttons, etc.), dial pads, slide switches, joysticks, click dials, etc. In some embodiments, input controller 160 is optionally coupled to (or not coupled to) any of the following: keyboard, infrared port, USB port, and pointing device such as mouse. One or more buttons (e.g., Figure 2 Optionally, 208) includes an increase / decrease button for volume control of speaker 111 and / or microphone 113. The one or more buttons optionally include a push-button (e.g., Figure 2(Ref. 206 in the original text). In some embodiments, the electronic device is a computer system that communicates with one or more input devices (e.g., via wireless communication, via wired communication). In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a touchpad, as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), such as for tracking user gestures (e.g., hand gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system.

[0101] A quick press of the push button optionally disengages the touchscreen 112 from its lock or optionally initiates a process of unlocking the device using gestures on the touchscreen, as described in U.S. Patent Application 11 / 322,549 (i.e., U.S. Patent No. 7,657,849), filed December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image," the entire contents of which are incorporated herein by reference. A long press of the push button (e.g., 206) optionally powers the device 100 on or off. The function of one or more buttons is optionally user-customizable. The touchscreen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0102] The touch-sensitive display 112 provides input and output interfaces between the device and the user. The display controller 156 receives electrical signals from and / or sends electrical signals to the touchscreen 112. The touchscreen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively, "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0103] Touchscreen 112 has a touch-sensitive surface, sensor, or sensor array that accepts input from a user based on tactile and / or haptic contact. Touchscreen 112 and display controller 156 (along with any associated modules and / or instruction set in memory 102) detect contact on touchscreen 112 (and any movement or interruption of that contact) and translate the detected contact into interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touchscreen 112. In an exemplary embodiment, the contact point between touchscreen 112 and the user corresponds to the user's finger.

[0104] Touchscreen 112 optionally employs LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, but other display technologies are used in other embodiments. Touchscreen 112 and display controller 156 optionally employ any of a variety of touch sensing technologies now known or to be developed hereafter, along with other proximity sensor arrays or other elements for determining one or more points of contact with touchscreen 112, to detect contact and any movement or interruption thereof. These various touch sensing technologies include, but are not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as that from Apple Inc. (Cupertino, California). and iPod The technology used.

[0105] In some embodiments of the touchscreen 112, the touch-sensitive display optionally resembles a multi-touch touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is incorporated herein by reference in its entirety. However, the touchscreen 112 displays visual output from the device 100, while the touch-sensitive touchpad does not provide visual output.

[0106] In some embodiments, the touchscreen 112 has a touch-sensitive display as described in the following patent applications: (1) U.S. Patent Application No. 11 / 381,313, filed May 2, 2006, entitled “Multipoint Touch Surface Controller”; (2) U.S. Patent Application No. 10 / 840,862, filed May 6, 2004, entitled “Multipoint Touchscreen”; (3) U.S. Patent Application No. 10 / 903,964, filed July 30, 2004, entitled “Gestures For Touch Sensitive Input Devices”; (4) U.S. Patent Application No. 11 / 048,264, filed January 31, 2005, entitled “Gestures For Touch Sensitive Input Devices”; and (5) U.S. Patent Application No. 18, 2005, entitled “Mode-Based Graphical User Interfaces For Touch Sensitive”. (6) U.S. Patent Application No. 11 / 038,590 entitled “Input Devices”; (7) U.S. Patent Application No. 11 / 228,758 entitled “Virtual Input Device Placement On A Touch Screen User Interface”, filed September 16, 2005; (8) U.S. Patent Application No. 11 / 228,700 entitled “Operation Of A Computer With A Touch Screen Interface”, filed September 16, 2005; (9) U.S. Patent Application No. 11 / 228,737 entitled “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard”, filed September 16, 2005; and (10) U.S. Patent Application No. 11 / 367,749 entitled “Multi-Functional Hand-Held Device”, filed March 3, 2006. The full text of all these applications is incorporated herein by reference.

[0107] Touchscreen 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touchscreen has a video resolution of approximately 160 dpi. Users optionally interact with touchscreen 112 using any suitable object or accessory such as a stylus, finger, etc. In some embodiments, the user interface is designed to operate primarily through finger-based touch and gestures, which may be less precise than stylus-based input due to the larger contact area of ​​a finger on the touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor positions or commands to perform the user-desired actions.

[0108] In some embodiments, in addition to the touchscreen, device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touchscreen, does not display visual output. Optionally, the touchpad is a touch-sensitive surface separate from the touchscreen 112, or an extension of the touch-sensitive surface formed by the touchscreen.

[0109] The device 100 also includes a power system 162 for supplying power to various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power in the portable device.

[0110] The device 100 may optionally also include one or more optical sensors 164. Figure 1AAn optical sensor 164 is shown coupled to an optical sensor controller 158 in the I / O subsystem 106. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with an imaging module 143 (also called a camera module), the optical sensor 164 optionally captures still images or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to a touchscreen display 112 on the front of the device, allowing the touchscreen display to be used as a viewfinder for still image and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device, allowing images of the user to be optionally acquired for video conferencing while the user views other video conferencing participants on the touchscreen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (e.g., by rotating the lenses and sensor within the device housing), allowing a single optical sensor 164 to be used in conjunction with the touchscreen display for both video conferencing and still image and / or video image acquisition.

[0111] The device 100 optionally also includes one or more depth camera sensors 175. Figure 1A A depth camera sensor is shown coupled to a depth camera controller 169 in I / O subsystem 106. Depth camera sensor 175 receives data from the environment to create a 3D model of an object (e.g., a face) within the scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with imaging module 143 (also referred to as a camera module), depth camera sensor 175 may optionally be used to determine depth maps of different portions of an image captured by imaging module 143. In some embodiments, the depth camera sensor is located at the front of device 100, such that user images with depth information are optionally acquired for video conferencing while the user views other video conferencing participants on a touchscreen display, and selfies with depth map data are captured. In some embodiments, depth camera sensor 175 is located at the rear of the device, or both the rear and front of device 100. In some embodiments, the position of depth camera sensor 175 may be changed by the user (e.g., by rotating a lens and sensor within the device housing), such that depth camera sensor 175 is used in conjunction with a touchscreen display for both video conferencing and still image and / or video image acquisition.

[0112] The device 100 may optionally also include one or more contact strength sensors 165. Figure 1AA contact strength sensor is shown coupled to a strength sensor controller 159 in I / O subsystem 106. The contact strength sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-force sensors, piezoelectric sensors, optical force sensors, capacitive touch-sensitive surfaces, or other strength sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact strength sensor 165 receives contact strength information (e.g., pressure information or a substitute for pressure information) from the environment. In some embodiments, at least one contact strength sensor is arranged juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact strength sensor is located on the rear of device 100, opposite to the touchscreen display 112 located on the front of device 100.

[0113] The device 100 optionally also includes one or more proximity sensors 166. Figure 1A A proximity sensor 166 coupled to a peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 may optionally be coupled to an input controller 160 in an I / O subsystem 106. The proximity sensor 166 may optionally perform as described in the following U.S. patent applications: No. 11 / 241,839, entitled "Proximity Detector In Handheld Device"; No. 11 / 240,788, entitled "Proximity Detector In Handheld Device"; No. 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; No. 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and No. 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", the entire contents of which are incorporated herein by reference. In some implementations, the proximity sensor is turned off and the touchscreen 112 is disabled when the multifunction device is placed near the user's ear (e.g., when the user is making a phone call).

[0114] The device 100 may optionally also include one or more tactile output generators 167. Figure 1AA haptic output generator coupled to a haptic feedback controller 161 in I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components for converting electrical signals into haptic outputs on the device). A contact intensity sensor 165 receives haptic feedback generation instructions from a haptic feedback module 133 and generates a haptic output on device 100 that can be felt by a user of device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a haptic surface (e.g., haptic display system 112) and optionally generates the haptic output by moving the haptic surface vertically (e.g., in / outward from the surface of device 100) or laterally (e.g., backward and forward in the same plane as the surface of device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touch screen display 112 located on the front of the device 100.

[0115] The device 100 may optionally also include one or more accelerometers 168. Figure 1A An accelerometer 168 coupled to a peripheral device interface 118 is shown. Alternatively, the accelerometer 168 may be coupled to an input controller 160 in an I / O subsystem 106. The accelerometer 168 may optionally perform as described in the following U.S. patent publications: U.S. Patent Publication No. 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and U.S. Patent Publication No. 20060017692, entitled "Methods and Apparatuses For Operating A Portable Device Based On An Accelerometer," both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed on a touchscreen display in portrait or landscape view based on analysis of data received from one or more accelerometers. Device 100 may optionally include, in addition to the accelerometer 168, a magnetometer and a GPS (or GLONASS or other global navigation system) receiver for acquiring information about the location and orientation (e.g., portrait or landscape) of device 100.

[0116] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and an application program (or instruction set) 136. Furthermore, in some embodiments, memory 102 ( Figure 1A ) or 370 ( Figure 3 Storage device / global internal state 157, such as Figure 1A and Figure 3 As shown in the figure. Device / global internal state 157 includes one or more of the following: active application state, which indicates which applications (if any) are currently active; display state, which indicates what applications, views or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and position information about the device's position and / or orientation.

[0117] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or embedded operating systems such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0118] The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is connected to… (Trademark of Apple Inc.) The same or similar and / or compatible multi-pin (e.g., 30-pin) connectors used in Apple Inc. devices.

[0119] The contact / motion module 130 optionally detects contact with the touchscreen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., touchpads or physical click-based rotary dials). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether a contact has occurred (e.g., detecting a finger press event), determining the contact intensity (e.g., the force or pressure of the contact, or an alternative to force or pressure), determining whether there is movement of the contact and tracking movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or a contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple-finger contact). In some implementations, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0120] In some implementations, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., determining whether the user has “clicked” an icon). In some implementations, at least a subset of the intensity thresholds is determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a specific physical actuator and can be adjusted without changing the physical hardware of device 100). For example, the mouse “click” threshold of a touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds without changing the touchpad or touchscreen display hardware. Additionally, in some specific implementations, the user of the device is provided with software settings for adjusting one or more intensity thresholds in a set (e.g., by adjusting the individual intensity thresholds and / or by adjusting multiple intensity thresholds at once using system-level clicks on the “intensity” parameter).

[0121] The touch / motion module 130 optionally detects gesture input performed by the user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timings, and / or intensities of the detected contact). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event and then detecting a finger lift-off (lift-away) event at the same (or substantially the same) location as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and subsequently detecting a finger lift-off (lift-away) event.

[0122] The graphics module 132 includes various known software components for rendering and displaying graphics on the touchscreen 112 or other display, including components for altering the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual properties). As used herein, the term "graphics" includes any object that can be displayed to a user, and non-limitingly includes text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0123] In some implementations, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes from applications, etc., specifying the graphics to be displayed, and also receives coordinate data and other graphic attribute data if necessary, and then generates screen image data for output to the display controller 156.

[0124] The haptic feedback module 133 includes various software components for generating instructions which are used by the haptic output generator 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0125] Optionally, the text input module 134, a component of the graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application that requires text input).

[0126] GPS module 135 determines the location of the device and provides that information for use in various applications (e.g., to phone 138 for use in location-based dialing; to camera 143 as image / video metadata; and to applications that provide location-based services, such as weather desktop apps, local yellow pages desktop apps, and map / navigation desktop apps).

[0127] Application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0128] • Contacts module 137 (sometimes called address book or contact list);

[0129] • Telephone module 138;

[0130] • Video conferencing module 139;

[0131] • Email client module 140;

[0132] • Instant Messaging (IM) module 141;

[0133] Fitness support module 142;

[0134] • Camera module 143 for still images and / or video images;

[0135] • Image management module 144;

[0136] • Video player module;

[0137] Music player module;

[0138] • Browser module 147;

[0139] • Calendar module 148;

[0140] • Desktop mini-program module 149, which optionally includes one or more of the following: weather desktop mini-program 149-1, stock market desktop mini-program 149-2, calculator desktop mini-program 149-3, alarm clock desktop mini-program 149-4, dictionary desktop mini-program 149-5, and other desktop mini-programs obtained by the user, and desktop mini-programs created by the user 149-6;

[0141] • Desktop app creator module 150 for creating user-created desktop apps 149-6;

[0142] • Search module 151;

[0143] • Video and music player module 152, which combines a video player module and a music player module;

[0144] • Notepad module 153;

[0145] • Map module 154; and / or

[0146] • Online video module 155.

[0147] Examples of other applications 136 that may be optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, Java-enabled applications, encryption, digital rights management, speech recognition, and speech duplication.

[0148] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (e.g., in the application internal state 192 of the contact module 137 stored in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing module 139, email 140, or IM 141; and so on.

[0149] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, telephone module 138 is optionally used to input character sequences corresponding to telephone numbers, access one or more telephone numbers in contact module 137, modify entered telephone numbers, dial corresponding telephone numbers, initiate conversations, and disconnect or hang up when a conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0150] Combining RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, video conferencing module 139 includes executable instructions to initiate, conduct, and terminate video conferences between the user and one or more other participants based on user instructions.

[0151] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user commands. Combined with image management module 144, email client module 140 makes it very easy to create and send emails containing still images or video images captured by camera module 143.

[0152] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, instant messaging module 141 includes executable instructions for: inputting a character sequence corresponding to an instant message, modifying previously input characters, transmitting a corresponding instant message (e.g., using Short Message Service (SMS) or Multimedia Messaging Service (MMS) protocols for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for internet-based instant messaging), receiving an instant message, and viewing a received instant message. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photographs, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Services (EMS). As used herein, "instant message" refers to both telephone-based messages (e.g., messages sent using SMS or MMS) and internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0153] Incorporating RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, fitness support module 142 includes executable instructions for creating fitness activities (e.g., with time, distance, and / or calorie burning goals); communicating with fitness sensors (executive devices); receiving fitness sensor data; calibrating sensors used to monitor fitness; selecting and playing music for fitness activities; and displaying, storing, and transmitting fitness data.

[0154] In conjunction with the touchscreen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for: capturing still images or videos (including video streams) and storing them in memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from memory 102.

[0155] Incorporating the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0156] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions for browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as links to attachments and other files on web pages.

[0157] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and associated data (e.g., calendar entries, to-dos, etc.) according to user instructions.

[0158] In conjunction with RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, desktop applet module 149 is optionally a micro-application downloaded and used by a user (e.g., weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, and dictionary desktop applet 149-5) or a user-created micro-application (e.g., user-created desktop applet 149-6). In some embodiments, the desktop applet includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop applet includes XML (Extensible Markup Language) files and JavaScript files (e.g., Yahoo! desktop applet).

[0159] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and browser module 147, the desktop applet creator module 150 can optionally be used by the user to create desktop applets (e.g., converting user-specified portions of a webpage into desktop applets).

[0160] In conjunction with the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to user instructions.

[0161] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions allowing users to download and play recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touchscreen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0162] Combining the touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do items, etc., according to user instructions.

[0163] Combining RF circuitry 108, touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 is optionally used to receive, display, modify, and store maps and map-related data (e.g., driving directions, data related to shops and other points of interest at or near a specific location, and other location-based data) according to user instructions.

[0164] Incorporating touchscreen 112, display controller 156, touch / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following actions: allowing users to access, browse, receive (e.g., via streaming and / or downloading), play back (e.g., on the touchscreen or on an external display connected via external port 124), send emails with links to specific online videos, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, an instant messaging module 141 is used instead of the email client module 140 to send links to specific online videos. Further descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are incorporated herein by reference in their entirety.

[0165] Each of the above modules and applications corresponds to an executable instruction set for performing one or more of the functions described above and the methods described in this patent application (e.g., computer-implemented methods and other information processing methods as described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs, processes, or modules; therefore, various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., Figure 1A (e.g., video and music player module 152). In some embodiments, memory 102 optionally stores subgroups of the aforementioned modules and data structures. Additionally, memory 102 optionally stores other modules and data structures not described above.

[0166] In some implementations, device 100 is a device on which the operation of a predefined set of functions is performed solely via a touchscreen and / or touchpad. By using a touchscreen and / or touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push-buttons, dials, etc.) on device 100 can be optionally reduced.

[0167] A predefined set of functions, uniquely performed via a touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 from any user interface displayed on device 100 to the main menu, home menu, or root menu. In such embodiments, a "menu button" is implemented using a touchpad. In some other embodiments, the menu button is a physical push-button or other physical input control device, rather than a touchpad.

[0168] Figure 1B This is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 This includes an event classifier 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 137 to 151, 155, 380 to 390).

[0169] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information should be delivered. Event classifier 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine the application view 191 to which the event information should be delivered.

[0170] In some implementations, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or ready to be displayed by the application 136-1, a state queue for enabling the user to return to the previous state or view of the application 136-1, and a repeat / undo queue for the user's previous actions.

[0171] Event monitor 171 receives event information from peripheral device interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripheral device interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (via audio circuitry 110). The information received by peripheral device interface 118 from I / O subsystem 106 includes information from touch-sensitive display 112 or touch-sensitive surfaces.

[0172] In some implementations, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other implementations, peripheral device interface 118 transmits event information only when a significant event occurs (e.g., receiving input above a predetermined noise threshold and / or receiving input for a predetermined duration).

[0173] In some implementations, the event classifier 170 also includes a hit view determination module 172 and / or an activity event recognizer determination module 173.

[0174] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where a sub-event has occurred within one or more views. A view consists of controls and other elements that the user can see on the display.

[0175] Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a procedural level within the application's procedural or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct input is optionally determined at least in part based on the hit view of the initial touch that initiates a touch-based gesture.

[0176] The hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchical structure, the hit view determination module 172 identifies the hit view as the lowest-level view in the hierarchical structure from which the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events forming an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source to which it was identified as the hit view.

[0177] The activity event recognizer determination module 173 determines which views(s) within the view hierarchy should receive a specific sub-event sequence. In some embodiments, the activity event recognizer determination module 173 determines that only the hit view should receive the specific sub-event sequence. In other embodiments, the activity event recognizer determination module 173 determines that all views including the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive the specific sub-event sequence. In other embodiments, even if the touch sub-event is entirely confined to the area associated with a particular view, higher views in the hierarchy will still remain actively participating views.

[0178] Event assigner module 174 assigns event information to event identifiers (e.g., event identifier 180). In embodiments that include active event identifier determination module 173, event assigner module 174 delivers event information to the event identifier determined by active event identifier determination module 173. In some embodiments, event assigner module 174 stores event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0179] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module (such as contact / motion module 130) stored in memory 102.

[0180] In some implementations, application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other implementations, one or more of the event recognizers 180 are part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some implementations, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handlers 190 optionally utilize or invoke the data updater 176, the object updater 177, or the GUI updater 178 to update the application's internal state 192. Alternatively, one or more application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0181] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least one subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0182] Event receiver 182 receives event information from event classifier 170. The event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves touch movement, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a longitudinal orientation to a lateral orientation, or vice versa), and the event information includes corresponding information about the device's current orientation (also referred to as device orientation).

[0183] Event comparator 184 compares event information with predefined event or sub-event definitions and determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in event (187) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, event 1 (187-1) is defined as a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off of a predetermined duration (touch end), a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off of a predetermined duration (touch end). In another example, event 2 (187-2) is defined as a drag on a displayed object. For example, dragging includes a touch (or contact) on the displayed object for a predetermined duration, movement of the touch on the touch-sensitive display 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0184] In some implementations, event definition 187 includes definitions of events for corresponding user interface objects. In some implementations, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0185] In some implementations, the definition of the corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined whether the sub-event sequence actually corresponds to or does not correspond to the event type of the event recognizer.

[0186] When the corresponding event recognizer 180 determines that the sub-event sequence does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events based on touch gestures are ignored. In this case, other event recognizers (if any) that remain active in the hit view continue to track and process the ongoing sub-events based on touch gestures.

[0187] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists instructing how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing how or how event recognizers can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists instructing whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0188] In some implementations, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates the event handler 190 associated with the event. In some implementations, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and delaying) the sub-event to the corresponding hit view. In some implementations, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with that tag retrieves the tag and executes a predefined procedure.

[0189] In some implementations, event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or the actively participating view receives the event information and performs a predetermined procedure.

[0190] In some implementations, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some implementations, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on touch-sensitive display.

[0191] In some implementations, event handler 190 includes, or has access to, a data updater 176, an object updater 177, and a GUI updater 178. In some implementations, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other implementations, they are included in two or more software modules.

[0192] It should be understood that the above discussion regarding event handling for user touch on a touch-sensitive display also applies to other forms of user input used to operate the multifunction device 100 using an input device, and not all user input is initiated on the touchscreen. For example, mouse movement and mouse button presses optionally in conjunction with single or multiple keyboard presses or holds; touch movements on the touchpad, such as taps, drags, scrolls, etc.; stylus input; device movement; verbal commands; detected eye movements; biometric input; and / or any combination thereof may optionally be used as input corresponding to sub-events that define the event to be identified.

[0193] Figure 2A portable multifunction device 100 with a touchscreen 112 is shown according to some embodiments. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, selection of one or more graphics occurs when the user breaks contact with one or more graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some specific embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0194] Device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touchscreen 112.

[0195] In some embodiments, device 100 includes a touchscreen 112, a menu button 204, a push-button 206 for powering on / off and locking the device, one or more volume control buttons 208, a SIM card slot 210, a headset jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In another embodiment, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touchscreen 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0196] Figure 3This is a block diagram of an exemplary multi-functional device with a display and a touch-sensitive surface according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (e.g., a home controller or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touchscreen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, and a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to the reference above). Figure 1A The tactile output generator 167 and sensor 359 (e.g., optical sensor, accelerometer, proximity sensor, touch sensor and / or contact intensity sensor (similar to the one mentioned above)) are described. Figure 1A The contact strength sensor 165 is described above. The memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU 310. In some embodiments, the memory 370 stores information related to the portable multifunction device 100. Figure 1A The memory 370 stores programs, modules, and data structures similar to those in the memory 102 of the portable multifunction device 100, or subsets thereof. Additionally, the memory 370 optionally stores additional programs, modules, and data structures not present in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a rendering module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while the portable multifunction device 100 (… Figure 1A The memory 102 may optionally not store these modules.

[0197] Figure 3Each of the elements described above is optionally stored in one or more memory devices of the previously mentioned memory devices. Each of the modules described above corresponds to an instruction set for performing the functions described above. The modules or programs (e.g., instruction sets) described above need not be implemented as separate software programs, processes, or modules, and therefore various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores a subgroup of the modules and data structures described above. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0198] Now let’s turn our attention to the implementation of the user interface, which is optionally implemented on, for example, a portable multifunction device 100.

[0199] Figure 4A An exemplary user interface for an application menu on a portable multifunction device 100 according to some embodiments is shown. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements or a subset or superset thereof:

[0200] • Signal strength indicator 402 for wireless communications such as cellular signals and Wi-Fi signals;

[0201] • Time 404;

[0202] Bluetooth indicator 405;

[0203] • Battery status indicator 406;

[0204] • Tray tray 408 with icons for commonly used applications, such as:

[0205] ○ The telephone module 138 has an icon 416 labeled "telephone", which optionally includes an indicator 414 indicating the number of missed calls or voicemail messages;

[0206] ○ An icon 418 labeled "Mail" in the email client module 140, which optionally includes an indicator 410 for the number of unread emails;

[0207] ○ The icon 420 labeled "Browser" in browser module 147; and

[0208] ○ The icon 422 labeled "iPod" for the video and music player module 152 (also known as the iPod module 152, a trademark of Apple Inc.); and

[0209] • Icons of other applications, such as:

[0210] ○The icon 424 of the IM module 141 marked as "Message";

[0211] ○The calendar module 148 has an icon 426 labeled "Calendar";

[0212] ○ The icon 428 of the image management module 144 is labeled "Photo".

[0213] ○ The icon 430 of camera module 143, which is labeled "camera";

[0214] ○ The icon 432 of the online video module 155, which is labeled "Online Video";

[0215] ○ The icon 434 in the Stock Market Desktop Mini Program 149-2 that is marked as "Stock Market";

[0216] ○The icon 436 of the map module 154 that is labeled "map";

[0217] ○ The icon 438 in the Weather desktop mini-program 149-1 that is marked as "Weather";

[0218] ○ The icon 440 in the alarm clock desktop mini-program 149-4 that is marked as "clock";

[0219] ○ The icon 442 of the fitness support module 142 is labeled "fitness support";

[0220] ○ The icon 444 labeled "Notepad" in Notepad module 153; and

[0221] ○ The icon 446, labeled "Settings," is used to set the settings of the device 100 and its various applications 136.

[0222] It should be pointed out that, Figure 4A The icon labels shown are merely exemplary. For example, icon 422 of video and music player module 152 is labeled "Music" or "Music Player". Other labels may be optionally used for various application icons. In some embodiments, the label of a particular application icon includes the name of the application corresponding to that particular application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to that particular application icon.

[0223] Figure 4B A touch-sensitive surface 451 (e.g., separate from the display 450 (e.g., touchscreen display 112)) is shown. Figure 3 Devices such as tablets or touchpads (e.g., 355) Figure 3An exemplary user interface on the device 300. The device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of the sensors 359) for detecting the intensity of contact on the tactile surface 451 and / or one or more tactile output generators 357 for generating tactile outputs for the user of the device 300.

[0224] While some examples of input on a reference touchscreen display 112 (which combines a touch-sensitive surface and a display) are given below, in some implementations the device detects input on a touch-sensitive surface separate from the display, such as... Figure 4B As shown in the diagram. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a spindle (e.g., on the display (e.g., 450)). Figure 4B The spindle corresponding to 453 in the middle (e.g., Figure 4B (452 in the example). According to these embodiments, the device detects the position corresponding to the corresponding position on the display (e.g., in the example). Figure 4B In the middle, 460 corresponds to 468 and 462 corresponds to 470) is in contact with the touch-sensitive surface 451 (e.g., Figure 4B (460 and 462 in the text). Thus, when the touch-sensitive surface (e.g., ...) Figure 4B 451) and the display of a multi-functional device (e.g., Figure 4B When 450 is separated from the touch-sensitive surface, user input detected by the device on that touch-sensitive surface (e.g., touches 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods may optionally be used for other user interfaces described herein.

[0225] Additionally, while the examples below are primarily given with reference to finger input (e.g., finger touch, single-finger tap, finger swipe), it should be understood that in some implementations, one or more of these finger inputs may be replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture may optionally be replaced by a mouse click (e.g., instead of a touch), followed by movement of the cursor along the swipe path (e.g., instead of movement of the touch). Similarly, a tap gesture may optionally be replaced by a mouse click while the cursor is over the location of the tap gesture (e.g., instead of detection of touch, followed by cessation of touch detection). Likewise, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice may optionally be used simultaneously, or mouse and finger touch may optionally be used simultaneously.

[0226] Figure 5AAn exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include components relative to devices 100 and 300 (e.g., Figures 1A to 4B Some or all of the features described herein. In some embodiments, device 500 has a touch-sensitive display 504, referred to below as touchscreen 504. As an alternative to or complement to touchscreen 504, device 500 has a display and a touch-sensitive surface. Similar to devices 100 and 300, in some embodiments, touchscreen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., touch). The one or more intensity sensors of touchscreen 504 (or touch-sensitive surface) can provide output data representing the intensity of the touch. The user interface of device 500 can respond to touches based on the intensity of the touch, meaning that touches of different intensities can invoke different user interface operations on device 500.

[0227] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related patent applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled “Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application,” published as WIPO Patent Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled “Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships,” published as WIPO Patent Publication No. WO / 2014 / 105276, each of which is incorporated herein by reference in its entirety.

[0228] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508 (if included) may be physical. Examples of physical input mechanisms include push-buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, bangles, trousers, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow a user to wear device 500.

[0229] Figure 5B An exemplary personal electronic device 500 is illustrated. In some embodiments, device 500 may include a reference. Figure 1A , Figure 1B and Figure 3 Some or all of the components described herein. Device 500 has a bus 512 that operatively couples I / O portion 514 to one or more computer processors 516 and memory 518. I / O portion 514 may be connected to display 504, which may have touch-sensitive components 522 and optionally have an intensity sensor 524 (e.g., a contact intensity sensor). Furthermore, I / O portion 514 may be connected to communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 may optionally be a rotatable input device or a pressable input device and a rotatable input device. In some examples, input mechanism 508 may optionally be a button.

[0230] In some examples, the input mechanism 508 is optionally a microphone. The personal electronic device 500 optionally includes various sensors, such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operatively connected to the I / O section 514.

[0231] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 516, may cause the computer processors to perform, for example, the techniques described below, including processes 700, 800, 1000, 1400, 1600, 1800, 2000, and 2300. Figure 7 , Figure 8 , Figure 10 , Figure 14 , Figure 18 , Figure 20 , Figure 23 A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices are not limited to... Figure 5B It can be the components and configurations, or it can include other components or additional components in a variety of configurations.

[0232] As used herein, the term "power indication" refers optionally to the power indication in devices 100, 300, and / or 500 ( Figure 1A , Figure 3 and Figures 5A to 5B A user-interactive graphical user interface object displayed on a screen. For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) optionally each constitute a functional representation.

[0233] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some specific implementations that include a cursor or other positional marker, the cursor acts as a "focus selector," such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), the cursor is positioned on a touch-sensitive surface (e.g., a...). Figure 3 The touchpad 355 or Figure 4B When an input (e.g., a press input) is detected on the touch-sensitive surface 451 of the display, the specific user interface element is adjusted according to the detected input. This applies to touchscreen displays (e.g., those capable of enabling direct interaction with user interface elements on the touchscreen display) Figure 1A The touch-sensitive display system 112 or Figure 4AIn some embodiments of the touchscreen 112, a touch detected on the touchscreen acts as a "focus selector," such that when input (e.g., a press input by touch) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, that particular user interface element is adjusted according to the detected input. In some embodiments, focus moves from one area of ​​the user interface to another without corresponding movement of the cursor or movement of a touch on the touchscreen display (e.g., moving focus from one button to another using tab keys or arrow keys); in these embodiments, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user-controlled user interface element (or a touch on the touchscreen display) that delivers the user-expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user expects to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of the focus selector (e.g., a cursor, touch, or selection box) above the corresponding button will indicate to the user that they expect to activate the corresponding button (rather than other user interface elements shown on the device's display).

[0234] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected over a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted away, before or after contact begins to move, before contact ends, before or after contact intensity is detected to increase and / or before or after contact intensity decreases). The characteristic intensity of the contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half maximum value of the contact intensity, the 90% maximum value of the contact intensity, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the contact intensity over time). In some implementations, the feature intensity is compared to a set of one or more intensity thresholds to determine whether the user has performed an action. For example, the set of one or more intensity thresholds may optionally include a first intensity threshold and a second intensity threshold. In this example, contact with a feature intensity not exceeding the first threshold results in a first action, contact with a feature intensity exceeding the first intensity threshold but not exceeding the second intensity threshold results in a second action, and contact with a feature intensity exceeding the second threshold results in a third action. In some implementations, a comparison between the feature intensity and one or more thresholds is used to determine whether to perform one or more actions (e.g., whether to perform the corresponding action or abandon performing the corresponding action) rather than to determine whether to perform the first action or the second action.

[0235] Figure 5C This demonstrates the use of multiple intensity sensors 524A-524D to detect multiple contacts 552A-552E on a touch-sensitive display 504. Figure 5C It also includes an intensity graph, which shows the current intensity measurements of intensity sensors 524A-524D relative to intensity units. In this example, the intensity measurements of intensity sensors 524A and 524D are both 9 intensity units, and the intensity measurements of intensity sensors 524B and 524C are both 7 intensity units. In some embodiments, the cumulative intensity is the sum of the intensity measurements of the multiple intensity sensors 524A-524D, which is 32 intensity units in this example. In some implementations, each contact is assigned a corresponding intensity, i.e., a portion of the cumulative intensity. Figure 5DThe diagram illustrates the allocation of cumulative intensity to contacts 552A-552E based on their distance from the center of force 554. In this example, each of contacts 552A, 552B, and 552E is assigned a contact intensity of 8 intensity units of the cumulative intensity, and each of contacts 552C and 552D is assigned a contact intensity of 4 intensity units of the cumulative intensity. More generally, in some specific implementations, each contact j is assigned a corresponding intensity Ij according to a predefined mathematical function Ij = A·(Dj / ΣDi), which is a portion of the cumulative intensity A, where Dj is the distance of the corresponding contact j from the center of force, and ΣDi is the sum of the distances of all corresponding contacts (e.g., i = 1 to the last) from the center of force. Reference operations can be performed using electronic devices similar to or equivalent to devices 100, 300, or 500. Figures 5C-5D The operation described above. In some embodiments, the characteristic intensity of the contact is based on one or more intensities of the contact. In some embodiments, an intensity sensor is used to determine the intensity of a single characteristic (e.g., a single characteristic intensity of a single contact). It should be noted that the intensity map is not part of the displayed user interface, but is included within... Figures 5C-5D This is intended to assist readers.

[0236] In some implementations, a portion of the gesture is identified to determine the characteristic intensity. For example, the touch-sensitive surface optionally receives a series of swipe contacts that transition from a starting position to an ending position, where the contact intensity increases. In this example, the characteristic intensity of the contact at the ending position is optionally based only on a portion of the series of swipe contacts, rather than the entire swipe contact (e.g., only the portion of the swipe contact at the ending position). In some implementations, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: unweighted moving average smoothing algorithm, triangular smoothing algorithm, median filter smoothing algorithm, and / or exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of the swipe contact to achieve the purpose of determining the characteristic intensity.

[0237] Optionally, the contact intensity on a touch-sensitive surface is characterized relative to one or more intensity thresholds, such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button on a physical mouse or touchpad.

[0238] In some implementations, the deep press strength threshold corresponds to an intensity at which the device will perform an action different from that typically associated with clicking a button on a physical mouse or touchpad.

[0239] In some implementations, when a contact with a characteristic intensity below a light press intensity threshold (e.g., and above a nominal contact detection intensity threshold, where contacts below the nominal contact detection intensity threshold are no longer detected) is detected, the device will move the focus selector based on the movement of the contact on the touch-sensitive surface, without performing the operations associated with the light press intensity threshold or the deep press intensity threshold. Generally, unless otherwise stated, these intensity thresholds are consistent across different sets of user interface figures.

[0240] An increase in contact intensity from below a light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in contact intensity from below a deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in contact intensity from below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in contact intensity from above a contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting a contact being lifted off the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0241] In some embodiments described herein, one or more operations are performed in response to detecting a gesture including a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting that the intensity of the contact (or multiple contacts) increases to above a press input intensity threshold. In some embodiments, the corresponding operation is performed in response to detecting that the intensity of the corresponding contact increases to above a press input intensity threshold (e.g., a "downward stroke" of the corresponding press input). In some embodiments, the press input includes the intensity of the corresponding contact increasing to above a press input intensity threshold and the intensity of the contact subsequently decreasing to below the press input intensity threshold, and the corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases to below the press input threshold (e.g., an "upward stroke" of the corresponding press input).

[0242] Figures 5E-5H The detection of gestures is shown, including gestures involving contact 562 with an intensity ranging from less than... Figure 5E The light press intensity threshold (e.g., "IT") L The intensity of ”) increases to higher than Figure 5H The deep press intensity threshold (e.g., "IT") DThe intensity of the press input corresponds to the strength of the press. On the user interface 570, which includes application icons 572A-572D displayed in the predefined area 574, when the cursor 576 is displayed above the application icon 572B corresponding to application 2, the gesture performed by contact 562 is detected on the touch-sensitive surface 560. In some embodiments, the gesture is detected on the touch-sensitive display 504. The intensity sensor detects the intensity of the contact on the touch-sensitive surface 560. The device determines the intensity of the contact 562 within a deep press intensity threshold (e.g., "IT"). D The intensity reaches its peak value above 562. Contact is maintained on the touch-sensitive surface 560. In response to a detected gesture, and based on the intensity increasing to a deep press intensity threshold (e.g., "IT") during the gesture... D The above contact 562 displays the scale representation 578A-578C (e.g., thumbnails) of the document recently opened for application 2, such as... Figures 5F-5H As shown. In some embodiments, the intensity is a characteristic intensity of the contact compared to one or more intensity thresholds. It should be noted that the intensity map for contact 562 is not part of the displayed user interface, but is included in... Figures 5E-5H This is intended to assist readers.

[0243] In some implementations, the display of icons 578A-578C includes animation. For example, icon 578A might initially appear near application icon 572B, such as... Figure 5F As shown, as the animation progresses, icon 578A moves upwards, and icon 578B appears near application icon 572B, as shown. Figure 5G As shown in the diagram. Then, representation 578A moves upward, 578B moves upward toward representation 578A, and representation 578C is displayed near application icon 572B, as shown. Figure 5H As shown in the diagram. This indicates that 578A-578C form an array above icon 572B. In some implementations, the animation progresses according to the intensity of contact 562, such as... Figures 5F-5G As shown, where 578A-578C appear and increase with the intensity of contact 562 towards the pressure threshold (e.g., "IT"). D The animation progresses upwards as it increases in intensity. In some implementations, the intensity upon which the animation progresses is based is the characteristic intensity of the contact. Reference can be performed using electronic devices similar to or equivalent to devices 100, 300, or 500. Figures 5E-5H The aforementioned operation.

[0244] In some implementations, the device employs intensity hysteresis to avoid unintended inputs sometimes referred to as "jitter," wherein the device defines or selects a hysteresis intensity threshold that has a predefined relationship with a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the press input intensity threshold). Therefore, in some implementations, a press input includes an increase in the intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of that contact below the hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases below the hysteresis intensity threshold (e.g., an "upstroke" of the corresponding press input). Similarly, in some implementations, a press input is detected only when the device detects that the contact intensity increases from an intensity equal to or below the hysteresis intensity threshold to an intensity equal to or above the press input intensity threshold and optionally the contact intensity subsequently decreases to an intensity equal to or below the hysteresis intensity threshold, and a corresponding operation is performed in response to detecting a press input (e.g., an increase or decrease in contact intensity depending on the environment).

[0245] For ease of explanation, optionally, the description of an operation triggered in response to a press input associated with a press input strength threshold or in response to a gesture including a press input is provided in response to detecting any of the following conditions: the contact strength increases to above the press input strength threshold, the contact strength increases from below a hysteresis strength threshold to above the press input strength threshold, the contact strength decreases to below the press input strength threshold, and / or the contact strength decreases to below the hysteresis strength threshold corresponding to the press input strength threshold. Additionally, in the example where the operation is described as being performed in response to detecting a decrease in contact strength below the press input strength threshold, the operation is optionally performed in response to detecting a decrease in contact strength below a hysteresis strength threshold corresponding to and less than the press input strength threshold.

[0246] Figure 5I An exemplary electronic device 580 is illustrated. Device 580 includes a body 580A. In some embodiments, device 580 may include components relative to devices 100, 300, and 500 (e.g., Figures 1A to 5BThe device 580 may include some or all of the features described herein. In some embodiments, the device 580 has one or more speakers 580B (concealed within a body 580A), one or more microphones 580C, one or more touch-sensitive surfaces 580D, and one or more displays 580E. Alternatively, or as a supplement to the displays and touch-sensitive surfaces 580D, the device may also have a touch-sensitive display (also referred to as a touchscreen). Similar to devices 100, 300, and 500, in some embodiments, the touch-sensitive surface 580D (or touchscreen) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). The one or more intensity sensors of the touch-sensitive surface 580D (or touchscreen) may provide output data representing the intensity of the touch. The user interface of the device 580 may respond to these touches based on the intensity of the touches, meaning that touches of different intensities may invoke different user interface operations on the device 580. In some embodiments, the one or more displays 580E are one or more light-emitting diodes (LEDs). For example, a display can be a single LED, an LED cluster (e.g., red, green, and blue LEDs), multiple discrete LEDs, multiple discrete LED clusters, or other arrangements of one or more LEDs. For example, display 580E can be an array of nine discrete LED clusters arranged in a circle (e.g., a ring). In some examples, the one or more displays contain one or more light-emitting elements of another type.

[0247] Figure 5J An exemplary personal electronic device 580 is depicted. In some embodiments, device 580 may include, relative to... Figure 1A , Figure 1B , Figure 3 and Figures 5A to 5B Some or all of the components described herein. Device 580 has a bus 592 operatively coupling I / O portion 594 to one or more computer processors 596 and memory 598. I / O portion 594 may be connected to display 582, which may have touch-sensitive component 584 and optionally have intensity sensor 585 (e.g., contact intensity sensor). In some embodiments, touch-sensitive component 584 is a component separate from display 582. Furthermore, I / O portion 594 may be connected to communication unit 590 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 580 may include input mechanism 588. In some examples, input mechanism 588 is optionally a button. In some examples, input mechanism 588 is optionally a microphone. Input mechanism 588 is optionally a plurality of microphones (e.g., microphone array).

[0248] Electronic device 580 includes a speaker 586 for outputting audio. Device 580 may include audio circuitry (e.g., in I / O section 594) that receives audio data, converts the audio data into electrical signals, and transmits the electrical signals to speaker 586. Speaker 586 converts the electrical signals into sound waves that are audible to humans. The audio circuitry (e.g., in I / O section 594) also receives electrical signals converted from sound waves by a microphone (e.g., input mechanism 588). The audio circuitry (e.g., in I / O section 594) converts these electrical signals into audio data. The audio data may optionally be retrieved by I / O section 594 from and / or transmitted to memory 598 and / or RF circuitry (e.g., in communication unit 590).

[0249] The memory 598 of the personal electronic device 580 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions, which, when executed by one or more computer processors 596, may cause the computer processors to perform, for example, the techniques described below, including process 800. Figure 8 ), 1000 Figure 10 ), 1200 Figure 12 ), 1400 Figure 14 ), 1600 Figure 16 ), 1800 Figure 18 ), 2000 Figure 20 ) and 2300 ( Figure 23 A computer-readable storage medium can be any medium that can tangibly contain or store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media can include, but is not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory, solid-state drives, etc. Personal electronic devices are not limited to... Figure 5J It is not the components and configurations of a single entity, but rather other components or additional components that may include a variety of configurations.

[0250] As used herein, "installed application" refers to a software application that has been downloaded to an electronic device (e.g., device 100, 300, and / or 500) and is ready to be launched on the device (e.g., become open). In some implementations, the downloaded application becomes an installed application using an installer that extracts program portions from the downloaded software package and integrates the extracted portions with the computer system's operating system.

[0251] As used herein, the terms "open application" or "running application" refer to a software application that maintains state information (e.g., as part of device / global internal state 157 and / or application internal state 192). An open or running application is optionally any of the following types of applications:

[0252] • The currently active application displayed on the screen of the device that is using the application;

[0253] • Background applications (or background processes) that are not currently displayed but whose one or more processes are being handled by one or more processors; and

[0254] • Not running but stored in memory (either volatile or non-volatile).

[0255] It can also be used to resume the execution of paused or hibernating applications by providing state information.

[0256] As used herein, the term "closed application" refers to a software application that does not retain state information (e.g., the state information of a closed application is not stored in the device's memory). Therefore, closing an application includes stopping and / or removing the application's process and removing the application's state information from the device's memory. Generally, opening a second application while the first application is running does not close the first application. When the second application is displayed and the first application stops displaying, the first application becomes a background application.

[0257] Now let’s turn our attention to the implementation of user interfaces (“UIs”) and associated processes on electronic devices such as portable multifunction devices 100, 300, 500, and 580.

[0258] Figures 6A to 6P Exemplary user interfaces for controlling audio playback are shown according to some embodiments. The user interfaces in these figures are used to illustrate including... Figures 7 to 8 The process described below is the method described in the text.

[0259] Devices can use the satisfaction of proximity conditions to perform one or more functions. The use of proximity between one device and another can serve as a clear indication that a user (e.g., holding one of the devices) wants to perform an action (e.g., invoke an interface) on one or both devices. For example, this can prevent wasting device resources by avoiding excessive user input to perform functions (e.g., navigating one or more menus on the device display). Furthermore, this can also save user time, for example, by reducing the amount of user input required to perform a function (e.g., invoking an interface on the display).

[0260] Figure 6AExemplary electronic devices 600 (e.g., a telephone) and 650 (e.g., a smart speaker) are illustrated. In some embodiments, device 600 includes one or more features of device 100, 300, or 500. Device 650 includes one or more features of device 580. In some embodiments, device 650 includes one or more features of device 100, 300, or 500. Figure 6A In the illustrated embodiment, device 650 is located in the kitchen and designated as a "kitchen speaker". Devices 600 and 650 are connected (e.g., via Bluetooth, near-field communication, or a network connection) so that each device can obtain information about the other. Such information may include information about audio currently playing or recently played on the device.

[0261] Figures 6A to 6P An exemplary physical arrangement of devices 600 and 650 is shown. Figure 6A A first scenario is shown, including devices 650 and 600 at a relatively long distance from each other. Figure 6A An exemplary top view of the first scene is shown. (As shown) Figure 6A As shown, devices 600 and 650 are far apart. Figure 6AA proximity condition range indicator 652 is shown around device 650. The proximity condition range indicator is also referred to herein as a "proximity area indicator" or simply a "proximity area". Device 600 is not within the proximity condition range indicator 652. The proximity condition range indicator is included in the figures to aid understanding but is not shown on device 600 or device 650. The proximity condition range indicator 652 is included as a visual aid and is intended to indicate the physical proximity at which a proximity condition will be satisfied. For example, the range indicator 652 may indicate the range of a near-field communication detection circuit of device 650. In some embodiments, any suitable technology may be used to detect proximity between devices. For example, in some examples, a broadband wireless connection is used. A broadband wireless connection is used, for example, to determine one or more of the following: the directionality, distance, and orientation of one or more devices. Therefore, the presence of a detectable device (e.g., partially or completely) within the proximity condition range indicator 652 will satisfy the proximity condition, but not if the detectable device is located outside the range indicator 652. Those skilled in the art will understand that the detection range of physical proximity may be non-uniform, susceptible to various variables (e.g., wireless interference, air humidity, etc.), and may include points in three-dimensional space, all of which are contemplated within the scope of this disclosure. Therefore, the graphical representation of the proximity condition range indicator 652 is not intended to limit the range for determining whether proximity conditions are met. Furthermore, the figures are not necessarily drawn to scale and are included herein only as visual aids. Therefore, unless otherwise stated, the dimensions and scales of the features depicted in the figures are not intended as limitations on the distances required to be in very close proximity or to meet proximity conditions.

[0262] Figure 6A Audio activity indicator 625, indicating audio activity of device 600, and audio activity indicator 675, indicating audio activity of device 650, are shown. Audio activity indicator 625 and audio activity indicator 675 indicate... Figure 6A In the scenario shown, neither device 600 nor device 650 is currently outputting audio (e.g., not playing back media), as indicated by the slashes on the notes in audio activity indicators 625 and 675. The audio activity indicators are included in the figures to aid understanding but are not displayed on device 600 or device 650. In addition to the activity indicators, some of the figures described below also include audio representations (e.g., 672a to 672c), which are also included in the figures to aid understanding of a specific media item playing on device 600. These audio representations are also not displayed on device 600 or device 650.

[0263] Figure 6B A second scenario is shown, including device 600 which is located at a short distance from device 650. Figure 6B An exemplary top view of the second scene is shown. (As shown) Figure 6B As shown, devices 600 and 650 are close to each other, and device 600 is now at least partially within the proximity condition range indicator 652. Since the proximity condition range indicator 652 represents the physical proximity at which the proximity condition is satisfied, device 600 detects an indication that the physical proximity between devices 600 and 650 satisfies the proximity condition (e.g., and in response, initiates communication with device 650, for example, to send the condition-satisfied indication). In some examples, device 650 detects the proximity condition-satisfied indication (e.g., and in response, initiates communication with device 600, for example, to send the condition-satisfied indication).

[0264] In response to an indication that physical proximity meets a proximity condition, device 600 displays a media information indication 606 representing a media item recently played on device 600. Exemplary media items include, but are not limited to, music tracks, audiobooks (or portions thereof, such as chapters), podcasts, videos, and telephone calls. In some embodiments, based on the determination that neither device 600 nor device 650 is currently playing audio, the media information indication 606 represents a media item recently played on device 600. Figure 6B In the media information display, 606 represents a music track (e.g., a song).

[0265] The media information display 606 optionally includes several graphical elements that provide information about the media item, including: (1) an external device indicator 606-1 that indicates the device (e.g., device 650, also referred to as a kitchen speaker) that is near the device 600; (2) an action indicator 606-2 that indicates an action or function associated with the media information display 606 (e.g., an action performed in response to a selection of the media information display 606); (3) a media item indicator 606-3 that provides information identifying the media item (e.g., song title (“Summertime”) and artist “DJ AP”); (4) a media item status indicator 606-4 that describes the status of the media item (e.g., “recently played”); and (5) a media item graphic 606-5 that includes an image associated with the media item (e.g., the album cover of the album to which the song belongs).

[0266] As indicated by action indicator 606-2, selection of media information display 606 can cause the media item associated with media information display 606 to play on device 650. In some embodiments, the action performed in response to selection of media information display 606 depends on the type of input. For example, in response to input on media information display 606, such as input 610a, device 650 can perform an operation. For example, if the input is a tap gesture, playback of the media item is initiated on device 650, while if the input is a downward swipe gesture that begins on or near media information display 606, a different operation is performed.

[0267] like Figure 6B As shown, device 600 can receive input (e.g., a swipe) on media information indication 606. In response to receiving input, and based on determining that the input is a swipe (e.g., the opposite of a tap), device 600 displays media information indication 608, as shown. Figure 6C As shown. In Figures 6B to 6C In the illustrated embodiment, device 600 also stops displaying media information provision 606 and partially blurs (e.g., grays out, blurs) the portion of user interface 604 not occupied by media information display 608. Media information display 608 optionally includes several graphical elements providing information about media items, including: (1) an external device indicator 608-1 indicating a device (e.g., device 650, also referred to as a kitchen speaker) near device 600; (2) an action indicator 608-2 indicating an action or function associated with media information display 608 (e.g., an action performed in response to a selection by media information display 608); (3) a media item indicator 608-3 providing information identifying the media item (e.g., song title (“Summertime”) and artist “DJ AP”); (4) a media item status indicator 608-4 describing the status of the media item (e.g., “Recently Played”); and (5) a media item graphic 608-5 including an image associated with the media item (e.g., album cover of the album to which the song belongs). In some embodiments, media information display 608 differs from media information display 606. Figures 6B to 6C In the illustrated embodiment, media information display 608 is larger than media information display 606 and is displayed in or near the center of display 602, while media information display 606 is displayed at or near the top of display 602. For example, the media information provided in display 608 may be an enlarged and extended version of the media information provided in display 606.

[0268] In response to an input corresponding to the selection of a media item when the media information display 608 is displayed (e.g., a tap on the media item graphic 608-5 or a tap on the media information display 608 when the media item graphic 608-5 is entered on the media information display 608), playback of the media item "DJ AP's Summertime" is initiated on the device 650.

[0269] In addition, the media information display 608 includes media items that can be selected for playback on the device 650 (e.g., Figure 6C The media item graphics (608-5, 608-6) queue in the image. For example... Figure 6C As shown, the media information display 608 includes a media item graphic 608-6 corresponding to the second media item. In Figure 6C In this process, device 600 receives input on media information display 608 (e.g., input 601b representing a right-to-left swipe). In response to receiving input 610b, device 600 scrolls the media item graphic horizontally in media information display 608, as shown... Figure 6D As shown. The scrolling media information display 608 includes partially moving media item graphic 608-5 away from the left edge of the media information display 608, fully displaying media item graphic 608-6 in the middle of the media information display 608, and partially displaying media item graphic 608-7 that was previously completely hidden (e.g., not displayed). As a result of scrolling, media item indicator 608-3 is replaced (or updated) by media item indicator 608-3a, which provides information identifying the media item corresponding to media item graphic 608-6 (e.g., playlist (“Hit Mix”) and user “Jane”). In some embodiments, the media item corresponding to media item graphic 608-6 is the media item played immediately before the media item corresponding to media item graphic 608-5 and is considered the most recently played media item, such as... Figure 6D The media item status indicator 608-4 is shown in the diagram. In some embodiments, the queue in the media information display 608 includes media items scheduled to play after the media items corresponding to media item graphic 608-5 (e.g., the next 2, 4, 5, 8 media items to be played). In some such embodiments, the media items corresponding to media item graphic 608-6 represent media items scheduled to play after the media items corresponding to media item graphic 608-5, and optionally, the media item status indicator 608-4 is replaced or updated, for example, with a media item status indicator such as "one above".

[0270] exist Figure 6DIn this process, device 600 receives input 610c (e.g., a tap) corresponding to the selection of a media item represented by media item graphics 608-6. In response to receiving input 610c, device 600 transmits a command to device 650 to initiate playback of the media item represented by media item graphics 608-6 on device 650. Figure 6E The state of devices 600 and 650 is shown after playback of a media item is initiated on device 650. Figure 6E In the audio activity indicator 675, device 650 is currently playing audio, and audio activity indicator 625 indicates that device 600 is still not playing audio. Media information indicator 608 is optionally updated to reflect the current state; the updated activity indicator 608-2a indicates that the media item "Jane's Hit Mix" is now playing on device 650, as indicated by external device indicator 608-1, which still identifies device 650 ("kitchen speaker").

[0271] Figure 6F A scenario is illustrated where device 600 is not within the proximity condition range indicator 652, device 600 is playing audio (as indicated by audio activity indicator 625), and device 650 is not playing audio (as indicated by audio activity indicator 675). Figure 6F In the middle, device 600 displays a user interface 614 of a music application running on device 600, which indicates the media item currently being played on device 600. Figure 6G The diagram illustrates that when device 600 is playing audio and device 650 is not (e.g., the song "The Sound" from the album "Sophomore" by artist "Mac Band"), device 600 moves at least partially within proximity condition range indicator 652. Since proximity condition range indicator 652 represents the physical proximity at which a proximity condition is satisfied, device 600 detects an indication that the physical proximity between device 600 and device 650 satisfies the proximity condition (e.g., and in response, initiates communication with device 650, such as to send the condition-satisfied indication (e.g., to send a media item between device 600 and device 650)). In some examples, device 650 detects an indication that a proximity condition is satisfied (e.g., and in response, initiates communication with device 600, such as to send the condition-satisfied indication).

[0272] In response to an indication that a physical proximity condition is met, device 600 displays a media information indication 616 indicating the media item currently being played on device 600. In some embodiments, the media information indication 616 indicates the media item currently being played on device 600 based on a determination that device 600 is currently playing a media item.

[0273] The media information display 616 optionally includes several graphical elements that provide information about the media item currently playing on the device 600, including: (1) an external device indicator 616-1 that indicates a device (e.g., device 650, also referred to as a kitchen speaker) that is near the device 600; (2) an action indicator 616-2 that indicates an action or function associated with the media information display 616 (e.g., an action performed in response to a selection of the media information display 616); (3) a media item indicator 616-3 that provides information identifying the media item (e.g., song title (“The Sound”) and artist “MacBand”); (4) a media item status indicator 616-4 that describes the status of the media item (e.g., “from phone”); and (5) a media item graphic 616-5 that includes an image associated with the media item (e.g., the album cover of the album to which the song belongs).

[0274] As indicated by action indicator 616-2, selecting media information enable indicator 616 can cause the media item associated with media information enable indicator 616 to be played on device 650. In addition, media item status indicator 616-4 indicates the source of the media item to be played back (e.g., "from telephone").

[0275] In some implementations, the media information display 616 corresponding to the currently playing media item is similar to the media information display 606 corresponding to the most recently played media item. For example, the action performed in response to a selection of the media information display 616 depends on the type of input. In some implementations, a tap gesture on the media information display 616 initiates playback of the media item currently playing on device 600 on device 650, while a downward swipe gesture starting on or near the media information display 616 causes device 600 to display a media information display similar to the media information display 608 described above, but with the currently playing media item... Figure 6G Information about the media items played on device 600.

[0276] exist Figure 6G In this process, device 600 receives input 610d (e.g., a tap) corresponding to a selection on media information display 616. In response to receiving input 610d, device 600 transmits a command to device 650 to initiate playback of the media item currently playing on device 600. For example... Figure 6H As shown, in response to receiving input 610d, device 600 displays an updated media information display 616a (e.g., replacing media information display 616 with a new display, or maintaining the display of media information display 616 but changing at least a portion of the information displayed therein). Figure 6H In this context, the media information indication 616a includes an updated action indicator 616-2a that indicates that playback has been initiated (e.g., "turn to speaker"). In some embodiments, the device 600 responds to the media information indication 606 ( Figure 6B A tap on the device displays a media information display similar to that of Media Information Display 616a, except that it includes information corresponding to the recently played media item "DJAP's Summertime" instead of the currently playing media item "Mac Band's The Sound".

[0277] like Figure 6I As shown, after displaying the updated media information enablement indicator 616a (e.g., in response to an indication that playback has successfully started on device 650 or after a predetermined amount of time has elapsed since receiving input 610d or displaying media information enablement indicator 616a), device 600 stops displaying media information enablement indicator 616a and updates user interface 614 using graphical elements 614-2 to indicate that audio is playing back on device 650 (as indicated by audio status indicator 675) on the “kitchen speaker”. When playback begins on device 650 (e.g., in response to input 610d), audio status indicator 625 instructs device 600 to stop playing audio. In some embodiments, device 600 continues playing audio (e.g., “The Sound from the Mac Band”) during playback on device 650.

[0278] Figure 6J A scenario is illustrated where device 600 is not within proximity condition range indicator 652, device 600 is playing audio (as indicated by audio activity indicator 625), and device 650 is playing audio different from the audio played on device 600 (as indicated by audio activity indicator 675). Specifically, in Figure 6J In this context, device 600 is playing "O Christmas Tree" as shown on user interface 618, and device 650 is playing "Old MacDonald" as indicated by audio representation 672a. Audio representation 672a is provided for illustrative purposes only. Figure 6J In the middle, device 600 displays the user interface 618 of a music application running on device 600, which indicates the media item currently being played on device 600. Figure 6K The illustration shows that when both devices 600 and 650 are playing their respective audio media items, device 600 moves at least partially within proximity condition range indicator 652. Since proximity condition range indicator 652 represents the physical proximity at which a proximity condition is satisfied, device 600 detects an indication that the physical proximity between devices 600 and 650 satisfies the proximity condition (e.g., and in response, initiates communication with device 650, such as to send an indication that the condition is satisfied). In some examples, device 650 detects an indication that the proximity condition is satisfied (e.g., and in response, initiates communication with device 600, such as to send an indication that the condition is satisfied).

[0279] like Figure 6K As shown, in response to an indication that a physical proximity condition is met, device 600 displays a media information indication 620, which indicates the media item currently being played on device 600. In some embodiments, based on a determination that device 600 is currently playing a media item, media information indication 620 indicates the media item currently being played on device 600 (e.g., regardless of whether device 650 is playing audio). In some embodiments, based on a determination that both device 600 and device 650 are currently playing audio, media information indication 620 indicates the media item currently being played on device 600.

[0280] The media information display 620 optionally includes several graphical elements that provide information about the media item currently being played on device 600, including: (1) an external device indicator 620-1 that indicates a device (e.g., device 650) that is near device 600. Figure 6K (1) also referred to as “kitchen speaker”); (2) action indicator 620-2, which indicates an action or function associated with media information enablement 620 (e.g., an action performed in response to selection by media information enablement 620); (3) media item indicator 620-3, which provides information identifying the media item (e.g., song title (“O Christmas Tree”) and artist “Caroler”); (4) media item status indicator 620-4, which describes the status of the media item (e.g., “from phone”); and (5) media item graphic 620-5, which includes an image associated with the media item (e.g., album cover of the album to which the song belongs).

[0281] As indicated by action indicator 620-2, selection of media information enable indicator 620 can cause the media item associated with media information enable indicator 620 to be played on device 650. In addition, media item status indicator 620-4 indicates the source of the media item to be played back (e.g., "from telephone").

[0282] In some implementations, media information display 620 is similar to media information display 616. For example, the action performed in response to a selection on media information display 620 depends on the type of input. In some implementations, a tap gesture on media information display 620 initiates playback of the media item currently playing on device 600 on device 650, while a downward swipe gesture starting on or near media information display 620 causes device 600 to display a media information display similar to media information display 608 described above, but with the currently selected media item... Figure 6K Information about the media items played on device 600.

[0283] exist Figure 6K In this embodiment, device 600 receives input 610e (e.g., a tap) corresponding to a selection of media information indication 620. In response to receiving input 610e, device 600 transmits a command to device 650 to initiate playback of the media item currently playing on device 600. In some embodiments, in response to receiving input 610e, device 600 displays an updated media information indication, similar to an updated media information indication 616a, to indicate that the media item currently playing on device 600 is being moved to playback on device 650.

[0284] like Figure 6L As shown, upon receiving input 610e (e.g., in response to receiving input 610e), device 600 stops displaying media information enablement 620 and updates user interface 618 with graphical elements 618-2 to indicate that audio is playing back on device 650's "kitchen speaker" (as indicated by audio status indicator 675). When (e.g., in response to input 610e) playback begins on device 650, audio status indicator 625 instructs device 600 to stop playing audio. Audio status indicator 675 indicates that device 650 is playing audio (e.g., "O Christmas Tree" as indicated by audio representation 672b). In some embodiments, device 600 continues playing audio (e.g., "O Christmas Tree") during playback on device 650.

[0285] Figure 6M A scenario is illustrated where device 600 is not within proximity condition range indicator 652, device 600 is not playing audio (as indicated by audio activity indicator 625), and device 650 is playing audio (“Anything” sung by various artists (e.g., 672c), as indicated by audio activity indicator 675). Figure 6M In the process, device 600 displays user interface 604 (e.g., a home screen with application icons). Figure 6NThe illustration shows that when both devices 600 and 650 are playing their respective audio media items, device 600 moves at least partially within proximity condition range indicator 652. Since proximity condition range indicator 652 represents the physical proximity at which a proximity condition is satisfied, device 600 detects an indication that the physical proximity between devices 600 and 650 satisfies the proximity condition (e.g., and in response, initiates communication with device 650, such as to send an indication that the condition is satisfied). In some examples, device 650 detects an indication that the proximity condition is satisfied (e.g., and in response, initiates communication with device 600, such as to send an indication that the condition is satisfied).

[0286] In response to an indication that physical proximity meets a proximity condition, device 600 displays a media information indication 622 indicating the media item currently being played on device 650. In some embodiments, based on a determination that device 650 is currently playing a media item and device 600 is not currently playing a media item, media information indication 622 indicates the media item currently being played on device 650. In some embodiments, based on a determination that device 650 is currently playing a media item, media information indication 620 indicates the media item currently being played on device 650 (e.g., regardless of whether device 600 is playing a media item).

[0287] The media information display 622 optionally includes several graphical elements that provide information about the media item currently playing on the device 650, including: (1) an external device indicator 622-1 that indicates the device (e.g., device 650, also referred to as a kitchen speaker) that is near the device 600; (2) an action indicator 622-2 that indicates an action or function associated with the media information display 622 (e.g., an action performed in response to a selection by the media information display 622); (3) a media item indicator 622-3 that provides information identifying the media item (e.g., the song title (“Anything”) and the artist “Various Artists”); and (4) a media item graphic 622-4 that includes an image associated with the media item (e.g., the album cover of the album to which the song belongs).

[0288] As indicated by action indicator 622-2, selection of media information enablement 622 enables media items associated with media information enablement 622 (e.g., transferred from device 650) to be played on device 600.

[0289] In some implementations, the action performed in response to a selection on the media information display 622 depends on the type of input. For example, a tap gesture on the media information display 622 initiates playback of the media item currently playing on device 650 on device 600, while a downward swipe gesture starting on or near the media information display 622 causes device 600 to display a media information display similar to the media information display 608 described above, but with the currently selected media item... Figure 6N Information about the media items played on device 650.

[0290] exist Figure 6N In this context, device 600 receives input 610f (e.g., a tap) corresponding to a selection on media information enablement display 622. In response to receiving input 610f, device 600 initiates playback of the media item currently playing on device 650 on device 600. For example... Figure 6O As shown, in response to receiving input 610f, device 600 displays an updated media information indication 622a (similar to an updated media information indication 616a), wherein the updated action indicator 622-2a indicates that the media item currently playing on device 650 is being transferred to be played back on device 600.

[0291] like Figure 6P As shown, after displaying an updated media information indication 622a (e.g., in response to an indication that playback has successfully started on device 600 or after a predetermined amount of time has elapsed since receiving input 610f or displaying media information indication 622a), device 600 stops displaying media information indication 622a and displays user interface 624 to indicate that audio is playing back on device 600 (as indicated by audio status indicator 625). In some embodiments, user interface 624 corresponds to a music application launched in response to initiating playback on device 600 and allows the user to further control playback on device 600. When playback begins on device 600 (e.g., in response to input 610f), audio status indicator 675 instructs device 650 to stop playing audio. In some embodiments, device 650 continues playing audio (e.g., “Anything” by various artists) during playback on device 600.

[0292] Figure 7 This is a flowchart illustrating a method for controlling audio playback using an electronic device according to some embodiments. Method 700 is performed at a device (e.g., 100, 300, 500, or 600) having a display device (e.g., 602). Some operations in method 700 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0293] As described below, method 700 provides an intuitive way to control audio playback. This method reduces the cognitive burden on users when controlling audio playback, thus creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to control audio playback more quickly and effectively, saving power and increasing the interval between battery charges.

[0294] In some embodiments, the electronic device (e.g., 600) includes a computer system. The computer system optionally communicates with a display generating component and one or more input devices (e.g., wired communication, wireless communication). The display generating component is configured to provide visual output, such as a display via a CRT monitor, a display via an LED monitor, or a display via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. One or more input devices are configured to receive input, such as a touch-sensitive surface that receives user input. In some embodiments, one or more input devices are integrated with the computer system. In some embodiments, one or more input devices are separate from the computer system. Therefore, the computer system can transmit data (e.g., image data or video data) via wired or wireless connections to an integrated or external display generating component to visually generate content (e.g., using a display device), and can receive input from one or more input devices via wired or wireless connections.

[0295] When an electronic device is connected to an external device (e.g., 650), it detects (702) an indication that the physical proximity between the electronic device and the external device meets a proximity condition (e.g., represented by 652). In some embodiments, the indication is provided in response to determining that the physical proximity between the electronic device and the external device meets the proximity condition. In some embodiments, the proximity condition is met if the distance between the electronic devices is less than a threshold distance (e.g., 6 inches, 12 inches, 18 inches, 24 inches).

[0296] In response to an indication that physical proximity meets a proximity condition, the electronic device displays (704) a first media information enablement (e.g., 606, 616, 620) indicating the first media item (e.g., a music track, an audiobook (or a portion thereof, such as a chapter), a podcast, a video, a telephone call; a media item currently playing on the electronic device, a recently played media item that has ended, stopped, or paused on the electronic device, or a media item scheduled to play next on the electronic device when the user launches a media application or selects a playlist, such as the first track in a playlist). This first media information enablement includes a first set of information (e.g., 606-1 to 606-5) about the first media item. Displaying the first media information enablement in response to an indication that physical proximity meets a proximity condition automatically provides feedback to the user, thereby reducing the amount of input required to display the first media information enablement. Providing improved feedback when a set of conditions have been met, reducing the amount of input required to perform an operation, and performing an operation without further user input enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This also reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0297] The electronic device receives (706) a first input (e.g., 610a, 610d, 610e) indicating a selection of the first media information enablement representation.

[0298] In response to receiving the first input at (708), and based on determining that the first input is a first type of input (e.g., a tap gesture at a location corresponding to the first media information indication on a touch-sensitive display of the electronic device), the electronic device transmits (710) an instruction to an external device to initiate playback of the first media item on the external device (e.g., Figures 6H to 6I ).

[0299] In response to receiving the first input (708), and based on determining that the first input is a second type of input different from the first type of input (e.g., a directional swipe gesture on the first media information display), the electronic device displays (712) a second media information display (e.g., 608) representing the first media item via a display device. This second media information display differs from the first media information display and includes a second set of information about the first media item. Instructions are transmitted to an external device to initiate playback of the first media item on the external device or to display the second media information display representing the first media item. Depending on the type of input selected for the first media display, additional control options related to the first media display are provided without cluttering the user interface with additional displayed controls. Providing additional control options without cluttering the UI with additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0300] In some implementations, the electronic device displays a second media information enablement without transmitting instructions to an external device to initiate playback of the first media item on the external device. In some implementations, the second set of information includes the first set of information, but is displayed differently. In some implementations, the second media information enablement includes additional information (e.g., more details) about the first media item and / or a larger graphical representation of the first media item. In some implementations, the second set of information includes a queue of media items, including media items that are after the first media item (e.g., media items scheduled to play after the first media item) and / or media items that are before the first media item (e.g., media items played before the first media item or positioned before the first media item in a media item list such as a playlist).

[0301] In some implementations, the first media information display includes an indication to identify an external device (e.g., 606-1, 606-2). In some implementations, the second media information display includes an indication to identify an external device. In some implementations, the first media information display includes an indication that selecting the first media information display will initiate playback of the first media item on the external device.

[0302] In some implementations, in response to receiving a first input, and based on determining that the first input is a first type of input, the electronic device changes the visual appearance of the first media information indication (e.g., 612-2a) to indicate that playback of the first media item on an external device has been initiated (see, for example...). Figure 6HIn some embodiments, changing the visual appearance of the first media information display includes displaying text indicating that playback of the first media item has been initiated on an external device, while optionally maintaining the display of at least some of the information in the first set of information about the first media item. In some embodiments, changing the visual appearance of the first media information display includes replacing the indication that the selection of the first media item to initiate playback on an external device with an indication (e.g., text) that playback of the first media item has been initiated on an external device.

[0303] In some implementations, the first set of information indicating the first media item (e.g., 606-5, 902-4) corresponds to a type of media item. Exemplary types of media items include, but are not limited to, music tracks, audiobooks, podcasts, videos, and telephone calls. In some implementations, the first set of information for a telephone call media item includes an image of the caller's avatar.

[0304] In some implementations, as part of initiating playback of the first media item on an external device, the electronic device transmits data of the first media item to the external device based on the determination that the first media item meets data transmission conditions (e.g., the first media item is a telephone call; the first media item is provided by an application, service, or account available to the electronic device but not to the external device). In some implementations, as part of initiating playback of the first media item on an external device, the electronic device abandons transmitting data of the first media item to the external device based on the determination that the first media item does not meet data transmission conditions (e.g., the first media item is provided by a specific application, service, or account that the external device has access to). Transmitting (or not transmitting) the first media item to the external device based on whether the first media item meets data transmission conditions avoids requiring the user to provide additional input for data transmission. Performing operations when a set of conditions has been met without further user input enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0305] In some implementations, a media item queue, including the first media item, exists on the electronic device before playback is initiated. In some such implementations, initiating playback includes transferring the queue to an external device. In some implementations, the first media item is accessible to both the electronic device and the external device via a public user account. In some implementations, the external device obtains the first media item from the public user account (e.g., playback of the first item does not include streaming the first media item from the electronic device to the external device). In some implementations, the external device obtains the queue from the public user account.

[0306] In some implementations, after transmitting a command to an external device to initiate playback of a first media item on the external device, in response to determining that a time condition is met (e.g., a threshold amount of time has elapsed since the first input of the first type was received), the electronic device stops displaying a first media information indication of the first media item (see, for example...). Figure 6I ).

[0307] In some embodiments, upon displaying a first media information enablement indicating a first media item and before receiving a first input, the electronic device detects an indication that the physical proximity between the electronic device and the external device satisfies a physical separation condition (e.g., represented by 656). In some embodiments, an indication is provided in response to determining that the physical proximity between the electronic device and the external device satisfies the physical separation condition. In some embodiments, the physical separation condition is satisfied if the distance between the electronic device and the external device exceeds a predefined threshold distance. In some embodiments, the threshold distance associated with the physical separation condition is greater than the first threshold distance associated with the (first) proximity condition. For example, the physical separation condition is satisfied if the distance between the electronic device and the external device exceeds a third threshold distance (e.g., 20 feet), and the first proximity condition is satisfied if the distance between the electronic device and the external device is less than a first threshold distance (e.g., 1-2 feet), wherein the third threshold distance is greater than the first threshold distance. In some embodiments, upon displaying a first media information enablement indicating a first media item and before receiving a first input, in response to detecting an indication that the physical proximity satisfies the physical separation condition, the electronic device stops displaying the first media information enablement indicating the first media item (e.g., 656). Figure 11D In response to an indication that physical proximity meets physical separation conditions, the display of the first media information enablement indicator for the first media item ceases to provide feedback to the user, indicating that playback on an external device can no longer be initiated via selection of the first media information enablement indicator. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0308] In some implementations, the first type of input includes a tap gesture (e.g., 610d), and the second type of input includes a swipe or drag gesture (e.g., 610a; a vertical up or vertical down swipe or drag gesture).

[0309] In some embodiments, when displaying a second media information enablement indicating a first media item, the electronic device receives a second input (e.g., 610c) indicating a selection of the second media information enablement. In some embodiments, in response to receiving the second input indicating a selection of the second media information enablement, the electronic device transmits a command to an external device to initiate playback of the second media item on the external device. In some embodiments, the second media item is the first media item. In some embodiments, the second media information enablement indicates a queue of media items including a representation of the first media item and representations of at least one other media item. In some such embodiments, the electronic device scrolls the queue in response to an input (e.g., a horizontal swipe or drag gesture). In some embodiments, in response to an input corresponding to a selection of a media item in the queue, the electronic device transmits a command to an external device to initiate playback of the selected media item on the external device. Displaying a second media information enablement (e.g., with additional information about the first media item) and initiating playback of the second media item on the external device in response to a selection of the second media information enablement provides improved feedback by providing the user with additional information and / or control over playback on the external device. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0310] In some implementations, while the electronic device is causing playback of the first media item, an indication is detected that the physical proximity between the electronic device and the external device meets a proximity condition (e.g., ...). Figure 6G In some embodiments, inducing playback includes outputting a media item via one or more speakers of the electronic device, or inducing output at one or more connected external speakers. In some embodiments, in response to receiving a first input, and based on determining that the first input is a first type of input, the electronic device stops inducing playback of the first media item at the electronic device (see, for example...). Figure 6I In some implementations, the electronic device continues to play the first media item.

[0311] In some implementations, a first media item is playing on an electronic device when an indication is received that the physical proximity between the electronic device and an external device meets a proximity condition. A first media information indication indicates that the first media item (e.g., ...) is playing on the electronic device. Figure 6G In some implementations, based on the determination that the first media item is currently being played on an electronic device, a command is executed to transmit to an external device to initiate playback of the first media item on the external device.

[0312] In some implementations, as part of initiating playback of the first media item on the external device, the electronic device stops playing the third media item on the external device (see, for example...). Figure 6L ).

[0313] In some implementations, the electronic device is not playing media items (e.g., when an indication is received that the physical proximity between the electronic device and the external device meets a proximity condition) Figure 6B The first media item is the media item most recently played on the electronic device before an indication is detected that the physical proximity between the electronic device and the external device meets a proximity condition. In some embodiments, if the external device is not playing a media item when the indication is detected that the physical proximity between the electronic device and the external device meets a proximity condition, the first media information indicator may indicate the media item most recently played on the electronic device.

[0314] It should be noted that the above description is in contrast to the process described in method 700 (e.g., Figure 7 The details of the above also apply in a similar manner to the methods described below. For example, method 800 and / or method 1000 may optionally include one or more features of the various methods described above with reference to method 700. For example, operation 706 of method 700 may be performed to select the first media information enablement representation described in operation 1004 of method 1000, and operations 708, 710, and / or 712 may be performed as part of method 1000 in response to this selection. For the sake of brevity, these details will not be repeated below.

[0315] Figure 8 This is a flowchart illustrating a method for controlling audio playback using an electronic device according to some embodiments. Method 800 is performed at a device (e.g., 100, 300, 500, or 600) having a display device (e.g., 602). Some operations in method 800 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0316] As described below, method 800 provides an intuitive way to control audio playback. This method reduces the cognitive burden on users when controlling audio playback, thus creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to control audio playback more quickly and effectively, saving power and increasing the interval between battery charges.

[0317] In some embodiments, the electronic device (e.g., 600) includes a computer system. The computer system optionally communicates with a display generating component and one or more input devices (e.g., wired communication, wireless communication). The display generating component is configured to provide visual output, such as a display via a CRT monitor, a display via an LED monitor, or a display via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. One or more input devices are configured to receive input, such as a touch-sensitive surface that receives user input. In some embodiments, one or more input devices are integrated with the computer system. In some embodiments, one or more input devices are separate from the computer system. Therefore, the computer system can transmit data (e.g., image data or video data) via wired or wireless connections to an integrated or external display generating component to visually generate content (e.g., using a display device), and can receive input from one or more input devices via wired or wireless connections.

[0318] When an electronic device is connected to an external device (e.g., 650), it detects (802) an indication that the physical proximity between the electronic device and the external device meets a proximity condition (e.g., represented by 652). In some embodiments, the indication is provided in response to determining that the physical proximity between the electronic device and the external device meets the proximity condition. In some embodiments, the proximity condition is met if the distance between the electronic devices is less than a threshold distance (e.g., 6 inches, 12 inches, 18 inches, 24 inches).

[0319] In response to an indication that physical proximity meets a proximity condition, the electronic device displays (804) a first media information indication (e.g., 622) via a display device indicating that a first media item (e.g., a music track, an audiobook (or a portion thereof, such as a chapter), a podcast, a video, or a telephone call) is currently playing on an external device. In some embodiments, the first media information indication indicates the first media item based on the determination that the first media item is currently playing on an external device and is not currently playing on the electronic device. In some embodiments, the first media information indication includes a first set of information about the first media item (e.g., 622-1 to 622-4). Displaying the first media information indication indicating the first media item in response to an indication that physical proximity meets a proximity condition automatically provides feedback to the user, thereby reducing the amount of input required to display the first media information indication. Providing improved feedback when a set of conditions have been met, reducing the amount of input required to perform an operation, and performing an operation without further user input enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This also reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0320] The electronic device receives (806) a first input (e.g., 610f) indicating a selection of the first media information enablement representation.

[0321] In response to receiving the first input at (808), and based on determining that the first input is a first type of input (e.g., a tap gesture at a location corresponding to the first media information indication on the touch-sensitive display of the electronic device), the electronic device initiates (810) playback of the first media item on the electronic device.

[0322] In response to receiving a first input at (808), and based on determining that the first input is a second type of input different from the first type of input (e.g., a directional swipe gesture on the first media information display), the electronic device displays (812) a second media information display (e.g., 608) representing the first media item via a display device. This second media information display differs from the first media information display and includes a second set of information about the first media item. Playback of the first media item on the electronic device is initiated based on the type of input selected for the first media display, or the display of the second media information display representing the first media item is shown. Additional control options associated with the first media display are provided without cluttering the user interface with additional displayed controls. Providing additional control options without cluttering the UI with additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends the device's battery life by enabling the user to use the device more quickly and efficiently.

[0323] In some implementations, the second set of information includes the first set of information, but is displayed in a different manner. In some implementations, the second media information may include additional information (e.g., more details) about the first media item and / or a larger graphical representation of the first media item. In some implementations, the second set of information includes a queue of media items, including media items that are after the first media item in the queue (e.g., media items scheduled to play after the first media item) and / or media items that are before the first media item (e.g., media items played before the first media item or located before the first media item in a media item list such as a playlist).

[0324] In some implementations, the electronic device is not playing audio media items (e.g., when an indication is received that the physical proximity between the electronic device and the external device meets a proximity condition). Figure 6N The electronic device displays the first media information of the first media item currently being played on the external device.

[0325] The first media information indication includes an indication identifying the electronic device (e.g., 622-2). In some embodiments, the second media indication includes an indication identifying the electronic device. In some embodiments, the first media information indication includes an indication that selecting the first media information indication will initiate playback of the first media item on the electronic device.

[0326] In some embodiments, in response to receiving a first input, and based on determining that the first input is a first type of input, the electronic device changes the visual appearance of a first media information display (e.g., 622-2a) to indicate that playback of the first media item has been initiated on the electronic device. In some embodiments, changing the visual appearance of the first media information display includes displaying text indicating that playback of the first media item has been initiated on the electronic device, while optionally maintaining the display of at least some of the information in a first set of information about the first media item. In some embodiments, changing the visual appearance of the first media information display includes replacing the indication of the first media information display that the selection to initiate playback of the first media item on the electronic device with an indication (e.g., text) that playback of the first media item has been initiated on the electronic device.

[0327] In some implementations, the first media information indication includes an indication that identifies an external device as the source of the first media item (e.g., 622-1).

[0328] In some implementations, the first set of information indicating the first media item (e.g., 622-4, 902-4) corresponds to a type of media item. Exemplary types of media items include, but are not limited to, music tracks, audiobooks, podcasts, videos, and telephone calls. In some implementations, the first set of information for a telephone call media item includes an image of the caller's avatar.

[0329] In some implementations, as part of initiating playback of a first media item on an electronic device, the electronic device receives data of the first media item from an external device based on the determination that the first media item meets data transmission conditions (e.g., the first media item is provided by an application or account available on an external device but not on the electronic device). In some implementations, a media item queue including the first media item exists on the external device before playback is initiated. In some such implementations, initiating playback includes transferring the queue to the electronic device. In some implementations, as part of initiating playback of the first media item on an electronic device, the electronic device abandons receiving data of the first media item from the external device based on the determination that the first media item does not meet data transmission conditions (e.g., the electronic device is transmitting data of the first media item to an external device; the first media item is a telephone call; the first media item is provided by a specific application, service, or account that the electronic device has access to). Receiving (or not receiving) data of the first media item from the external device based on whether the first media item meets data transmission conditions avoids requiring the user to provide additional input to receive data. Performing an operation when a set of conditions has already been met and no further user input is required enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0330] In some implementations, the media item queue, including the first media item, exists on an external device before playback is initiated. In some such implementations, initiating playback includes transferring the queue to the electronic device. In some implementations, the first media item is accessible to both the electronic device and the external device via a public user account. In some implementations, the electronic device obtains the first media item from the public user account (e.g., playback of the first item does not include streaming the first media item from the external device to the electronic device). In some implementations, the electronic device obtains the queue from the public user account.

[0331] In some implementations, after initiating playback of the first media item on the electronic device, in response to determining that a time condition is met (e.g., a threshold amount of time has elapsed since the first input of the first type was received), the electronic device stops displaying a first media information indication of the first media item (see, for example...). Figure 6P ).

[0332] In some embodiments, after displaying a first media information indication representing a first media item and before receiving the first input, the electronic device detects an indication that the physical proximity between the electronic device and the external device satisfies a physical separation condition (e.g., represented by 656). In some embodiments, the indication is provided in response to determining that the physical proximity between the electronic device and the external device satisfies the physical separation condition. In some embodiments, the physical separation condition is satisfied if the distance between the electronic device and the external device exceeds a predefined threshold distance. In some embodiments, the threshold distance associated with the physical separation condition is greater than the first threshold distance associated with the (first) proximity condition. For example, the physical separation condition is satisfied if the distance between the electronic device and the external device exceeds a third threshold distance (e.g., 20 feet), and the first proximity condition is satisfied if the distance between the electronic device and the external device is less than a first threshold distance (e.g., 1-2 feet), wherein the third threshold distance is greater than the first threshold distance. In some embodiments, after displaying a first media information indication representing a first media item and before receiving the first input, in response to detecting an indication that the physical proximity satisfies the physical separation condition, the electronic device stops displaying the first media information indication representing the first media item (see, for example...). Figure 11D In response to an indication that physical proximity meets physical separation conditions, the display of the first media information enablement indicator for the first media item is stopped, providing feedback to the user that playback on the electronic device can no longer be initiated via selection of the first media information enablement indicator. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0333] In some implementations, the first type of input includes tap gestures (e.g., 610f), and the second type of input includes swipe or drag gestures (e.g., 610a; vertical up or vertical down swipe or drag gestures).

[0334] In some embodiments, when a second media information enablement indicating a first media item is displayed (e.g., 608), the electronic device receives a second input (e.g., 610c) indicating a selection of the second media information enablement. In some embodiments, in response to receiving the second input indicating a selection of the second media information enablement, the electronic device initiates playback of the second media item on the electronic device. In some embodiments, the second media item is the first media item. In some embodiments, the second media information enablement indicates a queue of media items including a representation of the first media item and representations of at least one other media item. In some such embodiments, the electronic device scrolls the queue in response to an input (e.g., a horizontal swipe or drag gesture). In some embodiments, in response to an input corresponding to a selection of a media item in the queue, the electronic device initiates playback of the selected media item on the electronic device. Displaying a second media information enablement (e.g., with additional information about the first media item) and initiating playback of the second media item on the electronic device in response to a selection of the second media information enablement provides improved feedback by providing the user with additional information and / or control over playback on an external device. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0335] In some implementations, in response to receiving a first input, and based on determining that the first input is a first type of input, the external device stops playing the first media item (see, for example...). Figure 6P In some implementations, the external device continues to play the first media item.

[0336] It should be noted that the process described above is different from that described in method 800 (e.g., Figure 8 The details of the above also apply in a similar manner to the methods described below. For example, method 1000 may optionally include one or more features of the various methods described above with reference to method 800. For example, operation 806 of method 800 may be performed to select the first media information enablement representation described in operation 1004 of method 1000, and operations 808, 810, and / or 812 may be performed as part of method 1000 in response to this selection. For the sake of brevity, these details will not be repeated below.

[0337] Figures 9A to 9D Exemplary methods and user interfaces for controlling media playback are illustrated according to some embodiments. In some embodiments, references are made to... Figures 9A to 9D The technology allows users to initiate playback of media items on different devices using touch input.

[0338] Figure 9A The above reference is shown. Figures 6A to 6P The aforementioned devices 600 and 650, and proximity condition range indicator 652. Additionally... Figure 9A Playback proximity condition range indicator 654 is shown. As detailed below, playback of a media item can be automatically initiated by placing the device 600 within the playback proximity condition range indicator 654 (e.g., without selecting media information enablement representations such as 606, 608, 616, 620, and 622).

[0339] Playback proximity condition range indicator 654 is included as a visual aid and is intended to indicate the physical proximity at which the playback proximity condition will be met. The presence of a detectable device (e.g., 600) (e.g., partially or completely) within the playback proximity condition range indicator 654 will satisfy the playback proximity condition, but if the detectable device is located outside the playback proximity condition range indicator 654, the playback proximity condition will not be satisfied. The graphical representation of the playback proximity condition range indicator 654 is not intended to limit the range for determining whether the playback proximity condition is met. Furthermore, the figures are not necessarily drawn to scale and are included herein only as a visual aid. Therefore, unless otherwise stated, the dimensions and scales of the features depicted in the figures are not intended as limitations on the distances required to be in very close proximity or to satisfy the playback proximity condition. However, as Figure 9A As indicated by arrows d1 and d2, the playback proximity condition range indicator 654 is inside the proximity condition range indicator 652 (e.g., d1 is greater than d2), such that the device 600 must be closer to the device 650 to satisfy the playback proximity condition associated with the playback proximity condition range indicator 654 compared to satisfying the proximity condition associated with the proximity condition range indicator 652.

[0340] Figure 9A A scenario is illustrated where device 600 is not within the proximity condition range indicator 652, device 600 has an active telephone call (as indicated by the telephone user interface 900 and audio activity indicator 625), and device 650 is not playing audio (as indicated by the audio activity indicator 675). Figure 9A In this context, device 600 provides audio output for telephone calls (as indicated by audio status indicator 900-1) and receives audio input for telephone calls. Although the reference to the telephone media item describes... Figures 9A to 9D However, the described technology can be applied to other types of media items (e.g., video calls, music tracks, audiobooks, podcasts). Similarly, the above references... Figures 6A to 6P The technology described can be applied to telephone calls.

[0341] Figure 9B The illustration shows that during a telephone call on device 600, device 600 moves at least partially within proximity condition range indicator 652. Since proximity condition range indicator 652 represents the physical proximity at which proximity conditions are met, device 600 detects an indication that the physical proximity between device 600 and device 650 meets the proximity conditions (e.g., and in response, initiates communication with device 650, such as to send the indication that the conditions are met). In some examples, device 650 detects an indication that the proximity conditions are met (e.g., and in response, initiates communication with device 600, such as to send the indication that the conditions are met).

[0342] In response to an indication that physical proximity meets a proximity condition, device 600 displays a media information enablement representation 902 indicating the media item currently playing on device 600 (e.g., a telephone call associated with user interface 900). Media information enablement representation 902 is similar to media information enablement representations 606, 608, 616, 620, and 622 applied to media items in a telephone call.

[0343] Media information display can optionally include providing information about Figure 9B The information of the telephone call includes several graphical elements, including: (1) an external device indicator 902-1, which indicates the device near which device 600 is located (e.g., device 650, also referred to as a kitchen speaker); (2) an action indicator 902-2, which indicates an action or function associated with media information enablement 902 (e.g., an action performed in response to selection by media information enablement 902); (3) a media item indicator 902-3, which provides information identifying the media item (e.g., the caller (“John Appleseed”) and the current call source “telephone”); and (4) a media item graphic 902-4, which includes an image associated with the media item. Figure 9B In the illustrated implementation, media item graphic 902-4 includes a representation of the caller at the other end of the call (e.g., an avatar) and an icon or thumbnail representing the telephone. In some implementations, the representation of the caller in media item graphic 902-4 includes the initials or full name of the caller.

[0344] Figure 9BThe illustration shows that when the media information enablement indicator 902 is displayed on device 600 and a telephone call is in progress on device 600, device 600 moves at least partially within the playback proximity condition range indicator 654. Since the playback proximity condition range indicator 654 represents the physical proximity at which the playback proximity condition is satisfied, device 600 detects an indication that the physical proximity between device 600 and device 650 satisfies the playback proximity condition (e.g., and in response, initiates communication with device 650, e.g., to send the indication that the playback proximity condition is satisfied). In some examples, device 650 detects an indication that the playback proximity condition is satisfied (e.g., and in response, initiates communication with device 600, e.g., to send the indication that the playback proximity condition is satisfied).

[0345] In response to an indication that the physical proximity meets the playback proximity condition, device 600 initiates playback of the telephone call on device 650. For example... Figure 9C As shown, in response to an indication that physical proximity meets playback proximity conditions, device 600 displays an updated media information indication 902a (similar to updated media information indications 616a and 622a), wherein the updated action indicator 902-2a indicates that a telephone call currently in progress on device 600 is being transferred to device 600. In some embodiments, initiating playback of a telephone call includes outputting the audio of the telephone call on device 650. In some embodiments, in response to an indication that physical proximity meets playback proximity conditions, device 650 begins receiving audio input for the telephone call (e.g., audio received by device 650 is transmitted to another participant). In some embodiments, device 600 maintains the source of the telephone call.

[0346] like Figure 9D As shown, after displaying the updated media information enablement 902a (e.g., in response to an indication that playback has been successfully started on device 650, or an indication that physical proximity meets the playback proximity condition, or after a predetermined amount of time has elapsed since displaying media information enablement 902a), device 600 stops displaying media information enablement 902a and updates user interface 900 to indicate that audio is being played back on device 650 (as indicated by the updated audio status indicator 900-1a).

[0347] Figure 10 This is a flowchart illustrating a method for controlling audio playback using an electronic device according to some embodiments. Method 1000 is performed at a device (e.g., 100, 300, 500, or 600) having a display device (e.g., 602). Some operations in method 1000 are optionally combined, some operations are optionally changed in order, and some operations are optionally omitted.

[0348] As described below, Method 1000 provides an intuitive way to control audio playback. This method reduces the cognitive burden on users when controlling audio playback, thus creating a more efficient human-computer interface. For battery-powered computing devices, it enables users to control audio playback more quickly and effectively, saving power and increasing the interval between battery charges.

[0349] In some embodiments, the electronic device (e.g., 600) includes a computer system. The computer system optionally communicates with a display generating component and one or more input devices (e.g., wired communication, wireless communication). The display generating component is configured to provide visual output, such as a display via a CRT monitor, a display via an LED monitor, or a display via image projection. In some embodiments, the display generating component is integrated with the computer system. In some embodiments, the display generating component is separate from the computer system. One or more input devices are configured to receive input, such as a touch-sensitive surface that receives user input. In some embodiments, one or more input devices are integrated with the computer system. In some embodiments, one or more input devices are separate from the computer system. Therefore, the computer system can transmit data (e.g., image data or video data) via wired or wireless connections to an integrated or external display generating component to visually generate content (e.g., using a display device), and can receive input from one or more input devices via wired or wireless connections.

[0350] When an electronic device is connected to an external device (e.g., 650), it detects (1002) an indication that the physical proximity between the electronic device and the external device meets a first proximity condition (e.g., represented by 652). In some embodiments, the indication is provided in response to determining that the physical proximity between the electronic device and the external device meets the first proximity condition. In some embodiments, the first proximity condition is met if the distance between the electronic device and the external device is less than a first threshold distance (e.g., 6 inches, 12 inches, 18 inches, 24 inches).

[0351] In response to an indication that physical proximity meets a first proximity condition, the electronic device displays (1004) a first media information enablement (e.g., 902) indicating a first media item (e.g., a music track, an audiobook (or a portion thereof, such as a chapter), a podcast, a video, a telephone call; a media item to be played on the electronic device (e.g., a media item currently playing on the electronic device, a recent media item that has been played on the electronic device and has ended, stopped, or paused, a media item scheduled to be played next on the electronic device when the user launches a media application or selects a playlist, such as the first track in a playlist); a media item to be played on an external device). Displaying the first media information enablement in response to an indication that physical proximity meets the first proximity condition automatically provides feedback to the user, thereby reducing the amount of input required to display the first media information enablement. Providing improved feedback when a set of conditions have been met, reducing the amount of input required to perform an operation, and performing an operation without further user input enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This also reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0352] When displaying a first media information enablement indication representing a first media item, the electronic device detects (1006) an indication that the physical proximity between the electronic device and the external device satisfies a second proximity condition (e.g., represented by 654). In some embodiments, the indication is provided in response to determining that the physical proximity between the electronic device and the external device satisfies the second proximity condition. In some embodiments, the second proximity condition is satisfied if the distance between the electronic device and the external device is less than a second threshold distance, wherein the second threshold distance is less than a first threshold distance (e.g., 0 inches, 0.5 inches, 1 inch, 1.5 inches, 2 inches, less than 6 inches). In some embodiments, the second proximity condition corresponds to an electronic device that is closer to the external device than the proximity required to satisfy the first proximity condition (e.g., the proximity required to trigger the display of the first media information enablement indication).

[0353] In response to an indication that the physical proximity meets a second proximity condition, the electronic device initiates (1008) playback of the first media item (e.g., Figure 9CIn some embodiments, playback of the first media item is initiated without detecting a selection of the first media information indication. In some embodiments, initiating playback of the first media item causes the first media item to play on an electronic device. In some embodiments, initiating playback of the first media item causes the first media item to play on an external device. In some embodiments, initiating playback of the first media item includes transmitting a command to an external device to initiate playback of the first media item on the external device. Initiating playback of the first media item in response to an indication that physical proximity meets a second proximity condition automatically provides feedback by instructing the user that physical proximity meets the proximity condition, thereby reducing the amount of input required to display the first media information indication. Providing improved feedback when a set of conditions has been met, reducing the amount of input required to perform an operation, and performing an operation without further user input enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0354] In some implementations, a first proximity condition is met if the physical proximity between the electronic device and the external device is less than a first threshold distance (e.g., 652, D1). In some implementations, a second proximity condition is met if the physical proximity between the electronic device and the external device is less than a second threshold distance (e.g., 654, D2), where the second threshold distance is less than the first threshold distance.

[0355] In some embodiments, the first media information capability indication includes an indication identifying a device on which playback of the first media item can be initiated (e.g., 902-1, 902-2). In some embodiments, the second media information capability indication includes an indication identifying a device (e.g., a target device) that is initiating playback of the first media item. In some embodiments, the first media information capability indication includes an indication of a device on which playback of the first media item can be initiated, and the selection of the first media information capability indication will initiate playback of the first media item on the indicated device.

[0356] In some embodiments, as part of initiating playback of the first media item, the electronic device alters (e.g., 902-2a) the visual appearance of the first media information display to indicate that playback of the first media item has been initiated. In some embodiments, altering the visual appearance of the first media information display includes displaying text indicating that playback of the first media item has been initiated, while optionally maintaining the display of at least some information about the first media item. In some embodiments, altering the visual appearance of the first media information display includes replacing the indication of the first media information display that the selection to initiate playback of the first media item with an indication (e.g., text) that playback of the first media item has been initiated.

[0357] In some implementations, the first media information enablement indication (e.g., 606-5, 622-4, 902-4) corresponds to a type of media item of the first media item. Exemplary types of media items include, but are not limited to, music tracks, audiobooks, podcasts, videos, and telephone calls. In some implementations, the first media information enablement indication for a telephone call media item includes the avatar of the caller.

[0358] In some implementations, as part of initiating playback of a first media item, the electronic device transfers data of the first media item between the electronic device and an external device based on the determination that the first media item meets data transfer conditions (e.g., the first media item is provided by an application, service, or account that is not available on the device on which playback of the first media item is initiated). In some implementations, a media item queue including the first media item exists on the device that was originally the source of the first media item before playback was initiated. In some such implementations, initiating playback includes transferring the queue to the device on which playback of the first media item is being initiated. In some implementations, as part of initiating playback of the first media item, the electronic device waives the right to transfer data for the first media item between the electronic device and the external device based on the determination that the first media item does not meet data transfer conditions (e.g., the first media item is provided by a specific application, service, or account that both the electronic device and the external device have access to). Transferring (or not transferring) data of the first media item between the electronic device and the external device based on whether the first media item meets data transfer conditions avoids requiring the user to provide additional input for data transfer. Performing an operation when a set of conditions has already been met and no further user input is required enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0359] In some implementations, the queue of media items, including the first media item, exists before playback is initiated (e.g., on the electronic device or external device). In some such implementations, initiating playback includes transferring the queue between the electronic device and the external device. In some implementations, the first media item is accessible to both the electronic device and the external device via a public user account. In some implementations, the device initiating playback obtains the first media item from the public user account (e.g., playback of the first item does not include streaming the first media item between the electronic device and the external device). In some implementations, the device initiating playback obtains the queue from the public user account.

[0360] In some implementations, after initiating playback of the first media item, in response to determining that a time condition is met (e.g., a threshold amount of time has elapsed since an indication that physical proximity meets a second proximity condition is detected), the electronic device stops displaying a first media information indication of the first media item (see, for example...). Figures 9C to 9D ).

[0361] In some embodiments, after displaying a first media information indication representing a first media item and before detecting an indication that the physical proximity between the electronic device and the external device meets a physical separation condition (e.g., represented by 656), the electronic device detects an indication that the physical proximity between the electronic device and the external device meets a physical separation condition. In some embodiments, an indication is provided in response to determining that the physical proximity between the electronic device and the external device meets the physical separation condition. In some embodiments, the physical separation condition is met if the distance between the electronic device and the external device exceeds a predefined threshold distance. In some embodiments, the threshold distance associated with the physical separation condition is greater than the threshold distance associated with the first proximity condition (e.g., a first threshold distance). For example, the physical separation condition is met if the distance between the electronic device and the external device exceeds a third threshold distance (e.g., 20 feet), and the first proximity condition is met if the distance between the electronic device and the external device is less than the first threshold distance (e.g., 1-2 feet), wherein the third threshold distance is greater than the first threshold distance. In some implementations, after displaying a first media information enablement indication of a first media item and before detecting an indication that a second proximity condition is met, the electronic device stops displaying the first media information enablement indication of the first media item in response to an indication that a physical proximity condition is met (see, for example...). Figures 11C to 11DIn response to an indication that physical proximity meets physical separation conditions, the display of the first media information power indicator for the first media item is stopped, providing feedback to the user that playback can no longer be initiated by selecting the first media information power indicator. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0362] In some embodiments, before detecting an indication that physical proximity meets a second proximity condition, the electronic device detects an input indicating a selection of a first media information display (e.g., 610a, 610d, 610e, 610f). In some embodiments, before detecting an indication that physical proximity meets a second proximity condition, in response to receiving a first input, based on determining that the first input is a first type of input (e.g., a tap gesture at a location corresponding to the first media information display on the touch-sensitive display of the electronic device), the electronic device initiates playback of a first media item. In some embodiments, before detecting an indication that physical proximity meets a second proximity condition, in response to receiving a first input, based on determining that the first input is a second type of input different from the first type of input (e.g., a directional swipe gesture on the first media information display), the electronic device displays a second media information display (e.g., 608) indicating the first media item via a display device, which is different from the first media information display. Based on the type of input selected for the first media display, playback of the first media item is initiated, or a second media information display representing the first media item is shown. Additional control options related to the first media display are provided without cluttering the user interface with additional displayed controls. Providing additional control options without cluttering the UI with additional displayed controls enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device). This, in turn, reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0363] In some embodiments, when a second media information display indicating a first media item is shown, the electronic device receives a second input (e.g., 610c) indicating a selection of the second media information display. In some embodiments, in response to receiving the second input indicating a selection of the second media information display, the electronic device initiates playback of the second media item. In some embodiments, the second media item is the first media item. In some embodiments, the second media information display represents a queue of media items including a representation of the first media item and representations of at least one other media item. In some such embodiments, the electronic device scrolls the queue in response to an input (e.g., a horizontal swipe or drag gesture). In some embodiments, in response to an input corresponding to a selection of a media item in the queue, the electronic device initiates playback of the selected media item. Displaying a second media information display (e.g., with additional information about the first media item) and initiating playback of the second media item in response to a selection of the second media information display provides improved feedback by providing the user with additional information and / or control over playback. Providing improved feedback enhances device operability and makes the user-device interface more efficient (e.g., by helping users provide appropriate input and reducing user errors when operating / interacting with the device), which in turn reduces power consumption and extends device battery life by enabling users to use the device more quickly and effectively.

[0364] In some implementations, as part of initiating playback of the first media item, the first media item is played on the electronic device when an indication is received that the physical proximity between the electronic device and the external device meets a second proximity condition (e.g., Figure 6G The electronic device stops playing the first media item on the electronic device and initiates playback of the first media item on an external device (e.g., Figure 6I In some implementations, as part of initiating playback of the first media item, upon determination that the first media item is being played on the external device when a second proximity condition is detected, the electronic device stops playing the first media item on the external device and initiates playback of the first media item on the electronic device.

[0365] In some implementations, a first media item is playing on the electronic device when an indication is detected that the physical proximity between the electronic device and the external device meets a first proximity condition (e.g., see...). Figure 6G and Figure 6KIn some embodiments, initiating playback of the first media item includes initiating playback of the first media item on an external device. In some embodiments, a first media information enablement indicates the first media item when the electronic device is playing the first media item, based on an indication that a first physical proximity condition is met between the electronic device and the external device. In some embodiments, based on determining that the first media item is currently playing on the electronic device, an instruction to transmit to the external device is executed to initiate playback of the first media item on the external device. In some embodiments, initiating playback of the first media item includes stopping playback of the first media item on the electronic device.

[0366] In some implementations, as part of initiating playback of the first media item on the external device, the electronic device stops playing the third media item on the external device (e.g., Figure 6L ).

[0367] In some implementations, when an indication is received that the physical proximity between the electronic device and the external device meets a proximity condition, neither the electronic device nor the external device is playing a media item (e.g., Figure 6B The first media item is the media item most recently played on the electronic device before an indication is detected that the physical proximity between the electronic device and the external device meets a proximity condition. In some embodiments, the electronic device initiates playback of the first media item on the external device as part of initiating playback of the first media item. In some embodiments, the first media information indication indicates the media item most recently played on the electronic device, based on the determination that the external device was not playing a media item when an indication is detected that the physical proximity between the electronic device and the external device meets a proximity condition.

[0368] In some implementations, upon determination that the physical proximity between the electronic device and the external device meets a first proximity condition, the electronic device does not play a media item, and the external device is playing a media item (e.g., Figure 6N The first media item is a media item played on an external device. In some implementations, the electronic device initiates playback of the first media item on the electronic device as part of initiating playback of the first media item.

[0369] Note that the above reference method 1000 (for example, Figure 10The details of the process described above also apply in a similar manner to the methods described above. For example, method 700 and / or method 800 optionally include one or more features of the various methods described above with reference to method 1000. For example, operations 1006 and 1008 of method 1000 can be applied to method 700 and / or method 800 to initiate playback of the first media item described in method 700 and / or method 800. For the sake of brevity, these details will not be repeated below.

[0370] Figures 11A to 11D Techniques for stopping the display of graphical elements such as media information display representations 606, 608, 616, 620, 622 and 902 are shown. Figure 11A Devices 600 and 650, and proximity condition range indicator 652 are shown, as described above. Furthermore, Figure 11A An element removal condition range indicator 656 is included. The element removal condition range indicator 656 is included as a visual aid and is intended to indicate the range within which an element removal condition will be met. The presence of a detectable device (e.g., 600) (e.g., partially or completely) outside the element removal condition range indicator 656 will satisfy the element removal condition, but if the detectable device is within the element removal condition range indicator 656, the element removal condition will not be satisfied. The graphical representation of the element removal condition range indicator 656 is not intended to limit the range for determining whether an element removal condition is satisfied. Furthermore, the figures are not necessarily drawn to scale and are included herein only as a visual aid. Therefore, unless otherwise stated, the dimensions and scale of the features depicted in the figures are not intended as limitations on the distances required to satisfy the element removal condition. However, the element removal condition range indicator 656 is outside the proximity condition range indicator 652, such that device 600 must be further away from device 650 than to satisfy the proximity condition associated with proximity condition range indicator 652 to satisfy the element removal condition associated with element removal condition range indicator 656.

[0371] Figure 11A This illustrates a scenario where device 600 is not within the proximity condition range indicator 652 (e.g., Figure 6A , Figure 6F , Figure 6J , Figure 6M or Figure 9A ). Figure 11B This shows that device 600 moves at least partially within proximity condition range indicator 652 (e.g., Figure 6B , Figure 6G , Figure 6K , Figure 6N or Figure 9BSince the proximity condition range indicator 652 represents the physical proximity that satisfies the proximity condition, device 600 detects an indication that the physical proximity between device 600 and device 650 satisfies the proximity condition (e.g., and in response, initiates communication with device 650, for example, to send the indication that the condition is satisfied). In some examples, device 650 detects an indication that the proximity condition is satisfied (e.g., and in response, initiates communication with device 600, for example, to send the indication that the condition is satisfied). In response to detecting an indication that the physical proximity satisfies the proximity condition, device 600 displays a media information enablement representation 1100 representing a media item (e.g., media information enablement representation 606, 608, 616, 620, 622, or 902). In some embodiments, the media item represented by the media information enablement representation is based on the audio output state of device 600 and / or device 650, as described above.

[0372] Figure 11C This illustrates that when the media information display indicator 1100 is active, the device 600 is moved outside the proximity condition range indicator 652, but remains within the element removal condition range indicator 656. (As shown...) Figure 11C As shown, device 600 continues to display media information display indicator 1100.

[0373] Figure 11D The illustration shows that when displaying media information enable indicator 1100, device 600 moves outside element removal condition range indicator 656. Since element removal condition range indicator 656 represents the physical range that satisfies the element removal condition, device 600 detects that the physical range between device 600 and device 650 satisfies the element removal condition (e.g., and in response, initiates communication with device 650, e.g., to send an indication that the element removal condition is satisfied). In some examples, device 650 detects the indication that the element removal condition is satisfied (e.g., and in response, initiates communication with device 600, e.g., to send an indication that the element removal condition is satisfied). In response to detecting the indication that the physical range between device 600 and device 650 satisfies the element removal condition, device 600 stops displaying media information enable indicator 1100 (e.g., removes media information enable indicator 1100 from display 602).

[0374] Figure 12 Figure 1200 includes a physical structure according to some embodiments and a set of exemplary devices located in and around the physical structure. Figure 12 Used to illustrate the process described below, including Figure 14 , Figure 16 , Figure 18 , Figure 20 and Figure 23 The process is shown in the diagram.

[0375] Figure 1200 includes a home 1202 and a set of accessory devices (e.g., device 1246, device 1204, etc.). The interior of home 1202 includes multiple rooms, such as a kitchen 1202a, a dining room 1202b, a bedroom 1202c, a living room 1202d, and a sunroom 1202e. Kitchen 1202a is located in the upper left portion of home 1202, and dining room 1202b is located between kitchen 1202a and bedroom 1202c, with bedroom 1202c to the right of dining room 1202b. There is no wall between kitchen 1202a and dining room 1202b, but a wall exists between dining room 1202b and bedroom 1202c. Living room 1202d is located in the lower left portion of home 1202, and sunroom 1202e is located in the lower right portion of home 1202. A door frame 1202f is located on the exterior of home 1202. In some embodiments, home 1202 includes, except... Figure 12 Rooms other than those depicted and / or excluded Figure 12 The room depicted in Figure 1200. Although Figure 1200 depicts a home, it should be recognized that this is merely an example, and the techniques described herein can work with other types of physical structures such as office buildings, hotels, apartments, etc. Figure 12 As shown, each part of home 1202 (e.g., a room, a hallway) has at least one device. However, in some embodiments, a part of home 1202 does not have any devices.

[0376] like Figure 12 As shown, kitchen speaker 1206 is positioned on the countertop between the stove and the wall in kitchen 1202a. Dining room light 1212 and dining room speaker 1216 are located in dining room 1202b. Dining room light 1212 is positioned on the dining table, and dining room speaker 1216 is positioned to the right of the dining table. Additionally, a thermostat 1218 is present on the wall of dining room 1202b. In bedroom 1202c, bedroom light 1232 is adjacent to one side of the bed, and bedroom speaker 1236 is positioned on the opposite side of the bed. In living room 1202d, living room speaker 1246 and living room speaker 1248 are located on the opposite side of television 1244, and living room light 1242 is located in front of television 1244. In sunroom 1202e, sunroom speaker 1256 and sunroom speaker 1258 are positioned on opposite sides of the room. The equipment located on porch 1202f includes porch light 1262, camera doorbell 1264, and doorbell 1266.

[0377] In this example, Figure 12The speakers shown (e.g., kitchen speaker 1206, dining room speaker 1216, bedroom speaker 1236, living room speaker 1246, living room speaker 1248, sunroom speaker 1256, and sunroom speaker 1258), lights (e.g., dining room light 1212, bedroom light 1232, living room light 1242, porch light 1262), thermostat 1218, television 1244, doorbell camera 1264, and doorbell 1266 are assigned to home 1202 (e.g., programmedly mapped to (e.g., via a user account associated with home 1202) a corresponding set of devices). Furthermore, each individual speaker, light, television, thermostat, camera, and doorbell is also assigned to a portion of the house (e.g., a room, that portion) (e.g., mapped to a group corresponding to that portion). For example, kitchen speaker 1206 is assigned to kitchen 1202a (e.g., programmed to be mapped to a group corresponding to that kitchen); dining room light 1212, dining room speaker 1216, and thermostat 1218 are assigned to dining room 1202b; bedroom light 1232 and bedroom speaker 1236 are assigned to bedroom 1202c; living room light 1242, television 1244, living room speaker 1246, and living room speaker 1248 are assigned to living room 1202d; sunroom speaker 1256 and sunroom speaker 1258 are assigned to sunroom 1202e; and doorbell 1262, camera doorbell 1264, and doorbell 1266 are assigned to doorbell 1202f. In some embodiments, each group corresponding to a part of home 1202 is also assigned to the entire home 1202. Therefore, in this example, devices mapped to a room in home 1202 or a porch of home 1202 are also mapped to the entire home 1202.

[0378] In some implementation schemes, Figure 12 The speaker, light, camera, doorbell, and television shown are smart devices that are directly connected to each other or indirectly connected to each other via one or more networks (e.g., wireless networks such as Bluetooth, NFC, Wi-Fi, 4G, etc.). In some embodiments, the speaker includes one or more components of device 650 as described above. In some embodiments, other smart devices, such as smart appliances, smart thermostats, smart plugs and sockets, are connected to one or more networks. In some embodiments, one or more personal devices may be assigned to home 1202. In some embodiments, the personal devices assigned to home 1202 include electronic device 600. In some embodiments, the personal devices include one or more tablets, smartwatches, laptops, desktop computers, etc.

[0379] Figures 13A to 13M Exemplary user interfaces for managing controls are shown according to some implementation schemes. The user interfaces in these figures are used to illustrate the processes described below, including... Figure 14 The process is shown.

[0380] Figures 13A to 13I An exemplary scenario is shown for managing the display of multiple controls based on the user context of device 600. Figures 13A to 13I Most of these include a schematic diagram of a home (e.g., home 1202) and a representation of the physical characteristics of the equipment relative to the home (e.g., the location of the equipment relative to the walls or rooms of the home). For example, Figure 13A The diagram 1202 includes an indication (e.g., a symbol of a person) at location 1302a, representing the location of device 600. The schematic diagram is provided for illustrative purposes only as a visual aid to the description. Therefore, the schematic diagram is not intended to limit the range of determinations regarding whether device 600 is in a particular location. Furthermore, the drawings are not necessarily drawn to scale and are included herein only as a visual aid. Therefore, unless otherwise stated, the dimensions and scales of the features depicted in the drawings are not intended as limitations on the required location of device 600 in a particular use context.

[0381] Figure 13A A device 600 displaying a user interface 1320 is shown. The user interface 1320 includes a dynamic control portion 1324, which is visually positioned between a static control portion 1322 (located near the top of the user interface 1320) and a static control portion 1326 (located near the bottom of the user interface 1320). Each control portion (e.g., 1324, 1322, and 1326) is positioned... Figure 13A The interface is depicted as including multiple controls. However, as described herein, when the usage context of device 600 changes (e.g., as described herein), the controls presented in the static control portions (e.g., 1322 and 1326) of user interface 1320 do not change. Therefore, even when the usage context of the device changes, the controls presented in the static control portions still exist. However, when the usage context of device 600 changes (e.g., as described herein), the controls presented in the dynamic control portions (e.g., 1324) of user interface 1320 may change (e.g., when device 600 does not detect user input pointing to one or more controls). Therefore, the controls presented in the dynamic control portion 1324 are dynamic based on the changing usage context.

[0382] like Figure 13A As shown, the static control sections (e.g., 1322 and 1326) include multiple controls (“local operation controls”) for performing local operations of device 600. For example, as Figure 13AAs shown, the static control section 1322 includes multiple local operation controls, such as a wireless connection control 1322a, a cellular connection control 1322b, a music application control 1322c, and an orientation lock control 1332d. Similarly, the static control section 1326 includes a flashlight control 1332e and a calculator control 1332f. Each device control on the user interface 1320 modifies the state of device 600 when selected (e.g., without modifying the state of accessory devices such as those in home 1202).

[0383] Device controls can modify the state of device 600 in different ways. In some embodiments, the state of device 600 is modified when one or more settings of device 600 are changed, such as connectivity settings (e.g., wireless, cellular, Bluetooth settings) or display settings (e.g., changing or disabling the connection to a display when the orientation of device 600 changes) are turned on / off. For example, in some embodiments, device 600 detects a selection of wireless connectivity control 1322a and, in response to detecting that selection, modifies the wireless connectivity state of device 600 (e.g., turning on / off the ability of device 600 to connect to a wireless network). In some embodiments, the state of device 600 is modified when the media output of device 600 changes. For example, in some embodiments, device 600 detects a selection of music application control 1332c and, in response to detecting that selection, modifies (e.g., rewind, fast forward, play) the state of the media output of device 600. In some embodiments, the state of device 600 is modified when the display state of device 600 changes (e.g., displaying a different user interface in response to receiving a selection of one of the local operation controls, or displaying a different set of controls in response to receiving a selection of one of the local operation controls). For example, in some embodiments, device 600 detects a selection of calculator control 1332f and, in response to detecting that selection, modifies the state of device 600 by replacing user interface 1320 with the user interface of a calculator application. In some embodiments, the state of device 600 is modified when one or more hardware components of device 600 are turned on / off. For example, in some embodiments, device 600 detects a selection of flashlight control 1332e and, in response to detecting that selection, modifies the state of a light that is part of device 600.

[0384] like Figure 13A As shown, the dynamic control section 1324 includes dynamic accessory device controls displayed based on the usage context in which device 600 is operating, such as the home arrival scene control 1334a and the porch light control 1334b. These dynamic accessory device controls are used to control accessory devices external to device 600. Figure 13AIn the context of home scene control 1334a and porch light control 1334b, these are controls used to control accessory devices in home 1202 (e.g., as mentioned above regarding...). Figure 12 (as stated). In Figure 13A In this implementation, the home scene control 1334a is a macro that controls multiple devices. Therefore, in some embodiments, device 600 detects a tap gesture on the home scene control 1334a and, in response to detecting the tap gesture, transmits instructions to turn on the thermostat 1218 and set it to the desired temperature, turn on the living room light 1242, and play jazz music from the living room speaker 1248. In contrast, the porch light control 1334b controls only one device (e.g., porch light 1262) and performs one function (e.g., turning the porch light 1262 on / off). Therefore, in some embodiments, device 600 detects a tap gesture on the porch light control 1334b and, in response to detecting the tap gesture, transmits instructions to turn the porch light 1262 on or off.

[0385] As mentioned above Figure 13A As described above, the Home Scene control 1334a and the Pedestrian Light control 1334b are displayed in the Dynamic Controls section 1324 because the device 600 operates in a usage context corresponding to a location outside the home 1202 (e.g., as shown in 1302a). Therefore, in some embodiments, when the device 600 displays the Home Scene control 1334a and the Pedestrian Light control 1334a in the Dynamic Controls section 1324, it has been determined that the device 600 is outside the home 1202 (or in…). Figure 13A When operating in the corresponding usage context (as described in the example), these specific controls are more useful for users operating device 600.

[0386] like Figure 13A As shown, the home control 1333 is also displayed in the dynamic section 1324. Figure 13A In this context, home control 1333 is a control corresponding to home 1202, such that when home control 1333 is selected, device 600 displays one or more accessory device controls that are not currently displayed. Furthermore, the selection of home control 1333 is an access point that allows device 600 to navigate to a user interface (e.g., one or more user interface screens) with a complete set of accessory device controls that can be used to control the accessory devices assigned to home 1202.

[0387] exist Figure 13AIn this configuration, device 600 is associated with multiple households (e.g., authorized to manage accessory devices assigned to different households). Household control 1333 includes an identifier corresponding to household 1202 (e.g., "123 Avenue," the address of household 1202), because device 600 is currently configured to display accessory devices assigned to household 1202 rather than another household associated with device 600. In some implementations, device 600 is associated with a household when it is registered (e.g., in one or more databases) as one or more of the household's users, managers, guests, etc.

[0388] In some implementations, when device 600 is configured to display an accessory device control assigned to another household, device 600 displays on household control 1333 the same as... Figure 13A The identifiers displayed are different identifiers. In some embodiments, when device 600 is configured to display accessory devices assigned to another household, device 600 displays different dynamic device controls (e.g., device controls associated with a specific household). In some embodiments, when it is determined that the corresponding household is the closest household associated with device 600, device 600 is configured to display accessory device controls assigned to that household. In some of these embodiments, the determination is performed only if the "Current Household" setting is set to active (e.g., determination of the closest household). In some embodiments, when the "Current Household" setting is set to inactive, device 600 is configured to display accessory device controls assigned to the last household, and device 600 displays a set of accessory device controls for that last household. In some embodiments, device 600 is configured to display accessory device controls assigned to the last household, and device 600 transmits instructions for that last household, causing at least one accessory device of the corresponding household to perform an operation. In some embodiments, when device 600 is associated with only one household, device 600 displays "House" as an identifier on household control 1333. In some implementations, when device 600 is configured to display accessory device controls assigned to households that do not have any controls (or when the controls assigned to the household do not meet certain criteria (e.g., a specific number or type of accessory), device 600 resets the size of the household control 1333 (e.g., resets it to be larger than...). Figure 13A The dimensions depicted in the figure make the home control 1333 actually fill (e.g., occupy two rows) most of the dynamic section 1324.

[0389] exist Figure 13B The diagram illustrates that device 600 has moved from position 1302a (e.g., at...). Figure 13A(middle) Move to position 1302b. As shown in the diagram, device 600 is inside house 1202, and specifically in bedroom 1202c, while device 600 is at position 1302b.

[0390] exist Figure 13B At the location, it was determined that equipment 600 was in contact with... Figure 13A The device 600 operates in different usage contexts. Here, the device 600 operates in a different usage context because it has moved from a location outside the house 1202 (e.g., 1302a) to a location inside the house 1202 (e.g., 1302b). Specifically, in Figure 13B The device 600 is determined to be operating in a usage context corresponding to the device 600 being located inside the bedroom 1202c (and / or inside the house 1202).

[0391] like Figure 13B As shown, because the device is determined to be located within home 1202 (e.g., the device is located in a bedroom), device 600 updates the dynamic control portion 1324 to display the same information as the device. Figure 13A The image shows a different set of dynamic accessory device controls. For example... Figure 13B As shown, device 600 displays a reading time scene control 1334c, a bedroom light control 1334d (e.g., when selected, causes the bedroom light 1232 to turn on / off), a bedroom speaker control 1334e (e.g., when selected, causes the bedroom speaker 1236 to pause or initiate media playback), and a... Figure 13A The dynamic controls section 1324 displays different goodnight scene controls 1334f. Furthermore, Figure 13B The controls shown are used to control one or more accessory devices in bedroom 1202c (e.g., bedroom light 1232, bedroom speaker 1236), where Figure 13A There are no controls in the room for controlling any accessories in the bedroom. Therefore, in Figure 13B In the context of user and device 600 being located in bedroom 1202c, the controls displayed in the dynamic controls section 1324 may be more... Figure 13A The controls displayed in the dynamic controls section 1324 are more relevant to the user.

[0392] like Figure 13B As shown, regardless of how device 600 operates in different usage contexts, device 600 continues to display home control 1333. Figure 13B As shown, the home control 1333 has the same characteristics as... Figure 13A The home control 1333 shown in the image is the same size as the home control 1333 and is displayed in the same position in the dynamic control section 1324.

[0393] When comparing Figure 13A and Figure 13B When the dynamic control portion 1324 is in use, its size remains the same, occupying the same area of ​​the user interface 1320. However, in Figure 13A The dynamic control section 1324 displays three controls (e.g., home control 1333, home scene control 1334a, porch light control 1334b), while... Figure 13B The dynamic control section 1324 displays five controls (e.g., home control 1333, reading scene control 1334c, bedroom light control 1334d, bedroom speaker control 1334e, and goodnight scene control 1334f). Therefore, as... Figures 13A to 13B As shown, device 600 displays different numbers of controls in the dynamic control portion 1324 without changing the size of the dynamic control portion 1324. When comparing... Figures 13A to 13B When the number of controls displayed in the dynamic control section 1324 is larger due to the specific usage context of the device 600, the device 600 implements the user interface by displaying smaller-sized accessory device controls. For example, Figure 13A The area occupied by the porch control 1334b in the dynamic control portion 1324, together with the reading scene control 1334c and the goodnight scene control 1334f, is... Figure 13B The areas occupied by the control 1334a and the bedroom light control 1334d and bedroom speaker control 1334e are roughly the same. Furthermore, the area occupied by the dynamic control portion 1324 of the home scene control 1334a, along with the bedroom light control 1334d and bedroom speaker control 1334e, is... Figure 13B The areas they occupy are roughly the same.

[0394] like Figure 13B As shown, the home control 1333 maintains consistency with the dynamic control section 1324. Figure 13A The home control 1333 has the same size and position as the other controls in the dynamic control section 1324. Therefore, unlike the other controls in the dynamic control section 1324, the home control 1333 maintains the same size and is displayed in the same position as the other controls in the dynamic control section 1324. Figures 13A to 13B The size and / or position of the home 1333 will change when the device 600 operates in different usage contexts or based on the number of other controls displayed in the dynamic control section 1324 (e.g., a number greater than 0). Figure 13B In the middle, user interface 1320 is displaying 22:10 (10:10 PM).

[0395] exist Figure 13C Although equipment 600 is located there. Figure 13C The location and equipment 600 in Figure 13B Although they are in the same position, device 600 is operating in a different usage context. Here, device 600 operates in a different usage context because device 600 is in a different context than user interface 1320. Figure 13B The user interface displays the time shown in the image (22:10) at different times (e.g., 06:42 (6:42 AM)).

[0396] like Figure 13C As shown, because device 600 operates in the usage context corresponding to the user interface 1320 displayed at 06:42, device 600 displays a different set of controls. For example... Figure 13C As shown, device 600 has replaced the previous one with the Good Morning scene control 1334g. Figure 13B The "Goodnight" scene control 1334f is displayed in the image. In other words, in... Figure 13C At this point, device 600 is displayed in the previous usage context (e.g., in...). Figure 13B The same number of controls are displayed in the device 600; however, these group controls are different because the device 600 is based on different usage contexts. In some implementations,...

Claims

1. A method for displaying a user interface, comprising: At the computer system that communicates with the display generation component and one or more input devices: The user interface is displayed via the display generation component, the user interface including: A first optional user interface object for controlling a first group of one or more media playback devices; A second optional user interface object, the second optional user interface object being used to control a second group of one or more media playback devices different from the first group of one or more media playback devices; and Instructions for selecting the first optional user interface object; When the user interface includes the instruction, user input corresponding to the selection of the second optional user interface object is received; and In response to receiving user input corresponding to the selection of the second optional user interface object, and based on determining that a set of criteria is met, wherein the set of criteria includes criteria that are met when the indication to select the first optional user interface object and the indication to select the second optional user interface object are simultaneously displayed for a predetermined time period: The user interface displays a third optional user interface object, which is used to control a third group of one or more media playback devices, including the first group of one or more media playback devices and the second group of one or more media playback devices; and Stop displaying the first optional user interface object and the second optional user interface object.

2. The method according to claim 1, further comprising: When the third optional user interface object is displayed, a first user input corresponding to the selection of the third optional user interface object is received; as well as In response to receiving the first user input corresponding to the selection of the third optional user interface object: Based on the user interface including an instruction to select the third optional user interface object, stop displaying the instruction to select the third optional user interface object in the user interface; and The instruction to select the third optional user interface object is displayed in the user interface, provided that the user interface does not include the instruction to select the third optional user interface object.

3. The method according to any one of claims 1 to 2, wherein: The first optional user interface object includes an identifier associated with the first group of one or more media playback devices; The second optional user interface object includes an identifier associated with the second group of one or more media playback devices; as well as Displaying the third optional user interface object includes displaying an identifier associated with the third group of one or more media playback devices, wherein the identifier associated with the third group of one or more media playback devices includes a portion representing the identifier associated with the first group of one or more media playback devices, and a portion representing the identifier associated with the second group of one or more media playback devices.

4. The method according to any one of claims 1 to 2, further comprising: When the third optional user interface object is displayed, a second user input corresponding to the selection of the third optional user interface object is received; as well as In response to receiving the second user input corresponding to the selection of the third optional user interface object: Displays a first optional user interface object for controlling the first group of one or more media playback devices and the indication for selecting the first optional user interface object; as well as Displays a second optional user interface object for controlling the second group of one or more media playback devices and the indication for selecting the second optional user interface object.

5. The method according to claim 4, further comprising: When displaying the first optional user interface object for controlling the first group of one or more media playback devices and the instruction for selecting the first optional user interface object, and when displaying the second optional user interface object for controlling the second group of one or more media playback devices and the instruction for selecting the second optional user interface object, one or more user inputs corresponding to one or more selections of the first optional user interface object are received; as well as In response to receiving the one or more user inputs: Based on the determination of the last of the one or more selections of the first optional user interface object, the computer system stops displaying the instruction to select the first optional user interface object, and continues to display the first optional user interface object and the second optional user interface object, but does not display the third optional user interface object.

6. The method according to any one of claims 1 to 2, wherein: The user interface includes a fourth optional user interface object for controlling a fourth group of one or more media playback devices; as well as The method further includes: When displaying the third optional user interface object and the fourth optional user interface object, receive user input corresponding to the selection of the fourth optional user interface object; and In response to receiving the user input corresponding to the selection of the fourth optional user interface object: The user interface displays a fifth optional user interface object for controlling the first group of one or more media playback devices, the second group of one or more media playback devices, and the fourth group of one or more media playback devices; and Stop displaying the third optional user interface object and the fourth optional user interface object.

7. The method according to any one of claims 1 to 2, further comprising: In response to receiving user input corresponding to the selection of the second optional user interface object, grouping information indicating the grouping of the first optional user interface object and the second optional user interface object is transmitted to a second computer system different from the computer system.

8. The method according to any one of claims 1 to 2, further comprising: After the third optional user interface object is displayed in the user interface, data corresponding to the indication that the first group of one or more media playback devices are connected to the computer system is received; as well as In response to receiving the data, a notification is displayed instructing one or more media playback devices in the first group and one or more media playback devices in the second group to connect to the computer system.

9. The method according to any one of claims 1 to 2, further comprising: When the user interface is displayed, user input is received at a first location on the user interface; as well as In response to receiving the user input at the first location on the user interface, a second user interface is displayed via the display generation component. The second user interface includes an optional user interface object that, when selected via the one or more input devices, causes one or more external media playback devices that are playing media to modify the media playback operation.

10. The method according to any one of claims 1 to 2, wherein displaying the third optional user interface object during the display of the user interface includes displaying a single user interface object prior to displaying the third optional user interface object, the single user interface object comprising: The first portion of the first optional user interface object; as well as The first part of the second optional user interface object.

11. The method according to any one of claims 1 to 2, further comprising: After receiving the user input corresponding to the selection of the second optional user interface object, an optional user interface object for confirming the third optional user interface object is displayed.

12. A computer system, comprising: One or more processors, wherein the computer system communicates with a display generation component and one or more input devices; as well as The memory stores one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: The user interface is displayed via the display generation component, the user interface including: A first optional user interface object for controlling a first group of one or more media playback devices; A second optional user interface object, the second optional user interface object being used to control a second group of one or more media playback devices different from the first group of one or more media playback devices; and Instructions for selecting the first optional user interface object; When the user interface includes the instruction, user input corresponding to the selection of the second optional user interface object is received; and In response to receiving user input corresponding to the selection of the second optional user interface object, and based on determining that a set of criteria is met, wherein the set of criteria includes criteria that are met when the indication to select the first optional user interface object and the indication to select the second optional user interface object are simultaneously displayed for a predetermined time period: The user interface displays a third optional user interface object, which is used to control a third group of one or more media playback devices, including the first group of one or more media playback devices and the second group of one or more media playback devices; and Stop displaying the first optional user interface object and the second optional user interface object.

13. The computer system of claim 12, wherein the one or more programs further include instructions for performing the following operations: When displaying the third optional user interface object, receive first user input corresponding to a selection of the third optional user interface object; and In response to receiving the first user input corresponding to the selection of the third optional user interface object: Based on the user interface including an instruction to select the third optional user interface object, stop displaying the instruction to select the third optional user interface object in the user interface; and The instruction to select the third optional user interface object is displayed in the user interface, provided that the user interface does not include the instruction to select the third optional user interface object.

14. The computer system according to any one of claims 12 to 13, wherein: The first optional user interface object includes an identifier associated with the first group of one or more media playback devices; The second optional user interface object includes an identifier associated with the second group of one or more media playback devices; as well as Displaying the third optional user interface object includes displaying an identifier associated with the third group of one or more media playback devices, wherein the identifier associated with the third group of one or more media playback devices includes a portion representing the identifier associated with the first group of one or more media playback devices, and a portion representing the identifier associated with the second group of one or more media playback devices.

15. The computer system according to any one of claims 12 to 13, wherein the one or more programs further include instructions for performing the following operations: When displaying the third optional user interface object, receive second user input corresponding to the selection of the third optional user interface object; and In response to receiving the second user input corresponding to the selection of the third optional user interface object: Displaying a first optional user interface object for controlling the first group of one or more media playback devices and the indication for selecting the first optional user interface object; and Displays a second optional user interface object for controlling the second group of one or more media playback devices and the indication for selecting the second optional user interface object.

16. The computer system of claim 15, wherein the one or more programs further include instructions for performing the following operations: While displaying the first optional user interface object for controlling the first group of one or more media playback devices and the indication for selecting the first optional user interface object, and while displaying the second optional user interface object for controlling the second group of one or more media playback devices and the indication for selecting the second optional user interface object, one receives one or more user inputs corresponding to one or more selections of the first optional user interface object; and In response to receiving the one or more user inputs: Based on the determination of the last of the one or more selections of the first optional user interface object, the computer system stops displaying the instruction to select the first optional user interface object, and continues to display the first optional user interface object and the second optional user interface object, but does not display the third optional user interface object.

17. The computer system according to any one of claims 12 to 13, wherein: The user interface includes a fourth optional user interface object for controlling a fourth group of one or more media playback devices; and The one or more programs mentioned above also include instructions for the following operations: When displaying the third optional user interface object and the fourth optional user interface object, receive user input corresponding to the selection of the fourth optional user interface object; as well as In response to receiving the user input corresponding to the selection of the fourth optional user interface object: A fifth optional user interface object is displayed in the user interface for controlling the first group of one or more media playback devices, the second group of one or more media playback devices, and the fourth group of one or more media playback devices; as well as Stop displaying the third optional user interface object and the fourth optional user interface object.

18. The computer system according to any one of claims 12 to 13, wherein the one or more programs further include instructions for performing the following operations: In response to receiving user input corresponding to the selection of the second optional user interface object, grouping information indicating the grouping of the first optional user interface object and the second optional user interface object is transmitted to a second computer system different from the computer system.

19. The computer system according to any one of claims 12 to 13, wherein the one or more programs further include instructions for performing the following operations: After displaying the third optional user interface object in the user interface, data corresponding to the indication that the first group of one or more media playback devices are connected to the computer system is received; and In response to receiving the data, a notification is displayed instructing one or more media playback devices in the first group and one or more media playback devices in the second group to connect to the computer system.

20. The computer system according to any one of claims 12 to 13, wherein the one or more programs further include instructions for performing the following operations: When the user interface is displayed, user input is received at a first location on the user interface; and In response to receiving the user input at the first location on the user interface, a second user interface is displayed via the display generation component. The second user interface includes an optional user interface object that, when selected via the one or more input devices, causes one or more external media playback devices that are playing media to modify the media playback operation.

21. The computer system according to any one of claims 12 to 13, wherein displaying the third optional user interface object while displaying the user interface includes displaying a single user interface object prior to displaying the third optional user interface object, the single user interface object comprising: The first portion of the first optional user interface object; as well as The first part of the second optional user interface object.

22. The computer system according to any one of claims 12 to 13, wherein the one or more programs further include instructions for performing the following operations: After receiving the user input corresponding to the selection of the second optional user interface object, an optional user interface object for confirming the third optional user interface object is displayed.

23. A computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with a display generating component and one or more input devices, the one or more programs including instructions for performing the following operations: The user interface is displayed via the display generation component, the user interface including: A first optional user interface object for controlling a first group of one or more media playback devices; A second optional user interface object is used to control a second group of one or more media playback devices that are different from the first group of one or more media playback devices. as well as Instructions for selecting the first optional user interface object; When the user interface includes the instruction, user input corresponding to the selection of the second optional user interface object is received; as well as In response to receiving user input corresponding to the selection of the second optional user interface object, and based on determining that a set of criteria is met, wherein the set of criteria includes criteria that are met when the indication to select the first optional user interface object and the indication to select the second optional user interface object are simultaneously displayed for a predetermined time period: A third optional user interface object is displayed in the user interface, the third optional user interface object being used to control a third group of one or more media playback devices, including the first group of one or more media playback devices and the second group of one or more media playback devices; as well as Stop displaying the first optional user interface object and the second optional user interface object.

24. The computer-readable storage medium of claim 23, wherein the one or more programs further include instructions for performing the following operations: When displaying the third optional user interface object, receive first user input corresponding to a selection of the third optional user interface object; and In response to receiving the first user input corresponding to the selection of the third optional user interface object: Based on the user interface including an instruction to select the third optional user interface object, stop displaying the instruction to select the third optional user interface object in the user interface; and The instruction to select the third optional user interface object is displayed in the user interface, provided that the user interface does not include the instruction to select the third optional user interface object.

25. The computer-readable storage medium according to any one of claims 23 to 24, wherein: The first optional user interface object includes an identifier associated with the first group of one or more media playback devices; The second optional user interface object includes an identifier associated with the second group of one or more media playback devices; as well as Displaying the third optional user interface object includes displaying an identifier associated with the third group of one or more media playback devices, wherein the identifier associated with the third group of one or more media playback devices includes a portion representing the identifier associated with the first group of one or more media playback devices, and a portion representing the identifier associated with the second group of one or more media playback devices.

26. The computer-readable storage medium according to any one of claims 23 to 24, wherein the one or more programs further include instructions for performing the following operations: When displaying the third optional user interface object, receive second user input corresponding to the selection of the third optional user interface object; and In response to receiving the second user input corresponding to the selection of the third optional user interface object: Displaying a first optional user interface object for controlling the first group of one or more media playback devices and the indication for selecting the first optional user interface object; and Displays a second optional user interface object for controlling the second group of one or more media playback devices and the indication for selecting the second optional user interface object.

27. The computer-readable storage medium of claim 26, wherein the one or more programs further include instructions for performing the following operations: While displaying the first optional user interface object for controlling the first group of one or more media playback devices and the indication for selecting the first optional user interface object, and while displaying the second optional user interface object for controlling the second group of one or more media playback devices and the indication for selecting the second optional user interface object, one receives one or more user inputs corresponding to one or more selections of the first optional user interface object; and In response to receiving the one or more user inputs: Based on the determination of the last of the one or more selections of the first optional user interface object, the computer system stops displaying the instruction to select the first optional user interface object, and continues to display the first optional user interface object and the second optional user interface object, but does not display the third optional user interface object.

28. The computer-readable storage medium according to any one of claims 23 to 24, wherein: The user interface includes a fourth optional user interface object for controlling a fourth group of one or more media playback devices; and The one or more programs mentioned above also include instructions for the following operations: When displaying the third optional user interface object and the fourth optional user interface object, receive user input corresponding to the selection of the fourth optional user interface object; as well as In response to receiving the user input corresponding to the selection of the fourth optional user interface object: A fifth optional user interface object is displayed in the user interface for controlling the first group of one or more media playback devices, the second group of one or more media playback devices, and the fourth group of one or more media playback devices; as well as Stop displaying the third optional user interface object and the fourth optional user interface object.

29. The computer-readable storage medium according to any one of claims 23 to 24, wherein the one or more programs further include instructions for: In response to receiving user input corresponding to the selection of the second optional user interface object, grouping information indicating the grouping of the first optional user interface object and the second optional user interface object is transmitted to a second computer system different from the computer system.

30. The computer-readable storage medium according to any one of claims 23 to 24, wherein the one or more programs further include instructions for performing the following operations: After displaying the third optional user interface object in the user interface, data corresponding to the indication that the first group of one or more media playback devices are connected to the computer system is received; and In response to receiving the data, a notification is displayed instructing one or more media playback devices in the first group and one or more media playback devices in the second group to connect to the computer system.

31. The computer-readable storage medium according to any one of claims 23 to 24, wherein the one or more programs further include instructions for performing the following operations: When the user interface is displayed, user input is received at a first location on the user interface; and In response to receiving the user input at the first location on the user interface, a second user interface is displayed via the display generation component. The second user interface includes an optional user interface object that, when selected via the one or more input devices, causes one or more external media playback devices that are playing media to modify the media playback operation.

32. The computer-readable storage medium according to any one of claims 23 to 24, wherein displaying the third optional user interface object when displaying the user interface includes displaying a single user interface object prior to displaying the third optional user interface object, the single user interface object comprising: The first portion of the first optional user interface object; as well as The first part of the second optional user interface object.

33. The computer-readable storage medium according to any one of claims 23 to 24, wherein the one or more programs further include instructions for performing the following operations: After receiving the user input corresponding to the selection of the second optional user interface object, an optional user interface object for confirming the third optional user interface object is displayed.

34. A computer system communicating with a display generation component and one or more input devices, the computer system comprising: Apparatus for performing the method according to any one of claims 1 to 11.

35. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with a display generation component and one or more input devices, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 11.

Images