Content output device and user interface

Abstract

The present disclosure generally relates to outputting light and managing controllable devices. In some examples, an electronic device configured to provide an output associated with content displayed on a display generating component outputs light based on received information about the content displayed on the display generating component. In some examples, an electronic device determines whether to associate a remotely controllable external device with a context based on a set of one or more criteria and in response to receiving a request to associate the remotely controllable external device with the context.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Patent Application No. 18 / 230,107, entitled “CONTENT OUTPUT DEVICES AND USER INTERFACES,” filed on August 3, 2023, and U.S. Provisional Patent Application No. 63 / 403,495, entitled “CONTENT OUTPUT DEVICES AND USER INTERFACES,” filed on September 2, 2022. The contents of each of these patent applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates generally to computer user interfaces, and more particularly to techniques for outputting light and managing controllable devices. Background Art

[0004] The electronic device is configured to display visual content on the display, such as images and / or videos corresponding to media files. The light source is configured to generate light with different colors, brightness and / or other properties. In addition, some electronic devices can be used to group accessory devices so that the accessory devices can output content in conjunction with each other. Summary of the invention

[0005] However, some techniques for using electronic devices to output light and manage controllable devices are often cumbersome and inefficient. For example, some prior art techniques use complex and time-consuming user interfaces that may include multiple key presses or keystrokes. Prior art techniques require more time than necessary, which results in wasted user time and device energy. This latter consideration is particularly important in battery-powered devices.

[0006] Therefore, the present technology provides faster and more efficient methods and interfaces for outputting light and managing controllable devices for electronic devices. Such methods and interfaces optionally supplement or replace other methods for outputting light and managing controllable devices. Such methods and interfaces reduce the cognitive burden on users and produce more effective human-computer interfaces. Such methods and interfaces also reduce the number of unnecessary, irrelevant and / or repeated user inputs. For battery-powered computing devices, such methods and interfaces save power and increase the time interval between battery charges.

[0007] According to some embodiments, a method is described. The method is performed at a computer system in communication with one or more light sources. The method includes: when the computer system is configured to provide an output associated with content displayed on a display generation component, receiving information associated with the content displayed on the display generation component; and in response to receiving the information associated with the content displayed on the display generation component, outputting light via the one or more light sources according to the received information associated with the content displayed on the display generation component.

[0008] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with one or more light sources, the one or more programs including instructions for the following operations: when the computer system is configured to provide an output associated with content displayed on a display generation component, receiving information associated with the content displayed on the display generation component; and in response to receiving the information associated with the content displayed on the display generation component, outputting light via the one or more light sources according to the received information associated with the content displayed on the display generation component.

[0009] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system in communication with one or more light sources, the one or more programs including instructions for the following operations: when the computer system is configured to provide an output associated with content displayed on a display generation component, receiving information associated with the content displayed on the display generation component; and in response to receiving the information associated with the content displayed on the display generation component, outputting light via the one or more light sources according to the received information associated with the content displayed on the display generation component.

[0010] According to some embodiments, a computer system is described. The computer system communicates with one or more light sources. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: when the computer system is configured to provide an output associated with content displayed on a display generation component, receiving information associated with the content displayed on the display generation component; and in response to receiving the information associated with the content displayed on the display generation component, outputting light via the one or more light sources according to the received information associated with the content displayed on the display generation component.

[0011] According to some embodiments, a computer system is described. The computer system communicates with one or more light sources. The computer system includes: a component for receiving information associated with content displayed on the display generation component when the computer system is configured to provide an output associated with the content displayed on the display generation component; and a component for outputting light via the one or more light sources according to the received information associated with the content displayed on the display generation component in response to receiving the information associated with the content displayed on the display generation component.

[0012] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system in communication with one or more light sources, the one or more programs including instructions for: when the computer system is configured to provide an output associated with content displayed on a display generating component, receiving information associated with the content displayed on the display generating component; and in response to receiving the information associated with the content displayed on the display generating component, outputting light via the one or more light sources according to the received information associated with the content displayed on the display generating component.

[0013] According to some embodiments, a method is described. The method is performed at a computer system that communicates with one or more input devices and a display generation component. The method includes: when a user interface including a user interface object is displayed via the display generation component, a request to associate a second remotely controllable external device with a context is detected via the one or more input devices, and the user interface object provides an option for controlling a first remotely controllable external device when selected, wherein the first remotely controllable external device is associated with the context; and in response to detecting the request to associate the second remotely controllable external device with the context: according to determining that the second remotely controllable external device satisfies a set of one or more criteria, the second remotely controllable external device is associated with the context; and according to determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function that does not correspond to the second function of the first remotely controllable external device, and the second remotely controllable external device is abandoned from being associated with the context.

[0014] According to some embodiments, a non-transitory computer-readable storage medium is described. The non-transitory computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices and a display generation component, the one or more programs including instructions for the following operations: when a user interface including a user interface object is displayed via the display generation component, a request to associate a second remotely controllable external device with a context is detected via the one or more input devices, the user interface object provides an option for controlling a first remotely controllable external device when selected, wherein the first remotely controllable external device is associated with the context; and in response to detecting the request to associate the second remotely controllable external device with the context: according to determining that the second remotely controllable external device satisfies a set of one or more criteria, the second remotely controllable external device is associated with the context; and according to determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function that does not correspond to the second function of the first remotely controllable external device, the second remotely controllable external device is abandoned from being associated with the context.

[0015] According to some embodiments, a transient computer-readable storage medium is described. The transient computer-readable storage medium stores one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices and a display generation component, the one or more programs including instructions for the following operations: when a user interface including a user interface object is displayed via the display generation component, a request to associate a second remotely controllable external device with a context is detected via the one or more input devices, the user interface object providing an option for controlling a first remotely controllable external device when selected, wherein the first remotely controllable external device is associated with the context; and in response to detecting the request to associate the second remotely controllable external device with the context: according to determining that the second remotely controllable external device satisfies a set of one or more criteria, the second remotely controllable external device is associated with the context; and according to determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function that does not correspond to the second function of the first remotely controllable external device, the second remotely controllable external device is abandoned from being associated with the context.

[0016] According to some embodiments, a computer system is described. The computer system communicates with one or more input devices and a display generation component. The computer system includes: one or more processors; and a memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for the following operations: when a user interface including a user interface object is displayed via the display generation component, a request to associate a second remotely controllable external device with a context is detected via the one or more input devices, the user interface object provides an option for controlling a first remotely controllable external device when selected, wherein the first remotely controllable external device is associated with the context; and in response to detecting the request to associate the second remotely controllable external device with the context: according to determining that the second remotely controllable external device satisfies a set of one or more criteria, the second remotely controllable external device is associated with the context; and according to determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function that does not correspond to the second function of the first remotely controllable external device, and the second remotely controllable external device is abandoned from being associated with the context.

[0017] According to some embodiments, a computer system is described. The computer system communicates with one or more input devices and a display generation component. The computer system includes: a component for detecting a request to associate a second remotely controllable external device with a context via the one or more input devices when a user interface including a user interface object is displayed via the display generation component, the user interface object providing an option for controlling a first remotely controllable external device when selected, wherein the first remotely controllable external device is associated with the context; and a component for performing the following operations in response to detecting the request to associate the second remotely controllable external device with the context: according to determining that the second remotely controllable external device satisfies a set of one or more criteria, the second remotely controllable external device is associated with the context; and according to determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function that does not correspond to the second function of the first remotely controllable external device, and the second remotely controllable external device is abandoned from being associated with the context.

[0018] According to some embodiments, a computer program product is described. The computer program product includes one or more programs configured to be executed by one or more processors of a computer system communicating with one or more input devices and a display generation component, and the one or more programs include instructions for the following operations: when a user interface including a user interface object is displayed via the display generation component, a request to associate a second remotely controllable external device with a context is detected via the one or more input devices, and the user interface object provides an option for controlling a first remotely controllable external device when selected, wherein the first remotely controllable external device is associated with the context; and in response to detecting the request to associate the second remotely controllable external device with the context: according to determining that the second remotely controllable external device satisfies a set of one or more criteria, the second remotely controllable external device is associated with the context; and according to determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function that does not correspond to the second function of the first remotely controllable external device, the second remotely controllable external device is abandoned from being associated with the context.

[0019] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors. Executable instructions for performing these functions are optionally included in a transient computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0020] Thus, a faster, more efficient method and interface for outputting light and managing controllable devices is provided for devices, thereby improving the effectiveness, efficiency, and user satisfaction of such devices.Such methods and interfaces can supplement or replace other methods for outputting light and managing controllable devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0022] Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display according to some embodiments.

[0023] Figure 1B is a block diagram illustrating exemplary components for event processing according to some embodiments.

[0024] Figure 2 A portable multifunction device with a touch screen according to some embodiments is illustrated.

[0025] Figure 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.

[0026] Figure 4A An exemplary user interface for a menu of applications on a portable multifunction device according to some embodiments is illustrated.

[0027] Figure 4B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is illustrated according to some embodiments.

[0028] Figure 5A Personal electronic devices according to some embodiments are illustrated.

[0029] Figure 5B is a block diagram illustrating a personal electronic device according to some embodiments.

[0030] Figure 5C An electronic device according to some embodiments is illustrated.

[0031] Figure 5D is a block diagram illustrating an electronic device according to some embodiments.

[0032] FIG. 6A to FIG. 6O Example techniques for outputting light according to some embodiments are illustrated.

[0033] 7A to 7C is a flow chart of a method for outputting light according to some embodiments.

[0034] Figures 8A to 8V An exemplary user interface for managing controllable devices according to some embodiments is illustrated.

[0035] 9A to 9D is a flowchart of a method for managing a controllable device according to some embodiments. DETAILED DESCRIPTION

[0036] The following description sets forth exemplary methods, parameters, etc. However, it should be appreciated that such description is not intended to limit the scope of the present disclosure, but is provided as a description of exemplary embodiments.

[0037] There is a need for electronic devices that provide efficient methods and interfaces for outputting light and managing controllable devices. For example, there is a need for electronic devices that can provide output (such as audio output) and also cause light to be output based on content displayed on a display device. As another example, there is a need for electronic devices that can easily associate a first accessory device with a second accessory device that is currently outputting content so that the first accessory device and the second accessory device can operate in conjunction with each other. Such techniques can reduce the cognitive burden on a user of causing light to be output and / or managing controllable devices, thereby improving productivity. In addition, such techniques can reduce processor power and battery power that would otherwise be wasted on redundant user input.

[0038] In the following, Figure 1A to Figure 1B , Figure 2 , Figure 3 , FIG. 4A to FIG. 4B and FIG. 5A to FIG. 5D A description of example devices for performing techniques for outputting light and managing controllable devices is provided. FIG. 6A to FIG. 6O An exemplary user interface for outputting light is illustrated. 7A to 7C is a flow chart illustrating a method of outputting light according to some embodiments. FIG. 6A to FIG. 6O The user interface in is used to illustrate the process described below, including 7A to 7C process. Figures 8A to 8V An exemplary user interface for managing controllable devices is illustrated. 9A to 9D is a flow chart illustrating a method of managing a controllable device according to some embodiments. Figures 8A to 8V The user interface in is used to illustrate the process described below, including 9A to 9D process.

[0039] The processes described below enhance the operability of the device and make the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device) through various techniques, including by providing improved visual feedback to the user, reducing the number of inputs required to perform an operation, providing additional control options without cluttering the user interface with additional displayed controls, performing an operation without further user input when a set of conditions have been met, and / or additional techniques. These techniques also reduce power usage and extend the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0040] In addition, in the method described herein where one or more steps depend on one or more conditions being met, it should be understood that the method can be repeated in multiple repetitions so that in the process of repetition, all conditions for determining the steps in the method have been met in different repetitions of the method. For example, if the method needs to perform the first step (if the condition is met), and perform the second step (if the condition is not met), then the ordinary technician will know that the steps stated are repeated until both the condition is met and the condition is not met (in no particular order). Therefore, the method described as having one or more steps depending on one or more conditions being met can be rewritten as a method of repeating until each condition described in the method is met. However, this does not require the system or computer-readable medium to declare that the system or computer-readable medium contains instructions for performing a contingent operation based on the satisfaction of the corresponding one or more conditions, and is therefore able to determine whether the possible situation has been met without explicitly repeating the steps of the method until all conditions for determining the steps in the method have been met. It will also be understood by ordinary technicians in the art that, similar to the method with the contingent step, the system or computer-readable storage medium can repeat the steps of the method as many times as needed to ensure that all the contingent steps have been performed.

[0041] Although the following description uses the terms "first", "second", etc. to describe various elements, these elements should not be limited by the terms. In some embodiments, these terms are used to distinguish one element from another element. For example, a first touch can be named a second touch and similarly a second touch can be named a first touch without departing from the scope of the various described embodiments. In some embodiments, the first touch and the second touch are two separate references to the same touch. In some embodiments, both the first touch and the second touch are touches, but they are not the same touch.

[0042] The terms used in the description of various described embodiments herein are only for the purpose of describing specific embodiments, and are not intended to be limiting. As used in the description of various described embodiments and in the appended claims, the singular forms "one" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more items in the associated listed items. It will also be understood that the terms "include" and / or "comprising" are used in this specification to specify the presence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their grouping.

[0043] The term "if" is optionally interpreted to mean "when," "upon," or "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that ..." or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that ..." or "in response to determining that ..." or "upon detecting [a stated condition or event]," or "in response to detecting [a stated condition or event]," depending on the context.

[0044] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple Watch from Apple Inc. (Cupertino, California). Devices, iPod Equipment and Device. Optionally use other portable electronic devices, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads). In some embodiments, the electronic device is a computer system that communicates with the display generation component (e.g., via wireless communication, via wired communication). The display generation component is configured to provide visual output, such as display via a CRT display, display via an LED display, or display via image projection. In some embodiments, the display generation component is integrated with the computer system. In some embodiments, the display generation component is separated from the computer system. As used herein, "display" content includes sending data (e.g., image data or video data) to an integrated or external display generation component via a wired or wireless connection to visually generate content to display content (e.g., video data rendered or decoded by display controller 156).

[0045] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse and / or joystick.

[0046] The device typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0047] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports various applications with a user interface that is intuitive and clear to the user.

[0048] Attention is now turned to embodiments of portable devices having touch-sensitive displays. Figure 1A 1 is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display 112 is sometimes referred to as a "touch screen" for convenience, and is sometimes referred to as or referred to as a "touch-sensitive display system." The device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more contact force sensors 165 for detecting the intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0049] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or to a surrogate (surrogate) of the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted average) to determine an estimated contact force. Similarly, the pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted into an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of the contact as an attribute of the user input allows the user to access additional device functions that would otherwise be inaccessible to the user on a smaller device with limited real estate, which is used to display an enable indication (e.g., on a touch-sensitive display) and / or receive user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls, such as knobs or buttons).

[0050] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of the device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical property of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or a touchpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, movement of a touch-sensitive surface may optionally be interpreted or sensed by a user as "roughness" of the touch-sensitive surface even when there is no change in the smoothness of the touch-sensitive surface. While such a user's interpretation of touch will be limited by the user's individualized sensory perceptions, many sensory perceptions of touch are common to most users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., "press click," "release click," "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or a component thereof that would generate that sensory perception for a typical (or average) user.

[0051] It should be understood that device 100 is merely one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0052] The memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state memory devices. The memory controller 122 optionally controls the access rights of other components of the device 100 to the memory 102.

[0053] The peripheral device interface 118 can be used to couple the input peripheral devices and output peripheral devices of the device to the CPU 120 and the memory 102. The one or more processors 120 run or execute various software programs (such as computer programs (e.g., including instructions)) and / or instruction sets stored in the memory 102 to perform various functions of the device 100 and process data. In some embodiments, the peripheral device interface 118, the CPU 120, and the memory controller 122 are optionally implemented on a single chip such as the chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0054] RF (radio frequency) circuit 108 receives and transmits RF signals, also referred to as electromagnetic signals. RF circuit 108 converts electrical signals into / converts electromagnetic signals into electrical signals, and communicates with communication networks and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to antenna systems, RF transceivers, one or more amplifiers, tuners, one or more oscillators, digital signal processors, codec chipsets, user identity modules (SIM) cards, memories, and the like. RF circuit 108 optionally communicates with networks and other devices via wireless communications, such as the Internet (also referred to as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs)). RF circuit 108 optionally includes well-known circuits for detecting near field communications (NFC) fields, such as through short-range communications radio components. Wireless communication optionally uses any of a variety of communication standards, protocols and techniques, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE802.11n and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilizing Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)) and / or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols that have not been developed as of the filing date of this document.

[0055] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and sends the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signals into audio data and sends the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or sent to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., Figure 2 The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset having both output (e.g., a single or dual-ear headset) and input (e.g., a microphone).

[0056] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch screen 112 and other input control devices 116, to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, a depth camera controller 169, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / transmit electrical signals from / to other input control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some embodiments, the input controller 160 is optionally coupled to any one of the following (or none of the following): a keyboard, an infrared port, a USB port, and a pointing device such as a mouse. One or more buttons (e.g., Figure 2 208) optionally includes an increase / decrease button for volume control of the speaker 111 and / or the microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2206 in). In some embodiments, the electronic device is a computer system that communicates with one or more input devices (e.g., via wireless communication, via wired communication). In some embodiments, the one or more input devices include a touch-sensitive surface (e.g., a touchpad as part of a touch-sensitive display). In some embodiments, the one or more input devices include one or more camera sensors (e.g., one or more optical sensors 164 and / or one or more depth camera sensors 175), such as for tracking a user's gestures (e.g., hand gestures and / or air gestures) as input. In some embodiments, the one or more input devices are integrated with the computer system. In some embodiments, the one or more input devices are separate from the computer system. In some embodiments, an air gesture is a gesture that is detected without the user touching an input element that is part of the device (or is independent of an input element that is part of the device) and is based on detected movement of a part of the user's body through the air, including movement of the user's body relative to an absolute reference (for example, the angle of the user's arm relative to the ground or the distance of the user's hand relative to the ground), movement relative to another part of the user's body (for example, movement of the user's hand relative to the user's shoulder, movement of one of the user's hands relative to the user's other hand, and / or movement of the user's fingers relative to another finger or part of the user's hand), and / or absolute movement of a part of the user's body (for example, a tap gesture involving a hand moving a predetermined amount and / or speed in a predetermined position, or a shake gesture involving a predetermined speed or amount of rotation of a part of the user's body).

[0057] A quick press of the push button optionally disengages the lock of the touch screen 112 or optionally begins the process of unlocking the device using gestures on the touch screen, as described in U.S. Patent Application 11 / 322,549, filed on December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image" (i.e., U.S. Patent No. 7,657,849), which is hereby incorporated by reference in its entirety. A long press of the push button (e.g., 206) optionally turns the device 100 on or off. The functions of one or more buttons are optionally user-customizable. The touch screen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0058] The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch screen 112 and / or transmits electrical signals to the touch screen. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0059] The touch screen 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from a user based on tactile and / or haptic contact. The touch screen 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of the contact) on the touch screen 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the user's finger.

[0060] The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch screen 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112 to detect contact and any movement or interruption thereof. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as in the Apple ® from Apple Inc. (Cupertino, California). and iPod The technology used in.

[0061] The touch-sensitive display in some embodiments of touch screen 112 is optionally similar to the multi-touch-sensitive touchpad described in the following U.S. Patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and / or 6,677,932 (Westerman et al.), and / or U.S. Patent Publication 2002 / 0015024A1, each of which is hereby incorporated by reference in its entirety. However, touch screen 112 displays visual output from device 100, while a touch-sensitive touchpad does not provide visual output.

[0062] The touch-sensitive display in some embodiments of the touch screen 112 is described in the following applications: (1) U.S. patent application No. 11 / 381,313, filed on May 2, 2006, "Multipoint Touch Surface Controller"; (2) U.S. patent application No. 10 / 840,862, filed on May 6, 2004, "Multipoint Touchscreen"; (3) U.S. patent application No. 10 / 903,964, filed on July 30, 2004, "Gestures For Touch Sensitive Input Devices"; (4) U.S. patent application No. 11 / 048,264, filed on January 31, 2005, "Gestures For Touch Sensitive Input Devices"; (5) U.S. patent application No. 11 / 038,590, filed on January 18, 2005, "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices”; (6) U.S. patent application Ser. No. 11 / 228,758, filed on September 16, 2005, “Virtual Input Device Placement On A Touch Screen User Interface”; (7) U.S. patent application Ser. No. 11 / 228,700, filed on September 16, 2005, “Operation Of A Computer With A Touch Screen Interface”; (8) U.S. patent application Ser. No. 11 / 228,737, filed on September 16, 2005, “Activating Virtual Keys Of A Touch-Screen Virtual Keyboard”; and (9) U.S. patent application Ser. No. 11 / 367,749, filed on March 3, 2006, “Multi-Functional Hand-Held Device”. All of these applications are incorporated herein by reference in their entirety.

[0063] The touch screen 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen has a video resolution of about 160 dpi. The user optionally uses any suitable object or attachment such as a stylus, a finger, etc. to contact the touch screen 112. In some embodiments, the user interface is designed to work primarily through finger-based contacts and gestures, which may not be as accurate as stylus-based input due to the larger contact area of ​​the finger on the touch screen. In some embodiments, the device converts rough finger-based input into precise pointer / cursor positioning or commands for performing the actions desired by the user.

[0064] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch screen 112, or an extension of the touch-sensitive surface formed by the touch screen.

[0065] The device 100 also includes a power system 162 for powering the various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0066] Device 100 optionally also includes one or more optical sensors 164 . Figure 1AAn optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. The optical sensor 164 optionally includes a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with the imaging module 143 (also called a camera module), the optical sensor 164 optionally captures a static image or video. In some embodiments, the optical sensor is located on the rear of the device 100, opposite to the touch screen display 112 on the front of the device, so that the touch screen display can be used as a viewfinder for static images and / or video image acquisition. In some embodiments, the optical sensor is located on the front of the device so that when the user views other video conference participants on the touch screen display, the image of the user is optionally obtained for video conferencing. In some embodiments, the positioning of the optical sensor 164 can be changed by the user (e.g., by rotating the lens and sensor in the device housing) so that a single optical sensor 164 is used with the touch screen display for both video conferencing and static image and / or video image acquisition.

[0067] Device 100 optionally also includes one or more depth camera sensors 175 . Figure 1A A depth camera sensor coupled to a depth camera controller 169 in the I / O subsystem 106 is shown. The depth camera sensor 175 receives data from the environment to create a three-dimensional model of an object (e.g., a face) within the scene from a viewpoint (e.g., a depth camera sensor). In some embodiments, in conjunction with the imaging module 143 (also referred to as a camera module), the depth camera sensor 175 is optionally used to determine a depth map of different portions of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is located at the front of the device 100, so that an image of the user with depth information is optionally obtained for video conferencing while the user is viewing other video conference participants on the touch screen display, and a selfie with depth map data is captured. In some embodiments, the depth camera sensor 175 is located at the rear of the device, or at both the rear and front of the device 100. In some embodiments, the positioning of the depth camera sensor 175 can be changed by the user (e.g., by rotating the lens and sensor in the device housing) so that the depth camera sensor 175 is used with the touch screen display for both video conferencing and still image and / or video image acquisition.

[0068] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1AA contact force sensor is shown coupled to a force sensor controller 159 in the I / O subsystem 106. Contact force sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). Contact force sensor 165 receives contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the back of device 100, opposite to touch screen display 112 located on the front of device 100.

[0069] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is optionally coupled to the input controller 160 in the I / O subsystem 106. The proximity sensor 166 is optionally implemented as described in the following U.S. patent application numbers: 11 / 241,839, entitled "Proximity Detector In Handheld Device"; 11 / 240,788, entitled "Proximity Detector In Handheld Device"; 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", which are hereby incorporated by reference in their entirety. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and disables the touch screen 112.

[0070] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1AA tactile output generator is shown coupled to a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator 167 optionally includes one or more electroacoustic devices such as a speaker or other audio component; and / or an electromechanical device for converting energy into linear motion such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component for converting an electrical signal into a tactile output on the device). The contact force sensor 165 receives tactile feedback generation instructions from the tactile feedback module 133 and generates a tactile output on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one tactile output generator is arranged in parallel or adjacent to a touch-sensitive surface (e.g., a touch-sensitive display system 112) and optionally generates a tactile output by moving the touch-sensitive surface vertically (e.g., inward / outward of the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on the back of the device 100, opposite the touch screen display 112 located on the front of the device 100.

[0071] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. The accelerometer 168 is optionally implemented as described in the following U.S. Patent Publication Nos.: 20050190059, entitled "Acceleration-based Theft Detection System for Portable Electronic Devices" and 20060017692, entitled "Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer", both of which are incorporated herein by reference in their entirety. In some embodiments, information is displayed in a portrait view or a landscape view on the touch screen display based on analysis of data received from one or more accelerometers. The device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver in addition to the accelerometer 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0072] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 ( Figure 1A ) or 370( Figure 3 ) storage device / global internal state 157, such as Figure 1A and Figure 3 . The device / global internal state 157 includes one or more of the following: active application state, which indicates which application (if any) is currently active; display state, which indicates what applications, views, or other information occupy various areas of the touch screen display 112; sensor state, including information obtained from the device's various sensors and input control devices 116; and position information related to the device's position and / or posture.

[0073] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitating communication between various hardware components and software components.

[0074] The communication module 128 facilitates communication with other devices through one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices, or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is connected to (trademark of Apple Inc.) devices.

[0075] The contact / motion module 130 optionally detects contact with the touch screen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger press event), determining the contact strength (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of the contact point being represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multiple finger contacts). In some embodiments, the contact / motion module 130 and display controller 156 detect contact on the touch pad.

[0076] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of the device 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen can be set to any one of a large range of predefined thresholds. Additionally, in some specific implementations, a software setting is provided to the user of the device for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0077] The contact / motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event, and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event.

[0078] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch screen 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including but not limited to text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0079] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application or the like, along with coordinate data and other graphics attribute data if necessary, and then generates screen image data to be output to the display controller 156.

[0080] Haptic feedback module 133 includes various software components for generating instructions used by tactile output generator 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .

[0081] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (eg, contacts 137, -email 140, IM 141, browser 147, and any other application requiring text input).

[0082] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for use in location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services, such as a weather widget, a local yellow pages widget, and a map / navigation widget).

[0083] Application 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0084] Contacts module 137 (sometimes called address book or contact list);

[0085] Telephone module 138;

[0086] Video conferencing module 139;

[0087] Email client module 140;

[0088] Instant messaging (IM) module 141;

[0089] Fitness support module 142;

[0090] A camera module 143 for still images and / or video images;

[0091] Image management module 144;

[0092] Video player module;

[0093] Music player module;

[0094] Browser module 147;

[0095] Calendar module 148;

[0096] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock market widget 149-2, a calculator widget 149-3, an alarm widget 149-4, a dictionary widget 149-5, and other widgets acquired by the user, and a user-created widget 149-6;

[0097] A widget creator module 150 for forming a user-created widget 149 - 6 ;

[0098] Search module 151;

[0099] Video and music player module 152, which merges the video player module and the music player module;

[0100] Notepad module 153;

[0101] Map module 154; and / or

[0102] Online video module 155.

[0103] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

[0104] In combination with the touch screen 112, display controller 156, touch / motion module 130, graphics module 132 and text input module 134, the contact module 137 is optionally used to manage an address book or contact list (for example, stored in the application internal state 192 of the contact module 137 in memory 102 or memory 370), including: adding one or more names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses or other information with names; associating images with names; categorizing and classifying names; providing phone numbers or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing module 139, email 140 or IM 141; and the like.

[0105] In conjunction with RF circuit 108, audio circuit 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, phone module 138 is optionally used to enter a character sequence corresponding to a phone number, access one or more phone numbers in contact module 137, modify an entered phone number, dial a corresponding phone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0106] In combination with the RF circuit 108, the audio circuit 110, the speaker 111, the microphone 113, the touch screen 112, the display controller 156, the optical sensor 164, the optical sensor controller 158, the contact / motion module 130, the graphics module 132, the text input module 134, the contact module 137 and the telephone module 138, the video conferencing module 139 includes executable instructions for initiating, conducting and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0107] In conjunction with RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, transmitting, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and transmit emails with still images or video images captured by camera module 143.

[0108] In conjunction with the RF circuit 108, the touch screen 112, the display controller 156, the contact / motion module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for the following operations: entering a character sequence corresponding to an instant message, modifying previously entered characters, sending a corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for telephone-based instant messaging or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the instant messages sent and / or received optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both telephone-based messages (e.g., messages transmitted using SMS or MMS) and Internet-based messages (e.g., messages transmitted using XMPP, SIMPLE, or IMPS).

[0109] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the contact / motion module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154 and the music player module, the fitness support module 142 includes executable instructions for creating a fitness (e.g., with time, distance and / or calorie burn goals); communicating with fitness sensors (sports equipment); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for fitness; and displaying, storing and sending fitness data.

[0110] In conjunction with the touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132 and image management module 144, the camera module 143 includes executable instructions for the following operations: capturing still images or videos (including video streams) and storing them in the memory 102, modifying the characteristics of the still images or videos, or deleting the still images or videos from the memory 102.

[0111] In conjunction with touch screen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing) or otherwise manipulating, marking, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0112] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the touch / motion module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet in accordance with user instructions, including searching, linking to, receiving and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0113] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the touch / motion module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0114] In conjunction with RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is a mini-application that is optionally downloaded and used by a user (e.g., weather widget 149-1, stock market widget 149-2, calculator widget 149-3, alarm widget 149-4, and dictionary widget 149-5) or a mini-application created by a user (e.g., user-created widget 149-6). In some embodiments, the widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! widget).

[0115] In combination with the RF circuit 108, touch screen 112, display controller 156, touch / motion module 130, graphics module 132, text input module 134 and browser module 147, the widget creator module 150 is optionally used by a user to create widgets (e.g., converting a user-specified portion of a web page into a widget).

[0116] In combination with the touch screen 112, display controller 156, contact / motion module 130, graphics module 132 and text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0117] In conjunction with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, and executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch screen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (trademark of Apple Inc.).

[0118] In conjunction with the touch screen 112, display controller 156, contact / motion module 130, graphics module 132 and text input module 134, the notepad module 153 includes executable instructions for creating and managing notes, to-do lists, etc. according to user instructions.

[0119] In combination with the RF circuit 108, the touch screen 112, the display controller 156, the touch / motion module 130, the graphics module 132, the text input module 134, the GPS module 135 and the browser module 147, the map module 154 is optionally used to receive, display, modify and store maps and data associated with the maps (e.g., driving directions, data relating to stores and other points of interest at or near a particular location, and other location-based data) in accordance with user instructions.

[0120] In conjunction with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes instructions for the following operations: allowing a user to access, browse, receive (e.g., by streaming and / or downloading), playback (e.g., on the touch screen or on an external display connected via external port 124), send an email with a link to a specific online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a specific online video. Additional description of online video applications can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed on June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” and U.S. Patent Application No. 11 / 968,067, filed on December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos,” the contents of which are hereby incorporated by reference in their entirety.

[0121] Each of the modules and applications described above corresponds to an executable instruction set for performing one or more of the functions described above and the methods described in this patent application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) do not have to be implemented as independent software programs (such as computer programs (e.g., including instructions)), processes, or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. For example, a video player module is optionally combined with a music player module into a single module (e.g., Figure 1A In some embodiments, the memory 102 optionally stores a subset of the above modules and data structures. In addition, the memory 102 optionally stores additional modules and data structures not described above.

[0122] In some embodiments, the device 100 is a device where the operation of a predefined set of functions on the device is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operating the device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on the device 100 is optionally reduced.

[0123] A predefined set of functions that are performed exclusively through the touch screen and / or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 to a main menu, a home menu, or a root menu from any user interface displayed on the device 100. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0124] Figure 1B is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370( Figure 3 ) includes an event classifier 170 (e.g., in the operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 137 to 151, 155, 380 to 390).

[0125] Event classifier 170 receives event information and determines the application 136-1 and application view 191 of application 136-1 to which the event information is to be delivered. Event classifier 170 includes event monitor 171 and event distributor module 174. In some embodiments, application 136-1 includes application internal state 192, which indicates one or more current application views displayed on touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by event classifier 170 to determine which application(s) is currently active, and application internal state 192 is used by event classifier 170 to determine application view 191 to which the event information is to be delivered.

[0126] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when application 136-1 resumes execution, user interface state information indicating that information is being displayed or is ready to be displayed by application 136-1, a state queue for enabling a user to return to a previous state or view of application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0127] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about sub-events (e.g., user touches on touch-sensitive display 112 as part of a multi-touch gesture). Peripherals interface 118 sends information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (through audio circuit 110). The information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display 112 or a touch-sensitive surface.

[0128] In some embodiments, event monitor 171 transmits a request to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 sends event information. In other embodiments, peripheral device interface 118 sends event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0129] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0130] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0131] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are identified as correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0132] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should handle the sub-events. In most cases, the hit view is the lowest level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by hit view determination module 172, the hit view typically receives all sub-events related to the same touch or input source for which it is identified as the hit view.

[0133] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, higher views in the hierarchy will still remain as actively participating views.

[0134] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizers 180). In embodiments including active event recognizer determination module 173, event distributor module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores the event information in an event queue, which is retrieved by corresponding event receivers 182.

[0135] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or is part of another module stored in memory 102, such as contact / motion module 130.

[0136] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the user interface of the application. Each application view 191 of application 136-1 includes one or more event identifiers 180. Typically, the corresponding application view 191 includes multiple event identifiers 180. In other embodiments, one or more event identifiers in event identifiers 180 are part of an independent module, which is a higher-level object such as a user interface toolkit or application 136-1 from which methods and other properties are inherited. In some embodiments, the corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178 and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177 or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more corresponding event handlers 190. In addition, in some embodiments, one or more of the data updater 176, the object updater 177, and the GUI updater 178 are included in the corresponding application view 191.

[0137] The corresponding event identifier 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies the event based on the event information. The event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event identifier 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally includes sub-event delivery instructions).

[0138] Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (for example, from a longitudinal orientation to a transverse orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0139] Event comparator 184 compares event information with predefined event or sub-event definitions, and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of an event (e.g., a predefined sub-event sequence), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in events (e.g., 187-1 and / or 187-2) include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift-off (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift-off (touch end) of a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0140] In some embodiments, event definition 186 includes a definition of an event for a corresponding user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display 112, when a touch is detected on touch-sensitive display 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with a sub-event and the object that triggered the hit test.

[0141] In some embodiments, the definition of the corresponding event (187) also includes a delay action that delays the delivery of the event information until it has been determined that the sub-event sequence does or does not correspond to the event type of the event identifier.

[0142] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0143] In some embodiments, the corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0144] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from transmitting (and deferred transmission) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag obtains the flag and executes a predefined process.

[0145] In some embodiments, the event delivery instructions 188 include a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with a sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or with an actively participating view receives the event information and executes a predetermined process.

[0146] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video player module. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the positioning of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and transmits the display information to graphics module 132 for display on a touch-sensitive display.

[0147] In some embodiments, event handler 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0148] It should be understood that the above discussion of event processing of user touches on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate the multifunction device 100, and not all user input is initiated on the touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a touch pad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to sub-events that define the event to be distinguished.

[0149] Figure 2A portable multifunction device 100 with a touch screen 112 according to some embodiments is illustrated. The touch screen optionally displays one or more graphics within a user interface (UI) 200. In this embodiment and other embodiments described below, a user can select one or more of these graphics by, for example, making gestures on the graphics using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, when the user interrupts contact with one or more graphics, selection of one or more graphics will occur. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down) and / or rolling of fingers that have been in contact with the device 100 (from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with a graphic will not select the graphic. For example, when the gesture corresponding to the selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

[0150] The device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen 112.

[0151] In some embodiments, the device 100 includes a touch screen 112, a menu button 204, a push button 206 for turning the device on / off and for locking the device, one or more volume adjustment buttons 208, a user identity module (SIM) card slot 210, an earphone jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and keeping the button in a depressed state for a predefined time interval; lock the device by pressing the button and releasing the button before the predefined time interval passes; and / or unlock the device or initiate an unlocking process. In an alternative embodiment, the device 100 also accepts verbal input for activating or deactivating certain functions through a microphone 113. The device 100 also optionally includes one or more contact strength sensors 165 for detecting the strength of contact on the touch screen 112, and / or one or more tactile output generators 167 for generating tactile output for a user of the device 100.

[0152] Figure 3300 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a game system, or a control device (e.g., a home controller or an industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuits (sometimes referred to as a chipset) that interconnect system components and control communications between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touch screen display. The I / O interface 330 also optionally includes a keyboard and / or a mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on the device 300 (e.g., similar to the above reference Figure 1A The tactile output generator 167 described above), the sensor 359 (e.g., an optical sensor, an acceleration sensor, a proximity sensor, a touch sensitive sensor, and / or a contact intensity sensor (similar to the above reference Figure 1A Memory 370 optionally includes one or more storage devices located away from CPU 310. In some embodiments, memory 370 stores data related to portable multifunction device 100 ( Figure 1A ) or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100 ( Figure 1A )'s memory 102 optionally does not store these modules.

[0153] Figure 3Each element in the above-mentioned elements in is optionally stored in one or more memory devices of the previously mentioned memory device. Each module in the above-mentioned modules corresponds to the instruction set for performing the function described above. The above-mentioned modules or computer programs (for example, instruction sets or including instructions) do not have to be realized with independent software programs (such as computer programs (for example, including instructions)), processes or modules, and therefore the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores the subset of the above-mentioned modules and data structures. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0154] Attention is now turned to an embodiment of a user interface that is optionally implemented on, for example, portable multifunction device 100.

[0155] Figure 4A An exemplary user interface of an application menu on portable multifunction device 100 according to some embodiments is illustrated. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0156] Signal strength indicators 402 for wireless communications such as cellular signals and Wi-Fi signals;

[0157] Time 404;

[0158] Bluetooth indicator 405;

[0159] Battery status indicator 406;

[0160] A tray 408 with icons for commonly used applications, such as:

[0161] o An icon 416 of the phone module 138 labeled "Phone," which optionally includes an indicator 414 of the number of missed calls or voicemails;

[0162] o an icon 418 of the email client module 140 labeled “Mail”, which optionally includes an indicator 410 of the number of unread emails;

[0163] o An icon 420 labeled "Browser" of the browser module 147; and

[0164] ○ Video and music player module 152 (also known as iPod (trademark of Apple Inc.)

[0165] module 152) and an icon 422 labeled "iPod"; and

[0166] Icons for other apps, such as:

[0167] o Icon 424 labeled "Message" of IM module 141;

[0168] o An icon 426 labeled “Calendar” of the calendar module 148;

[0169] o Icon 428 labeled “Photos” of the image management module 144;

[0170] o An icon 430 labeled “Camera” of the camera module 143;

[0171] ○ Icon 432 labeled “Online Video” of the online video module 155;

[0172] ○ Icon 434 labeled “Stock Market” of the Stock Market Widget 149 - 2 ;

[0173] o An icon 436 labeled “Map” of the map module 154;

[0174] ○ Icon 438 labeled “Weather” of the weather widget 149-1;

[0175] ○ Icon 440 labeled as “Clock” of the alarm clock widget 149 - 4 ;

[0176] o An icon 442 labeled “Fitness Support” of the fitness support module 142;

[0177] o An icon 444 labeled "Notepad" of the Notepad module 153; and

[0178] o An icon 446 of a settings application or module labeled “Settings” that provides access to settings for the device 100 and its various applications 136 .

[0179] It should be noted that Figure 4A The illustrated icon labels are exemplary only. For example, the icon 422 of the video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0180] Figure 4B The example embodiment has a touch-sensitive surface 451 (eg, touch screen display 112) that is separate from a display 450 (eg, touch screen display 112). Figure 3 a tablet device or a touch pad 355) (e.g., Figure 3Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting intensity of contacts on touch-sensitive surface 451 and / or one or more tactile output generators 357 for generating tactile output for a user of device 300.

[0181] Although some of the examples below are given with reference to input on a touch screen display 112 (where a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the principal axis (for example, Figure 4B According to these embodiments, the device detects a position corresponding to a corresponding position on the display (e.g., Figure 4B 460 corresponds to 468 and 462 corresponds to 470) with touch-sensitive surface 451 (e.g., Figure 4B Thus, when the touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g., Figure 4B When the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is separated, the device is used to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0182] Additionally, although the following examples are primarily given with reference to finger inputs (e.g., finger contacts, single-finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detecting contact, followed by ceasing to detect contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or mice and finger contacts are optionally used simultaneously.

[0183] Figure 5AAn exemplary personal electronic device 500 is illustrated. Device 500 includes a body 502. In some embodiments, device 500 may include a body 502 relative to devices 100 and 300 (e.g., Figures 1A to 4B ) some or all of the features described in the foregoing. In some embodiments, device 500 has a touch-sensitive display screen 504, referred to hereinafter as touch screen 504. Alternatively, or in addition to touch screen 504, device 500 also has a display and a touch-sensitive surface. As with devices 100 and 300, in some embodiments, touch screen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of contact (e.g., touch) applied. One or more intensity sensors of touch screen 504 (or touch-sensitive surface) can provide output data representing the intensity of touch. The user interface of device 500 can respond to touch based on the intensity of the touch, which means that touches of different intensities can invoke different user interface operations on device 500.

[0184] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related applications: International patent application serial number PCT / US2013 / 040061, entitled “Device, Method, and Graphical User Interface for Displaying UserInterface Objects Corresponding to an Application,” filed on May 8, 2013, published as WIPO publication number WO / 2013 / 169849; and International patent application serial number PCT / US2013 / 069483, entitled “Device, Method, and Graphical User Interface for Transitioning Between TouchInput to Display Output Relationships,” filed on November 11, 2013, published as WIPO publication number WO / 2014 / 105276, each of which is hereby incorporated by reference in its entirety.

[0185] In some embodiments, the device 500 has one or more input mechanisms 506 and 508. The input mechanisms 506 and 508 (if included) can be physical. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, the device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) can allow the device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watchbands, bracelets, pants, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow the user to wear the device 500.

[0186] Figure 5B An exemplary personal electronic device 500 is depicted. In some embodiments, the device 500 may include a Figure 1A , Figure 1B and Figure 3 Some or all of the components described. Device 500 has a bus 512 that operatively couples an I / O portion 514 to one or more computer processors 516 and a memory 518. I / O portion 514 may be connected to a display 504, which may have a touch-sensitive component 522 and optionally an intensity sensor 524 (e.g., a contact intensity sensor). In addition, I / O portion 514 may be connected to a communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 is optionally a rotatable input device or a depressible input device and a rotatable input device. In some examples, input mechanism 508 is optionally a button.

[0187] In some examples, input mechanism 508 is optionally a microphone. Personal electronic device 500 optionally includes various sensors, such as GPS sensor 532, accelerometer 534, orientation sensor 540 (e.g., compass), gyroscope 536, motion sensor 538, and / or combinations thereof, all of which are operably connected to I / O portion 514.

[0188] The memory 518 of the personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by one or more computer processors 516, may, for example, cause the computer processors to perform the techniques described below, including processes 700 and 900 ( 7A to 7C and 9A to 9D). Computer-readable storage media can be any medium that can tangibly contain or store computer-executable instructions for use by or in conjunction with instruction execution systems, devices, and apparatuses. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical disks based on CD, DVD, or Blu-ray technology, and persistent solid-state memories such as flash memory, solid-state drives, and the like. Personal electronic device 500 is not limited to Figure 5B components and configurations, but may include other components or additional components in a variety of configurations.

[0189] As used herein, the term "indicative representation" refers to an optional feature of the apparatus 100, 300, and / or 500 ( Figure 1A , Figure 3 and FIG. 5A to FIG. 5B ) is a user-interactive graphical user interface object displayed on a display screen of a computer. For example, an image (e.g., an icon), a button, and text (e.g., a hyperlink) optionally each constitute an affordance.

[0190] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface that a user is interacting with. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., Figure 3 Touchpad 355 or Figure 4B In the event that an input (e.g., a press input) is detected on the touch-sensitive surface 451 in the display, the particular user interface element is adjusted according to the detected input. In the case that a touch-screen display (e.g., Figure 1A A touch-sensitive display system 112 or Figure 4AIn some implementations of the touch screen 112 in FIG. 1 , a contact detected on the touch screen acts as a “focus selector” such that when an input (e.g., a press input by the contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted in accordance with the detected input. In some implementations, the focus moves from one area of ​​the user interface to another area of ​​the user interface without corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move the focus from one button to another); in these implementations, the focus selector moves in accordance with the movement of the focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is typically a user interface element (or contact on the touch screen display) that is controlled by the user to deliver the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device display).

[0191] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted off, before or after contact starts to move, before contact ends, before or after contact intensity is detected to increase, and / or before or after contact intensity is detected to decrease). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the intensity of the contact, the mean value of the intensity of the contact, the average value of the intensity of the contact, the value at the top 10% of the intensity of the contact, the half-maximum value of the intensity of the contact, the 90% maximum value of the intensity of the contact, etc. In some embodiments, the duration of the contact is used when determining the characteristic intensity (e.g., when the characteristic intensity is the average value of the intensity of the contact over time). In some embodiments, the feature strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds optionally includes a first strength threshold and a second strength threshold. In this example, a contact whose feature strength does not exceed the first threshold results in a first operation, a contact whose feature strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose feature strength exceeds the second threshold results in a third operation. In some embodiments, a comparison between the feature strength and one or more thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding operation or to abandon the corresponding operation) rather than to determine whether to perform the first operation or the second operation.

[0192] Figure 5C An exemplary electronic device 580 is shown. Device 580 includes a body 580A. In some embodiments, device 580 may include a body 580A relative to devices 100, 300, and 500 (e.g., Figures 1A to 5B) Some or all of the features described in the main body 580A. In some embodiments, the device 580 has one or more speakers 580B (hidden in the main body 580A), one or more microphones 580C, one or more touch-sensitive surfaces 580D, and one or more displays 580E. Alternatively or in addition to the display and touch-sensitive surface 580D, the device also has a touch-sensitive display (also referred to as a touch screen). As with devices 100, 300, and 500, in some embodiments, the touch-sensitive surface 580D (or touch screen) optionally includes one or more intensity sensors for detecting the intensity of the applied contact (e.g., touch). The one or more intensity sensors of the touch-sensitive surface 580D (or touch screen) can provide output data representing the intensity of the touch. The user interface of the device 580 can respond to these touches based on the intensity of the touch, which means that touches of different intensities can invoke different user interface operations on the device 580. In some embodiments, the one or more displays 580E are one or more light emitting diodes (LEDs). For example, the display can be a single LED, a cluster of LEDs (e.g., red, green, and blue LEDs), multiple discrete LEDs, multiple discrete LED clusters, or other arrangements of one or more LEDs. For example, display 580E can be an array of nine discrete LED clusters arranged in a circle (e.g., an annular shape). In some examples, the one or more displays include one or more light-emitting elements of another type.

[0193] Figure 5D An exemplary personal electronic device 580 is depicted. In some embodiments, the device 580 may include a Figure 1A , Figure 1B , Figure 3 and FIG. 5A to FIG. 5B Some or all of the components described. Device 580 has a bus 592 that operatively couples an I / O portion 594 to one or more computer processors 596 and a memory 598. The I / O portion 594 may be connected to a display 582, which may have a touch-sensitive component 584 and optionally a strength sensor 585 (e.g., a contact strength sensor). In some embodiments, the touch-sensitive component 584 is a component separated from the display 582. In addition, the I / O portion 594 may be connected to a communication unit 590 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular and / or other wireless communication technologies. Device 580 may include an input mechanism 588. In some examples, the input mechanism 588 is optionally a button. In some examples, the input mechanism 588 is optionally a microphone. The input mechanism 588 is optionally a plurality of microphones (e.g., a microphone array).

[0194] The electronic device 580 includes a speaker 586 for outputting audio. The device 580 may include an audio circuit (e.g., in the I / O portion 594) that receives audio data, converts the audio data into an electrical signal, and sends the electrical signal to the speaker 586. The speaker 586 converts the electrical signal into a sound wave that can be heard by a human. The audio circuit (e.g., in the I / O portion 594) also receives an electrical signal converted from a sound wave by a microphone (e.g., input mechanism 588). The audio circuit (e.g., in the I / O portion 594) converts the electrical signal into audio data. The audio data is optionally retrieved from and / or sent to a memory 598 and / or an RF circuit (e.g., in the communication unit 590) by the I / O portion 594.

[0195] The memory 598 of the personal electronic device 580 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by one or more computer processors 596, may, for example, cause the computer processors to perform the techniques described below, including processes 700 and 900 ( 7A to 7C and 9A to 9D ). Computer-readable storage media can be any medium that can tangibly contain or store computer-executable instructions for use by or in conjunction with instruction execution systems, devices, and apparatuses. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical disks based on CD, DVD, or Blu-ray technology, and persistent solid-state memories such as flash memory, solid-state drives, and the like. Personal electronic device 580 is not limited to Figure 5D components and configurations, but may include other components or additional components for a variety of configurations.

[0196] Attention is now turned to embodiments of a user interface ("UI") and associated processes implemented on an electronic device, such as portable multifunction device 100, device 300, or device 500.

[0197] FIG. 6A to FIG. 6O Examples of techniques for outputting light according to some embodiments are illustrated. The user interfaces in these figures are used to illustrate the processes described below, including 7A to 7C process.

[0198] Fig. 6ASchematic diagram 600 of a device configuration is illustrated, the device configuration including a display 602, a first electronic device 604, a second electronic device 606, a first speaker accessory device 608, a second speaker accessory device 610, and a remote control 612. In some embodiments, the first electronic device 604 includes one or more features of the electronic device 100, 300, and / or 500. In some embodiments, the first electronic device 604 is a smart phone. In some embodiments, the second electronic device 606 includes one or more features of the electronic device 100, 300, and / or 500. In some embodiments, the second electronic device 606 is a media streaming device. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 include one or more features of the electronic device 100, 300, and / or 500. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 include a smart speaker.

[0199] exist Fig. 6A , the first electronic device 604 communicates with the second electronic device 606, the first speaker accessory device 608, and the second speaker accessory device 610, as indicated by the dotted lines. In some embodiments, the first electronic device 604 communicates with the display 602 via the second electronic device 606. In some embodiments, the first electronic device 604 communicates with the display 602 (e.g., direct communication, indirect communication, Bluetooth communication, Wi-Fi communication, and / or other Internet connection communication technology). The second electronic device 606 communicates with the display 602, the first electronic device 604, the first speaker accessory device 608, the second speaker accessory device 610, and the remote control 612, as indicated by the dotted lines.

[0200] In some embodiments, the first electronic device 604 is configured to control and / or adjust the settings of the second electronic device 606, the first speaker accessory device 608, and / or the second speaker accessory device 610 in response to detecting one or more user inputs. For example, in some embodiments, the first electronic device 604 can cause the second electronic device 606 to initiate playback of a media item (e.g., a movie, a television program, a song, a playlist, a podcast, a video game, and / or a slideshow), and / or pause and / or terminate playback of the media item. In some embodiments, the first electronic device 604 can adjust the volume of the audio output by the first speaker accessory device 608 and / or the second speaker accessory device 610.

[0201] In some embodiments, the second electronic device 606 causes the display 602 to display one or more images and / or videos associated with the media item. For example, in some embodiments, the second electronic device 606 transmits information to the display 602 and / or transmits information so that the display 602 can display the one or more images and / or videos. Fig. 6A At the remote control 612, the second electronic device 606 communicates with the remote control 612 and can cause the display 602 to initiate display of one or more images and / or videos in response to user input received and / or detected at the remote control 612. In some embodiments, the second electronic device 606 is configured to adjust the volume of the audio output of the first speaker accessory device 608 and / or the second speaker accessory device 610 in response to one or more user inputs received at the remote control 612. As described below, in some embodiments, the second electronic device 606 provides audio information (e.g., audio data related to content displayed on the display 602) to the first speaker accessory device 608 and / or the second speaker accessory device 610, so that the first speaker accessory device 608 and / or the second speaker accessory device 610 outputs audio associated with the media item being played back by the second electronic device 606.

[0202] In addition, in some embodiments, the second electronic device 606 provides information (e.g., information related to media content and / or data related to the color, brightness, and / or contrast of images and / or videos displayed on the display 602) to the first speaker accessory device 608 and / or the second speaker accessory device 610. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 outputs light based on and / or in accordance with the information. In some embodiments, the second electronic device 606 provides light information based on the media item (e.g., information associated with the color, brightness, and / or contrast of the light), and the first speaker accessory device 608 and / or the second speaker accessory device 610 outputs light based on the light information. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 simultaneously outputs audio and light based on information received from the second electronic device 606 (and / or another electronic device).

[0203] For example, in Figure 6B , a display 602, a second electronic device 606, a first speaker accessory device 608, and a second speaker accessory device 610 are illustrated in an environment 613 (e.g., a physical environment). The display 602 is positioned on (e.g., mounted on) a wall 613a of the environment 613, and the second electronic device 606, the first speaker accessory device 608, and the second speaker accessory device 610 are positioned on (e.g., resting on and / or placed on) a table 613b of the environment 613. Figure 6B, the second electronic device 606 communicates with the display 602 via a wired connection (e.g., a high-definition multimedia interface connection and / or another wired connection). In some embodiments, the second electronic device 606 communicates with the display 602 via a wireless communication technology (such as Bluetooth, Wi-Fi, and / or other Internet connection). The second electronic device 606 communicates with the first speaker accessory device 608 and the second speaker accessory device 610 via a wireless communication technology. In some embodiments, the second electronic device 606 communicates with the first speaker accessory device 608 and / or the second speaker accessory device 610 via a wired connection. As described above, the display 602, the second electronic device 606, the first speaker accessory device 608, and / or the second speaker accessory device 610 communicate (e.g., via wireless communication technology) with the first electronic device 604.

[0204] exist Figure 6B At, the display 602 displays the first trailer page 614 based on the information received from the second electronic device 606. The first trailer page 614 includes a preview of the media project configured to be played back and / or initiated by the second electronic device 606. The first trailer page 614 provides information about the media project and is displayed before the second electronic device 606 initiates the playback of the media project. For example, in some embodiments, the second electronic device 606 enables the display 602 to display one or more browsing menus (e.g., browsing menus of one or more applications) of the media project that the user can select to experience (e.g., watch and / or listen to). In some embodiments, when the user selects the user interface object corresponding to the corresponding media project, the second electronic device 606 enables the display 602 to display the first trailer page 614 that provides additional information about the media project to the user. In some embodiments, the second electronic device 606 does not initiate the playback of the media project when the display 602 displays the first trailer page 614. In some embodiments, the first trailer page 614 is static and does not change over time. In some embodiments, the first trailer page 614 includes a preview image, series of images, and / or video corresponding to the media item (e.g., a preview of the media item that does not include playback of the full duration of the media item).

[0205] exist Figure 6BAt , when the second electronic device 606 causes the display 602 to display the first trailer page 614, the second electronic device 606 provides (e.g., sends and / or transmits) information to the first speaker accessory device 608 and the second speaker accessory device 610. In some embodiments, the information includes information about the first trailer page 614, such as information about the color, brightness, and / or contrast of one or more images and / or videos of the first trailer page 614. In some embodiments, the information includes information about one or more colors, brightness, and / or contrast of light to be output by the first speaker accessory device 608 and / or the second speaker accessory device 610. Figure 6B Based on the information, the first speaker accessory device 608 outputs light 616a and the second speaker accessory device 610 outputs light 616b. Light 616a and light 616b include a first color, such as Figure 6B . The first trailer page 614 displayed on the display 602 also includes a background 614a having a first color, as indicated by the first shaded line. Thus, the first speaker accessory device 608 and the second speaker accessory device 610 output light 616a and light 616b, respectively, which include one or more attributes (e.g., color, brightness, and / or contrast) based on the first trailer page 614 (e.g., visual attributes of the first trailer page 614, such as color, brightness, and / or contrast). As described below, in some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 are configured to output light having a plurality of different colors (e.g., simultaneously output light having different colors). In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 simultaneously outputs light and outputs audio.

[0206] exist Figure 6BAt , first speaker accessory device 608 and second speaker accessory device 610 output light 616a and light 616b, respectively, such that light 616a and light 616b are projected onto wall 613a of environment 613. In some embodiments, light 616a and / or light 616b are projected onto different surfaces and / or objects of the environment (e.g., environment 613) in which first speaker accessory device 608 and / or second speaker accessory device 610 are located. First speaker accessory device 608 and second speaker accessory device 610 are configured to output light 616a and / or light 616b such that light 616a and / or light 616b are projected onto surfaces and / or objects so that a user can better view light 616a and / or light 616b. In some embodiments, first speaker accessory device 608 and / or second speaker accessory device 610 are positioned within the environment such that light 616a and / or light 616b are not projected onto objects and / or surfaces. In some embodiments, light 616a and / or light 616b are output from top portions of first speaker accessory device 608 and / or second speaker accessory device 610, respectively, and are emitted from one or more surfaces of first speaker accessory device 608 and / or second speaker accessory device 610, respectively. The output of light 616a and light 616b, respectively, by first speaker accessory device 608 and second speaker accessory device 610 based on first trailer page 614 provides a more pleasant and / or improved sensory experience to a user viewing first trailer page 614.

[0207] FIG. 6C to FIG. 6O The display 602, the first speaker accessory device 608, the second speaker accessory device 610, and the remote control 612 are illustrated as outputting various contents and / or receiving one or more user inputs. FIG. 6C to FIG. 6O 613, but the display 602, the first speaker accessory device 608, the second speaker accessory device 610, and / or the remote control 612 are configured as described with respect to FIG. 6C to FIG. 6O The description is in environment 613 and / or in a different environment. In addition, although FIG. 6C to FIG. 6O The first electronic device 604 and the second electronic device 606 are not shown, but the display 602, the first speaker accessory device 608, the second speaker accessory device 610 and / or the remote control 612 are configured to communicate with the first electronic device 604 and / or the second electronic device 606, as described above with reference to Fig. 6A and Figure 6B As explained.

[0208] Figure 6CThe first speaker accessory device 608 and the second speaker accessory device 610 are illustrated to output light based on one or more visual elements of the trailer page. Figure 6C At, the display 602 displays the second trailer page 618 based on the information received from the second electronic device 606. The second trailer page 618 corresponds to a media item, such as a movie and / or a television program. The second trailer page 618 provides the user with information related to the media item, and enables the user to cause the playback of the media item. For example, the second trailer page 618 includes a play user interface object 618a and more information user interface objects 618b. The second electronic device 606 communicates with the remote controller 612, which is configured to receive user inputs and enable the second electronic device 606 to perform operations in response to these user inputs. For example, in some embodiments, in response to detecting one or more user inputs of requesting to select more information user interface objects 618b at the remote controller 612, the second electronic device 606 causes the display 602 to display an information user interface including additional information and / or details related to the media item. As described below, in response to detecting user input 650a corresponding to a play button 612a of the remote control 612 (e.g., when focus is on the play user interface object 618a, as indicated by a border 618c displayed around the play user interface object 618a, or the play user interface object 618a is otherwise designated), the second electronic device 606 initiates playback of the media item and causes the display 602 to display one or more images and / or videos associated with the media item.

[0209] exist Figure 6C At , first speaker accessory device 608 outputs first light 616c and second light 616d, and second speaker accessory device 610 outputs third light 616e and fourth light 616f. As described above, first speaker accessory device 608 and second speaker accessory device 610 receive information from second electronic device 606 that includes information related to content displayed on display 602 and / or includes information about one or more properties of light that first speaker accessory device 608 and second speaker accessory device 610 are configured to output. For example, in some embodiments, the information includes information about one or more colors associated with an image and / or video displayed on display 602 (e.g., the color of second trailer page 618). In some embodiments, the information includes information about one or more colors, brightness, and / or contrast of first light 616c, second light 616d, third light 616e, and / or fourth light 616f. Thus, the first speaker accessory device 608 and / or the second speaker accessory device 610 output light based on one or more attributes (eg, one or more visual attributes, such as one or more colors) of the content displayed on the display 602 .

[0210] exist Figure 6C , the first light 616c includes the second color, as indicated by the second shaded line, and the second light 616d includes the third color, as indicated by the third shaded line. The second trailer page 618 displayed on the display 602 includes a first portion 618d and a second portion 618e, the first portion including the second color, as indicated by the third shaded line. Figure 6C As indicated by the second hatching shown at , the second portion includes a third color, such as Figure 6C Thus, the first speaker accessory device 608 outputs a first light 616c and a second light 616d that match the colors of the first portion 618d and the second portion 618e of the second trailer page 618 displayed on the display 602. Figure 6C , the third light 616e includes a fourth color, as indicated by a fourth shaded line, and the fourth light 616f includes a fifth color, as indicated by a fifth shaded line. The second trailer page 618 displayed on the display 602 includes a third portion 618f and a fourth portion 618g, the third portion including the fourth color, as indicated by a fifth shaded line. Figure 6C As indicated by the fourth hatching shown at , the fourth portion includes a fifth color, such as Figure 6C Thus, the second speaker accessory device 610 outputs third light 616e and fourth light 616f that match the colors of the third portion 618f and fourth portion 618g of the second trailer page 618 displayed on the display 602. Figure 6C At , the first speaker accessory device 608 and the second speaker accessory device 610 output light comprising two different colors. As described below, in some embodiments, the first speaker accessory device 608 and the second speaker accessory device 610 output light having a single color. In some embodiments, the first speaker accessory device 608 and the second speaker accessory device 610 output light having one or more colors that are different from each other.

[0211] In some embodiments, first speaker accessory device 608 and second speaker accessory device 610 output lights 616c, 616d, 616e, and / or 616f that are based on one or more colors of second trailer page 618, but do not match one or more colors of second trailer page 618 (e.g., the colors of lights 616c, 616d, 616e, and / or 616f complement and / or otherwise match the color scheme of second trailer page 618). Thus, first speaker accessory device 608 and second speaker accessory device 610 can provide a more pleasing and / or improved sensory experience to a user who is viewing second trailer page 618 on display 602.

[0212] exist Figure 6C 608 and the second speaker accessory device 610 do not output audio because the media item associated with the second trailer page 618 has not been played back (e.g., the second electronic device 606 has not yet initiated playback of the media item). In some embodiments, the second trailer page 618 includes a static image representing the media item associated with the second trailer page 618. In some embodiments, the second trailer page 618 includes a video and / or a series of images displayed over time. In some embodiments, the second trailer page 618 includes audio output via the first speaker accessory device 608 and / or the second speaker accessory device 610.

[0213] exist Figure 6C At , the second electronic device 606 receives an indication of a user input 650a (e.g., a press gesture or other selection / navigation input) corresponding to a play button 612a of the remote control 612 (e.g., when the focus is on the play user interface object 618a, as indicated by a border 618c displayed around the play user interface object 618a, or the play user interface object 618a is otherwise designated). In response to receiving the indication of the user input 650a, the second electronic device 606 initiates playback of the media item associated with the second trailer page 618 and causes the display 602 to display the first media content 620, as shown in FIG. Fig.6D as shown.

[0214] exist Fig.6D At , first speaker accessory device 608 and second speaker accessory device 610 output light based on one or more first visual elements of the media item at a first playback time of the media item. Fig.6D At, first media content 620 includes scenes and / or images of movies and / or television programs associated with the media item. The first media content 620 is associated with a first playback time of the media item. For example, the media item includes a playback duration, and the first media content 620 is associated with a first playback time within the playback duration of the media item. In some embodiments, as the playback duration of the media item proceeds (e.g., playback), the display 602 displays different content, and the first speaker accessory device 608 and / or the second speaker accessory device 610 output different audio and / or output different light. In some embodiments, the display 602 displays different content, and the first speaker accessory device 608 and / or the second speaker accessory device 610 output different audio and / or output different light when the media item is played back, without receiving an indication of a user input at the remote controller 612.

[0215] The first media content 620 includes a background 620a, which includes a first background color, such as Fig.6D As described above, the first speaker accessory device 608 and the second speaker accessory device 610 output light based on the received information (such as information about one or more colors, brightness, and / or contrast of one or more images displayed on the display 602). In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 receives information from the second electronic device 606. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 receives information from another electronic device (e.g., the display 602, the first electronic device 604, and / or a server). Fig.6D At , the first speaker accessory device 608 outputs light 616g having a first background color, such as Fig.6D , and the second speaker accessory device 610 outputs light 616h having a first background color, as indicated by the sixth hatched line at Fig.6D In some embodiments, light 616g and light 616h are based on information about the first media content 620, such as the color, brightness, and / or contrast of one or more images associated with the first media content 620 (e.g., background 620a of the first media content 620). Fig.6D , light 616g, light 616h, and background 620a of first media content 620 each include a first background color. In some embodiments, first speaker accessory device 608 and / or second speaker accessory device 610 receive information about one or more properties (e.g., one or more colors, brightness, and / or contrast) of light 616g and / or light 616h, respectively. In some embodiments, second electronic device 606 (and / or a different electronic device) determines one or more properties of light 616g and / or light 616h based on first media content 620.

[0216] In some embodiments, first media content 620 includes multiple colors such that light 616g and / or light 616h includes more than one color based on the multiple colors of first media content 620. Thus, first speaker accessory device 608 and second speaker accessory device 610 can provide a more pleasant and / or improved sensory experience for a user viewing first media content 620 (e.g., a media item associated with first media content 620).

[0217] exist Fig.6DAt , the first speaker accessory device 608 outputs the first audio 622a, and the second speaker accessory device 610 outputs the second audio 622b. The first audio 622a and the second audio 622b are associated with the first media content 620, and are based on the audio information received by the first speaker accessory device 608 and the second speaker accessory device 610, respectively. In some embodiments, the first audio 622a and the second audio 622b are different from each other. For example, in some embodiments, the first audio 622a includes a left channel audio output of the audio associated with the first media content 620 (e.g., a media item associated with the first media content 620), and the second audio 622b includes a right channel audio output of the audio associated with the first media content 620 (e.g., a media item associated with the first media content 620). In some embodiments, the first audio 622a and the second audio 622b are the same as each other. In some embodiments, the first audio 622a and / or the second audio 622b include audio outputs that include speech, dialogue, music, and / or sound effects associated with the first media content 620 (e.g., a media item associated with the first media content 620). Thus, the first speaker accessory device 608 and / or the second speaker accessory device 610 are configured to output audio associated with the media item. Thus, a user viewing the first media content 620 on the display 602 can listen to the audio of the media item via the first audio 622a and the second audio 622b.

[0218] exist Fig. 6E At , first speaker accessory device 608 and second speaker accessory device 610 output light based on one or more second visual elements of the media item at a second playback time of the media item that is different from the first playback time. Fig. 6E At , display 602 is displaying second media content 624, which includes a second scene and / or image of a movie and / or television program associated with the media item. The second media content 624 is associated with a second playback time of the media item, which is different from the first playback time associated with the first media content 620 (e.g., after the first playback time). Therefore, as the playback duration of the media item progresses (e.g., playback), the display 602 displays different content, and / or the first speaker accessory device 608 and / or the second speaker accessory device 610 outputs different audio and / or outputs different light (e.g., light with different characteristics and / or attributes). In some embodiments, the display 602 displays different content, and / or the first speaker accessory device 608 and / or the second speaker accessory device 610 outputs different audio and / or outputs different light when the media item is played back, without receiving an indication of a user input at the remote control 612.

[0219] exist Fig. 6E At , the second media content 624 includes a first part 624a and a second part 624b. The first part 624a includes a second background color, such as Fig. 6E , and the second portion 624b includes a third background color, such as Fig. 6E The first portion 624a of the second media content 624 is displayed on the first display area 602a of the display 602, and the second portion 624b of the second media content 624 is displayed on the second display area 602b of the display 602. Fig. 6E , first speaker accessory device 608 outputs light 616i having a second background color, as indicated by the seventh shaded line, and second speaker accessory device 610 outputs light 616j having a third background color, as indicated by the eighth shaded line. Thus, first speaker accessory device 608 outputs light 616i based on first portion 624a of second media content 624, and second speaker accessory device 610 outputs light 616j based on second portion 624b of second media content 624. Figure 6B As shown, the first speaker accessory device 608 is positioned in the environment 613 at a first position close to (eg, closer than the second speaker accessory device 610) the display area 602a. Fig. 6E At , the first portion 624a is displayed on the display 602 at the display area 602a. Figure 6B At , the second speaker accessory device 610 is positioned in the environment 613 at a second location proximate to (eg, closer than the first speaker accessory device 608) the display area 602b, at Fig. 6E , and second portion 624b is displayed on display 602 at display area 602b. Thus, first speaker accessory device 608 and second speaker accessory device 610 may output light with different properties based on the positioning and / or location of different portions of content displayed on display 602.

[0220] In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 include a configuration indicating a positioning of the first speaker accessory device 608 and / or the second speaker accessory device 610 relative to the display 602 (e.g., in the environment 613). In some embodiments, the second electronic device 606 (and / or another electronic device) causes the first speaker accessory device 608 and the second speaker accessory device 610 to output light having different properties from each other based on the configuration of the first speaker accessory device 608 and the second speaker accessory device 610. For example, in some embodiments, the second electronic device 606 (and / or another electronic device) provides first information to the first speaker accessory device 608, and provides second information different from the first information to the second speaker accessory device 610. In some embodiments, the first speaker accessory device 608 outputs light having a first property (e.g., light 616i) based on the first information, and the second speaker accessory device 610 outputs light having a second property different from the first property (e.g., light 616j) based on the second information. In some embodiments, the properties of the light include one or more of color, brightness, and / or contrast. In some embodiments, the first information includes information based on a display area 602 a of the display 602 , and the second information includes information based on a display area 602 b of the display 602 .

[0221] exist Fig. 6E At , first speaker accessory device 608 outputs audio 622c, and second speaker accessory device 610 outputs audio 622d. Audio 622c and audio 622d are associated with second media content 624 (e.g., the playback time of a media item associated with second media content 624), and are based on audio information received by first speaker accessory device 608 and second speaker accessory device 610, respectively. As described above, in some embodiments, audio 622c and audio 622d are different from each other. For example, in some embodiments, audio 622c includes a left channel audio output of audio associated with second media content 624, and audio 622d includes a right channel audio output of audio associated with second media content 624. In some embodiments, audio 622c and audio 622d are identical to each other. In some embodiments, audio 622c and / or audio 622d include an audio output that includes speech, dialogue, music, and / or sound effects associated with second media content 624 (e.g., a media item associated with second media content 624). Thus, first speaker accessory device 608 and / or second speaker accessory device 610 are configured to output audio associated with the media item. Thus, a user viewing second media content 624 on display 602 can listen to the audio of the media item via audio 622c and audio 622d.

[0222] exist Fig. 6F At a first time during the sporting event, first speaker accessory device 608 and second speaker accessory device 610 output light based on the attributes of the sporting event. Fig. 6F At , display 602 shows a first sports event 626, which includes a representation (e.g., image, video frame and / or live video frame) of a sports event (e.g., a football match). The first sports event 626 includes a scoreboard 628, which has a first team score 628a, a second team score 628b and a time indicator 628c (e.g., the time remaining in a part (e.g., half, quarter or a period) of the sports event and / or the time elapsed from the start of the sports event). The first team score 628a indicates that team A has two points, and the second team score 628b indicates that team B has zero points. Therefore, team A currently wins the sports event. The first team score 628a includes a first team color representing a first color scheme (e.g., team color) associated with team A, as indicated by the ninth hatched line. Similarly, the second team score 628b includes a second team color representing a second color scheme (e.g., team color) associated with team B, as indicated by the tenth hatched line.

[0223] exist Fig. 6F At , first speaker accessory device 608 and second speaker accessory device 610 receive information based on current scores and / or recent events of a sporting event from second electronic device 606 (and / or another electronic device). Fig. 6F , first speaker accessory device 608 and second speaker accessory device 610 output light 616k and light 616l, respectively, where light 616k and light 616l each include the first team color. First speaker accessory device 608 and second speaker accessory device 610 output light 616k and light 616l, respectively, because team A is the team currently winning the sporting event. Therefore, a user watching first sporting event 626 can easily determine which team participating in the sporting event is currently winning via light 616k and / or light 616l.

[0224] exist Fig. 6F At , first speaker accessory device 608 and second speaker accessory device 610 output audio 622e and audio 622f, respectively. Audio 622e and audio 622f are associated with first sporting event 626 (e.g., audio of a commentator of the sporting event and / or audio otherwise associated with the sporting event) and are based on audio information received by first speaker accessory device 608 and second speaker accessory device 610, respectively. Thus, first speaker accessory device 608 and second speaker accessory device 610 simultaneously output audio and light to improve the user's experience of watching first sporting event 626.

[0225] exist Figure 6G At , first speaker accessory device 608 and second speaker accessory device 610 output light based on the attribute of the sporting event at a second time different from the first time. Figure 6G , the display 602 is showing a second sporting event 630, which also includes a representation of a sporting event associated with the first sporting event 626 (e.g., the same sporting event associated with the first sporting event 626 at a different time in the sporting event). The second sporting event 630 includes a scoreboard 628 and an event indicator 632. The scoreboard 628 includes a first team score 628a indicating that team A has scored two points and a second team score 628b indicating that team B has scored three points. The event indicator 632 (e.g., "Goal!") indicates that team B has recently scored their third point and is ahead of team B. In some embodiments, the display 602 does not display the event indicator 632, but instead displays the second team score 628b with an updated score for team B.

[0226] exist Figure 6G At , the first speaker accessory device 608 and the second speaker accessory device 610 output light 616m and light 616n, respectively. Figure 6G 616m and light 616n each include a second team color associated with Team B, as indicated by the tenth hatched line. In some embodiments, first speaker accessory device 608 and second speaker accessory device 610 output light 616m and light 616n, respectively, based on Team B scoring and / or scoring a goal (as indicated by event indicator 632). In some embodiments, first speaker accessory device 608 and second speaker accessory device 610 output light 616m and light 616n, respectively, based on Team B leading Team A in a sporting event. In some embodiments, first speaker accessory device 608 and second speaker accessory device 610 output light 616m and light 616n, respectively, based on Team B scoring and / or scoring a goal and Team B leading Team A in a sporting event.

[0227] In some embodiments, when Team A and Team B are tied, first speaker accessory device 608 outputs light 616k having a first team color associated with Team A, and second speaker accessory device 610 outputs light 616n having a second team color associated with Team B. In some embodiments, when a team that is not currently winning the sporting event scores and / or scores a goal, first speaker accessory device 608 and / or second speaker accessory device 610 outputs light having a color associated with the team that scored and / or scored a goal for a predetermined period of time, and then outputs light having a color associated with the team that is currently leading. As described above, first speaker accessory device 608 and second speaker accessory device 610 receive information from second electronic device 606 that provides information about the sporting event and / or information about one or more properties of light that first speaker accessory device 608 and second speaker accessory device are configured to output.

[0228] Although Fig. 6F and Figure 6G First sporting event 626 and second sporting event 630 are illustrated as including representations of a soccer game, but first sporting event 626 and / or second sporting event 630 may include representations of any sporting event. Thus, first speaker accessory device 608 and / or second speaker accessory device 610 may output light to indicate the current winning team and / or player of the sporting event, the team and / or player that most recently scored and / or scored a goal, and / or the team and / or player that most recently received an advantage and / or penalty.

[0229] exist Figure 6H At , first speaker accessory device 608 and second speaker accessory device 610 output light based on one or more visual elements of the symbol. Figure 6H 6, the display 602 is displaying a logo 634, which includes a logo and / or a brand symbol. The logo 634 includes a first portion 634a having a first logo color and a second portion 634b having a second logo color, as indicated by the eleventh and twelfth hatched lines, respectively. In some embodiments, the display 602 displays the logo 634 as part of the media item currently being played. In some embodiments, the display 602 displays the logo 634 as part of an inactive screen and / or a screen saver.

[0230] exist Figure 6HAt , the first speaker accessory device 608 outputs light 616o having the first logo color, as represented by the eleventh hatched line, and the second speaker accessory device 610 outputs light 616p having the second logo color, as represented by the twelfth hatched line. The first speaker accessory device 608 and the second speaker accessory device 610 receive information related to the logo 634 (e.g., one or more visual attributes of the first portion 634a, the second portion 634b, and / or another portion of the logo 634) and / or information about one or more attributes of the light 616o and the light 616p. As described above, in some embodiments, based on the configuration of the first speaker accessory device 608 and the second speaker accessory device 610, such as the configuration (e.g., physical configuration) of the first speaker accessory device 608 and the second speaker accessory device 610 relative to the display 602, the first speaker accessory device 608 and the second speaker accessory device 610 respectively output light 616o and light 616p. For example, at Figure 6B At Figure 6H When displayed on the display 602, the first speaker accessory device 608 is positioned at a first position proximate to (eg, compared to) the second speaker accessory device 610) the first portion 634a. Figure 6B At, when the second portion 634b is Figure 6H When displayed on the display 602, the second speaker accessory device 610 is positioned at a second position proximate (eg, compared to the first speaker accessory device 608) to the second portion 634b.

[0231] In some embodiments, when display 602 is displaying logo 634, first speaker accessory device 608 and / or second speaker accessory device 610 outputs light having both the first logo color and the second logo color (e.g., rather than first speaker accessory device 608 outputting light having the first logo color and second speaker accessory device 610 outputting light having the second logo color). In some embodiments, when display 602 is displaying logo 634, first speaker accessory device 608 and / or second speaker accessory device 610 outputs audio simultaneously with light 616o and / or light 616p, respectively.

[0232] exist Fig.6I At , first speaker accessory device 608 and second speaker accessory device 610 output light based on one or more visual elements of the artist page and / or playlist page. Fig.6IAt , the display 602 is displaying a media page 636 including an artist page and / or a playlist page, which corresponds to one or more media items, such as one or more songs in a playlist and / or an album. The artist page and / or the playlist page provide information related to one or more media items to the user, and enable the user to cause the playback of one or more media items. For example, the media page 636 includes a playback user interface object 636a and a playlist user interface object 636b. The second electronic device 606 communicates with the remote controller 612, which is configured to receive user inputs and enable the second electronic device 606 to perform operations in response to these user inputs. For example, in some embodiments, in response to detecting one or more user inputs requesting to select a playlist user interface object 636b at the remote controller 612, the second electronic device 606 causes the display 602 to display an information user interface including additional information and / or details related to one or more media items (e.g., one or more songs included in a playlist). As described below, in response to detecting user input 650b corresponding to a play button 612a of the remote control 612 (e.g., when the focus is on the play user interface object 636a, as indicated by a border 636c displayed around the play user interface object 636a, or the play user interface object 636a is otherwise designated), the second electronic device 606 initiates playback of a media item from one or more media items and causes the display 602 to display one or more images and / or videos associated with the media item.

[0233] exist Fig.6I6, media page 636 includes background 636d having a fourth background color, as indicated by the thirteenth hatched line. First speaker accessory device 608 outputs light 616q having a fourth background color, and second speaker accessory device 610 outputs light 616r having a fourth background color. As described above, first speaker accessory device 608 and second speaker accessory device 610 receive information from second electronic device 606 that includes information related to content displayed on display 602 and / or includes information about one or more properties of light that first speaker accessory device 608 and second speaker accessory device 610 are configured to output. For example, in some embodiments, the information includes information about one or more colors associated with an image and / or video displayed on display 602 (e.g., the color of background 636d of media page 636). In some embodiments, the information includes information about one or more colors, brightness, and / or contrast of light 616q and / or 616r. Thus, first speaker accessory device 608 and / or second speaker accessory device 610 output light based on one or more attributes of media page 636 displayed on display 602 (eg, one or more visual attributes of media page 636, such as one or more colors).

[0234] Although Fig.6I Light 616q and light 616r are shown having a fourth background color that matches the fourth background color of background 636d, but in some embodiments, light 616q and / or light 616r include one or more different colors from the fourth background color of background 636d and / or another portion based on media page 636.

[0235] exist Fig.6I At 612a, the second electronic device 606 receives an indication of a user input 650b corresponding to the button 612b of the remote control 612. In response to receiving the indication of the user input 650b, the second electronic device 606 initiates playback of a media item (such as a song included in a song playlist) associated with the media page 636 from among the one or more media items. After the second electronic device 606 initiates playback of the media item, the display 602 displays the first media item content 638, such as Figure 6J as shown.

[0236] exist Figure 6JAt , first speaker accessory device 608 and second speaker accessory device 610 output light based on one or more visual attributes of first media item content 638. First media item content 638 includes information about the media item. For example, first media item content 638 includes background 638a, visual element 638b, and media item indicator 638c. Background 638a includes symbols, icons, images, and / or visual elements associated with the media item being played back (such as a song in a playlist). In some embodiments, background 638a includes a color based on one or more attributes of visual element 638b. For example, at Figure 6J At , visual element 638b includes a representation of an album cover and / or image associated with the media item. Both background 638a and visual element 638b include a first media item color, such as Figure 6J The media item indicator 638c includes text that identifies and / or provides information about the media item, such as a song title, artist name, album name, and / or playlist name.

[0237] exist Figure 6J At , first speaker accessory device 608 outputs light 616s having the first media item color, and second speaker accessory device 610 outputs light 616t having the first media item color. First speaker accessory device 608 and second speaker accessory device 610 receive information from second electronic device 606 (and / or another electronic device) that includes information about first media item content 638 (e.g., first speaker accessory device and / or second speaker accessory device 610 determine one or more properties of light 616s and / or light 616t, respectively) and / or information about one or more properties of light 616s and light 616t, wherein the one or more properties of light 616s and light 616t are based on first media item content 638 (e.g., second electronic device 606 and / or another electronic device determine one or more properties of light 616s and light 616t). Thus, first speaker accessory device 608 and second speaker accessory device 610 output light 616s and light 616t, respectively, based on one or more attributes (e.g., visual attributes) of first media item content 638, such as one or more colors of background 638a and / or visual element 638b.

[0238] exist Figure 6JAt , first speaker accessory device 608 outputs audio 622g, and second speaker accessory device 610 outputs audio 622h. Audio 622g and audio 622h are associated with first media item content 638 (e.g., audio of a song), and are based on audio information received by first speaker accessory device 608 and second speaker accessory device 610, respectively. In some embodiments, first speaker accessory device 608 and / or second speaker accessory device 610 receive audio information from second electronic device 606. In some embodiments, first speaker accessory device 608 and / or second speaker accessory device 610 receive audio information from another electronic device (e.g., first electronic device 604 and / or a server). Therefore, first speaker accessory device 608 and second speaker accessory device 610 simultaneously output audio and light to improve the user's experience of viewing first media item content 638.

[0239] In some embodiments, media page 636 is associated with the media item and the second media item. In some embodiments, after the media item corresponding to first media item content 638 ends (e.g., is played back for the entire duration of the media item and / or is skipped), second electronic device 606 (and / or another electronic device) causes playback of the second media item and / or causes display 602 to display second media item content 640, such as Figure 6K as shown.

[0240] exist Figure 6K At , first speaker accessory device 608 and second speaker accessory device 610 output light based on one or more visual attributes of second media item content 640. Second media item content 640 includes information about the second media item. For example, second media item content 640 includes background 640a, visual element 640b, and media item indicator 640c. Background 640a includes symbols, icons, images, and / or visual elements associated with the media item being played back (such as a song in a playlist). In some embodiments, background 640a includes a color based on one or more attributes of visual element 640b. For example, at Figure 6K At , visual element 640b includes a representation of an album cover and / or image associated with the media item. Both background 640a and visual element 640b include a second media item color, such as Figure 6K The media item indicator 640c includes text that identifies and / or provides information about the second media item, such as a song title, artist name, album name, and / or playlist name.

[0241] exist Figure 6KAt , first speaker accessory device 608 outputs light 616u having one or more first attributes (such as a first brightness and a second media item color), and second speaker accessory device 610 outputs light 616v having one or more second attributes (such as a second brightness and a second media item color). First speaker accessory device 608 and second speaker accessory device 610 receive information from second electronic device 606 (and / or another electronic device) that includes information about second media item content 640 (e.g., first speaker accessory device and / or second speaker accessory device 610 determine one or more attributes of light 616u and / or light 616v, respectively) and / or information about one or more attributes of light 616u and light 616v, wherein the one or more attributes of light 616u and light 616v are based on second media item content 640 (e.g., second electronic device 606 and / or another electronic device determine one or more attributes of light 616u and light 616v). As described below, in some embodiments, one or more attributes of light 616u and / or light 616v, respectively, are based on the volume level of audio output by first speaker accessory device 608 and second speaker accessory device 610. Thus, first speaker accessory device 608 and second speaker accessory device 610 output light 616u and light 616v, respectively, based on one or more attributes (e.g., visual attributes) of second media item content 640, such as one or more colors of background 640a and / or visual element 640b, and / or the volume level of audio output by first speaker accessory device 608 and second speaker accessory device 610.

[0242] exist Figure 6K At , first speaker accessory device 608 outputs audio 622i, and second speaker accessory device 610 outputs audio 622j. Audio 622i and audio 622j are associated with second media item content 640 (e.g., audio of a song), and are based on audio information received by first speaker accessory device 608 and second speaker accessory device 610, respectively. Thus, first speaker accessory device 608 and second speaker accessory device 610 simultaneously output audio and light to improve the user's experience of viewing second media item content 640. Figure 6K 6, the first speaker accessory device 608 outputs audio 622i at a first volume, as indicated by a first volume indicator 642a. The second speaker accessory device 610 outputs audio 622j at a second volume, as indicated by a second volume indicator 642b. In some embodiments, the first volume of audio 622i and the second volume of audio 622j are the same. For clarity, the illustrated first volume indicator 642a and second volume indicator 642b are provided, but they are not part of the second media item content 640 displayed on the display 602.

[0243] Light 616u includes a first brightness based on a first volume of audio 622i, and light 616v includes a second brightness based on a second volume of audio 622j. Figure 6K , the fifteenth shaded line representing light 616u indicates that light 616u includes both the second media item color and the first brightness. Similarly, the fifteenth shaded line representing light 616v indicates that light 616v includes both the second media item color and the second brightness. Thus, in some embodiments, one or more attributes of light output by first speaker accessory device 608 and / or second speaker accessory device 610 are based on the volume level of content displayed on display 602 and / or audio output by first speaker accessory device 608 and / or second speaker accessory device 610.

[0244] In some embodiments, first speaker accessory device 608 and second speaker accessory device 610 are configured to adjust the respective volumes of audio 622i and audio 622j in response to receiving an indication of user input (e.g., receiving an indication of user input corresponding to remote control 612 from second electronic device 606 and / or another electronic device). In some embodiments, first speaker accessory device 608 and second speaker accessory device 610 are configured to adjust the respective volumes of audio 622i and audio 622j based on information received from second electronic device 606 (and / or another electronic device) without receiving an indication of user input. For example, in some embodiments, audio information associated with a media item includes different volumes at different playback times. Therefore, first speaker accessory device 608 and second speaker accessory device 610 automatically adjust the volumes of audio 622i and audio 622j, respectively, based on the audio information associated with the media file.

[0245] exist Figure 6K At , the second electronic device 606 (and / or another electronic device) receives an indication of a user input 650c corresponding to a volume button 612c of the remote control 612. In response to receiving the indication of the user input 650c, the second electronic device 606 (and / or another electronic device) causes the first speaker accessory device 608 and the second speaker accessory device 610 to adjust the volume of the audio 622i and the audio 622j, respectively, as shown in FIG. Figure 6L as shown.

[0246] exist Figure 6LAt, the display 602 maintains the display of the second media item content 640, and displays the volume adjustment indicator 644 superimposed on the second media item content 640. In some embodiments, the display 602 displays the volume adjustment indicator 644 in response to receiving information indicating the user input 650c from the second electronic device 606 (and / or another electronic device). In some embodiments, the volume adjustment indicator 644 moves on the display 602 and / or is animated to visually show the adjustment of the volume of the audio output (e.g., the audio output from the speaker of the display 602 and / or the audio 622i and the audio 622j). In some embodiments, the display 602 does not include and / or displays the volume adjustment indicator 644. For example, as described above, in some embodiments, the volume of the audio output (e.g., audio 622i and / or audio 622j) associated with the second media item content 640 is adjusted without receiving an indication of the user input (e.g., user input 650c).

[0247] exist Figure 6L At , the first speaker accessory device 608 and the second speaker accessory device 610 adjust the brightness of the output light based on the adjustment of the volume of the audio output by the first speaker accessory device 608 and the second speaker accessory device 610. For example, at Figure 6L , first volume indicator 642a shows that audio 622i is at a third volume greater than the first volume, and second volume indicator 642b shows that audio 622j is at a fourth volume greater than the second volume. Thus, in response to receiving an indication of user input 650c from second electronic device 606 (and / or another electronic device), first speaker accessory device 608 and second speaker accessory device 610 increase the volume of audio 622i and audio 622j, respectively. Figure 6L , first speaker accessory device 608 outputs light 616w having one or more third attributes, such as a second media item color and a third brightness, and second speaker accessory device 610 outputs light 616x having one or more fourth attributes, such as a second media item color and a fourth brightness. Light 616w includes a third brightness greater than the first brightness based on a third volume of audio 622i, and light 616x includes a fourth brightness greater than the second brightness based on a fourth volume of audio 622j. For example, at Figure 6L At , the sixteenth hatched line representing light 616w indicates that light 616w includes the second media item color and the third brightness. Similarly, the sixteenth hatched line representing light 616x indicates that light 616x includes the second media item color and the fourth brightness.

[0248] Thus, the first speaker accessory device 608 outputs light 616w that maintains the second media item color, but includes a higher brightness than light 616u due to the increased volume of audio 622i. Similarly, the second speaker accessory device 610 outputs light 616x that maintains the second media item color, but includes a higher brightness than light 616v due to the increased volume of audio 622j. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 do not adjust the brightness of the output light based on changes in the volume of the audio output by the first speaker accessory device 608 and / or the second speaker accessory device 610, respectively. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 adjust different attributes of the output light based on changes in the volume of the audio output by the first speaker accessory device 608 and / or the second speaker accessory device 610, respectively.

[0249] The first speaker accessory device 608 and / or the second speaker accessory device 610 are configured to output light having one or more predetermined attributes (e.g., light having a predetermined color) when the second electronic device 606 (and / or another electronic device) may perform a function and / or operation. Figure 6M At , when display 602 is displaying second media item content 640 and when first speaker accessory device 608 outputs light 616w, second speaker accessory device 610 outputs light 616y that includes one or more properties that are different from light 616x, such as Figure 6M The light 616y provides guidance on the functions and / or operations that can be performed, and / or prompts the user to perform the functions and / or operations. In some embodiments, the second speaker accessory device 610 outputs light 616y based on information received from the second electronic device 606 (and / or another electronic device) that the functions and / or operations can be performed.

[0250] In some embodiments, the second electronic device 606 (and / or another electronic device) provides information to the second speaker accessory device 610 (and / or the first speaker accessory device 608) in response to receiving an indication of one or more user inputs (e.g., a touch gesture detected at the remote control 612), in response to receiving an indication of a notification, and / or in response to receiving an indication that an external device (e.g., headphones) is within the communication range of the second electronic device 606 (and / or another electronic device). Therefore, the second speaker accessory device 610 outputs light 616y to provide guidance to the user that the function and / or operation can be performed. In some embodiments, the second electronic device 606 (and / or another electronic device) can perform the function and / or operation in response to receiving an indication of one or more second user inputs at the remote control 612. As described below, in some embodiments, the function and / or operation causes the display 602 to display a setup user interface (e.g., setup user interface 648) that enables control of playback of media items, display of content on the display 602, launching and / or navigating to one or more applications, and / or connecting external devices.

[0251] exist Figure 6M , display 602 is displaying prompt 646 superimposed on second media item content 640. In some embodiments, prompt 646 is also displayed when a function and / or operation can be performed. Thus, in some embodiments, prompt 646 provides additional guidance to the user (e.g., in response to one or more user inputs) that a function and / or operation can be performed. Figure 6M At , prompt 646 is partially displayed on display 602 such that a portion of prompt 646 is not shown on display 602. Partially displaying prompt 646 on display 602 provides instructions and / or guidance to the user, namely, providing one or more user inputs to cause additional user interfaces and / or information to be displayed on display 602. For example, by displaying a portion of prompt 646 on display 602, the user may understand that additional visual elements are not displayed on display 602, and providing one or more user inputs may cause such additional elements to be displayed on display 602. In some embodiments, display 602 does not display prompt 646 when second speaker accessory device 610 outputs light 616y.

[0252] exist Figure 6M At , the second electronic device 606 (and / or another electronic device) receives an indication of a user input 650d corresponding to the button 612b of the remote control 612. In response to receiving the indication of the user input 650d, the second electronic device 606 (and / or another electronic device) initiates execution of the function and / or operation and causes the display of the setting user interface 648, such as Figure 6N as shown.

[0253] exist Figure 6N At , the second electronic device 606 (and / or another electronic device) causes the display 602 to display the setup user interface 648 in response to receiving an indication of the user input 650d. The setup user interface 648 includes user interface objects that enable control of the second electronic device 606 (and / or another electronic device). For example, in some embodiments, in response to receiving an indication of the user input selecting the user interface object 648a, the second electronic device 606 initiates a sleep mode and causes the display 602 to stop displaying the second media item content 640 and the setup user interface 648.

[0254] exist Figure 6N 648 is displayed on the display 602. In some embodiments, the first speaker accessory device 608 outputs light 616z and the second speaker accessory device 610 outputs light 616aa. Light 616z includes one or more properties (e.g., one or more colors, brightness, and / or contrast) that are different from light 616w, which was output by the first speaker accessory device 608 before (e.g., before) the display 602 displays the setup user interface 648. Similarly, light 616aa includes one or more properties (e.g., one or more colors, brightness, and / or contrast) that are different from light 616x and / or light 616y, which were output by the second speaker accessory device 610 before (e.g., before) the display 602 displays the setup user interface 648. Thus, in some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 adjust and / or change one or more properties of the outputted light based on the display 602 displaying the setup user interface 648. In some embodiments, first speaker accessory device 608 maintains output of light 616w when display 602 displays setup user interface 648, and second speaker accessory device 610 maintains output of light 616x and / or light 616y when display 602 displays setup user interface 648.

[0255] The first speaker accessory device 608 and / or the second speaker accessory device 610 are configured to perform functions other than outputting light and audio. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 are smart speakers that communicate with an external device (e.g., a server) via wireless communication technology (e.g., Bluetooth, Wi-Fi, and / or other Internet connections). Thus, in some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 can request and / or receive information from the external device, such as news, weather information, media items (e.g., music and / or podcasts), and / or location information. In some embodiments, the first speaker accessory device 608 and / or the second speaker accessory device 610 can perform various functions in response to user input (e.g., voice input) including predetermined words and / or phrases (e.g., "Hey, Assistant"). In some embodiments, in response to detecting the predetermined words and / or phrases, the first speaker accessory device 608 and / or the second speaker accessory device 610 are configured to detect the user's speech (e.g., speech) and determine the function to be performed based on the detected user's speech.

[0256] For example, in Figure 6N At , second speaker accessory device 610 detects user input 650e (e.g., a voice command) requesting second speaker accessory device 610 to perform a function (e.g., “Hey, assistant, what’s the weather like?”). Figure 6N In response to detecting user input 650e, second speaker accessory device 610 outputs light 616ab and outputs audio 622k, such as Fig.6O as shown.

[0257] exist Fig.6O At 616a, audio 622k includes a response "Sunny, 70° (Fahrenheit)", which is a response to a request associated with user input 650e. In addition, light 616ab includes a virtual auxiliary color, such as Fig.6O Light 616ab is different from light 616v, light 616x, light 616y, and / or light 616aa, and is associated with performing an operation including outputting audio 622k (e.g., performing a function in response to detecting user input 650e). Fig.6O, the first speaker accessory device 608, which has not detected the user input 650e, maintains the output of the light 616z and does not output the audio 622k. Thus, the first speaker accessory device 608 and / or the second speaker accessory device 610 can provide a visual indication associated with the current function and / or operation being performed by the first speaker accessory device 608 and / or the second speaker accessory device 610.

[0258] 7A to 7C 700 is a flowchart illustrating a method for outputting light using a computer system according to some embodiments. Method 700 is performed at a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) (e.g., an electronic device; a smart device such as a smart phone, a smart watch, and / or a smart speaker; a mobile device; a wearable device; and / or a group of devices that communicate with each other (e.g., two or more smart speakers that operate independently of each other or operate together as a stereo pair)), and the computer system communicates with one or more light sources (e.g., a light device (e.g., integrated into or connected to the computer system) and / or a light accessory such as a light bulb and / or a light emitting diode ("LED")). In some embodiments, the computer system communicates with a display generation component (e.g., a display controller, a touch-sensitive display system, a projector, a display screen, a display monitor, and / or a holographic display) via a wireless connection (e.g., Bluetooth, Wi-Fi, and / or another Internet connection). In some embodiments, the computer system communicates with the display generation component via an intermediate device (such as a server). In some embodiments, one or more light sources are included in and / or attached to a housing of a computer system. In some embodiments, one or more light sources communicate wirelessly (e.g., Bluetooth, Wi-Fi, and / or an Internet connection) and / or wired with the computer system. Some operations in method 700 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0259] As described below, method 700 provides an intuitive way to output light. The method reduces the cognitive burden on the user to output light, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to output light faster and more efficiently saves power and increases the time interval between battery charges.

[0260] Although the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) is configured to provide output (e.g., 622a-622j) (e.g., audio output) (in some embodiments, other than light output) associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802) (e.g., the computer system communicates with one or more devices (e.g., speakers) and The computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) receives (702) content (e.g., 614, 618, 620, 624) related to the content displayed (e.g., currently displayed) on the display generating component (e.g., 602 and / or 802), but the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) receives (702) content (e.g., 614, 618, 620, 624) related to the content displayed (e.g., currently displayed) on the display generating component (e.g., 602 and / or 802). , 626, 630, 634, 636, 638 and / or 640) (e.g., information about images, symbols, icons, graphics, logos, objects, teams, and / or other visual elements displayed on the display generating component, such as colors associated with (and / or included in) the images, symbols, icons, graphics, logos, objects, teams, and / or other visual elements; information about the type of content, such as movies, television shows, songs, playlists, albums, artists, sporting events, podcasts, and / or other types of multimedia; information about the timing of the content (e.g., what images are displayed at what playback time of the content); information about the status of the content (e.g., the current playback time, the team that won the sporting event, an event that occurred (such as a player or team score), the current volume of the audio output, and / or whether the content is currently playing or paused); and / or information about characteristics of the content, such as a network and / or brand associated with the content; the album, artist, playlist, and / or track titles of the songs of the content; and / or general colors associated with the content (e.g., colors that are generally associated with the content and are not based on the image currently displayed on the display generating component)).

[0261] In response to receiving information associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display (e.g., 602 and / or 802), a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) generates a display image according to the received information associated with the content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display generating component (e.g., 602 and / or 802). , 630, 634, 636, 638, and / or 640) (e.g., based on the received information) outputs (704) light (e.g., 616a-616ab) via one or more light sources (e.g., causing light to be emitted via one or more light sources in communication with the computer system) (e.g., one or more characteristics of the light (e.g., color, intensity, brightness, color temperature, contrast, hardness, and / or direction) are based on information about content displayed on the display generating component such that the light is perceived as part of an experience associated with the content). In some embodiments, the light (e.g., 616a-616ab) includes one or more colors based on the received information about content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802). In some embodiments, as playback of content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) progresses, the light (e.g., 616a-616ab) changes color, intensity, brightness, color temperature, contrast, hardness, and / or direction over time.

[0262] Outputting light in accordance with information received in association with content displayed on the display generating component improves the user's experience of viewing the content without requiring additional user input, thereby reducing the number of inputs required to perform operations.

[0263] In some embodiments, light (e.g., 616a-616ab) (e.g., visible light emitted from one or more light sources) includes a first color (e.g., a first group of one or more of the one or more light sources outputs a first portion of light having a first color based on information received associated with content displayed on the display generating component) and a second color different from the first color (e.g., a second group of one or more of the one or more light sources outputs a second portion of light having a second color different from the first color based on information received associated with content displayed on the display generating component). Outputting light including the first color and the second color different from the first color improves the user's experience of viewing content without requiring additional user input, thereby reducing the number of inputs required to perform operations.

[0264] In some embodiments, a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs light (e.g., 616a-616ab) via one or more light sources based on information received and associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display generation component (e.g., 602 and / or 802). The computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs light (e.g., 616a-616ab) based on information received and associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display generation component (e.g., 602 and / or 802). 8, 620, 624, 626, 630, 634, 636, 638, and / or 640) based on timing information associated with information (e.g., what image is displayed at what playback time of the content and / or the state of the content (e.g., the current playback time, the team that won the sporting event, the event that occurred (such as a score of a player or team)) to adjust (e.g., change an amount, transition between different amounts and / or states, and / or otherwise modify) one or more properties (e.g., color, intensity, brightness, color temperature, contrast, hardness, and / or direction) of the light (e.g., 616a-616ab) over time (e.g., while playing back the content) (e.g., the computer system adjusts the one or more properties of the light based on content being updated and / or played back over time). Adjusting the one or more properties of the light over time based on the timing information improves the experience of a user viewing the content without requiring additional user input, thereby reducing the number of inputs required to perform an operation.

[0265] In some embodiments, the light (e.g., 616a-616ab) includes a third color based on a fourth color of at least a portion of the content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display generating component (e.g., 602 and / or 802) (e.g., a third group of one or more of the one or more light sources outputs light having a third color based on a fourth color of at least a portion of the content displayed on the display generating component) (e.g., the third color matches and / or corresponds to at least one color associated with the content (e.g., a color currently displayed on the display generating component) and / or the third color is based on a group of one or more colors associated with the content (e.g., one or more colors currently displayed on the display generating component)). The light including a third color based on a fourth color of at least a portion of the content displayed on the display generating component improves the user's experience of viewing the content without requiring additional user input, thereby reducing the number of inputs required to perform operations.

[0266] In some embodiments, the content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802) includes a sporting event (e.g., a sporting event associated with the first sporting event 626 and / or the second sporting event 630) (e.g., a game, match, competition, and / or contest involving individuals and / or teams competing against each other, such as a basketball game, a football game, a soccer game, a soccer game, a soccer match ... sports event) displayed on the display generating component (e.g., The received information associated with the content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component includes the status (e.g., the score indicated by the scoreboard 628 and / or the event indicated by the event indicator 632) of the sporting event (e.g., the current score, the most recent goal and / or score, the leading and / or winning athlete and / or team, and / or the number of games that have been won) of the sporting event (e.g., the sporting event associated with the first sporting event 626 and / or the second sporting event 630). The light (e.g., 616a-616ab) includes one or more attributes based on the status (e.g., score indicated by scoreboard 628 and / or event indicated by event indicator 632) of the sporting event (e.g., a sporting event associated with first sporting event 626 and / or second sporting event 630) (e.g., the light includes one or more colors indicating a leading athlete and / or team, an athlete and / or team that has recently scored a goal and / or a point, an athlete and / or team that has won, and / or an athlete and / or team that has gained an advantage (e.g., free throws and / or outnumbered)). The light including one or more attributes based on the status of the sporting event provides the user with the ability to quickly and easily identify the current status and / or condition of the sporting event, thereby providing improved visual feedback.

[0267] In some embodiments, the content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generation component (e.g., 602 and / or 802) includes a trailer page (e.g., 614 and / or 618) (e.g., a preview user interface including one or more images, text, symbols, videos, and / or icons that provide information about a media file that can be played back and / or output from the trailer page). In some embodiments, the trailer page includes a playback user interface object, such as a selectable user interface object and / or an affordance that, when selected, causes a second computer system (e.g., the computer system or a different computer system) to initiate playback and / or output of a media file associated with the trailer page and / or content (e.g., causes the computer system to display a second content associated with the media file and / or trailer page). Content including trailer pages improves the user experience of viewing content without requiring additional user input, thereby reducing the amount of input required to perform an action.

[0268] In some embodiments, when a trailer page (706) (e.g., 614 and / or 618) is displayed, the trailer page (e.g., 614 and / or 618) includes a playback user interface object (e.g., 618a), and based on determining that user input (e.g., 650a) corresponding to the playback user interface object (e.g., 618a) has been received (e.g., user input requesting selection of a playback user interface object and / or initiation and / or output of a media file associated with the trailer page), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (708) light (e.g., 616a-616ab) via one or more light sources as dynamic light (e.g., one or more properties of the light change over time based on changes and / or updates to second content displayed in response to receiving the user input corresponding to the playback user interface object). While displaying a trailer page (706) (e.g., 614 and / or 618), the trailer page (e.g., 614 and / or 618) includes a playback user interface object (e.g., 618a), and based on determining that a user input (e.g., 650a) corresponding to the playback user interface object (e.g., 618a) has not been received (e.g., the computer system and / or the second computer system has not detected a user input requesting selection of a playback user interface object and / or initiation and / or output of a media file associated with the trailer page), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (710) light (e.g., 616a-616ab) via one or more light sources as static light (e.g., one or more properties of the light do not change over time, but are maintained while the trailer page with the playback user interface object is displayed). In some embodiments, when the computer system receives an indication that user input corresponding to the playback user interface object is received (e.g., received by the computer system and / or a second computer system), second content associated with the media file is displayed via the display generation component, wherein the second content includes dynamic content that changes over time, such as a video. In some embodiments, when the computer system does not receive an indication that user input corresponding to the playback user interface object is received, display of the trailer page with the playback user interface object is maintained.

[0269] Outputting light as dynamic light when user input corresponding to a playback user interface object is received and outputting light as static light when user input corresponding to a playback user interface object is not received provides confirmation to the user as to whether the user input is received, thereby providing improved visual feedback.

[0270] In some embodiments, before receiving an indication of a user input (e.g., 650a) corresponding to a playback user interface object (e.g., 618a) (e.g., before the computer system detects and / or receives an indication that another computer system has detected a user input selecting a playback user interface object), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (712) via one or more light sources a light having a first brightness (e.g., Figure 6C In response to receiving an indication of a user input (e.g., 650a) corresponding to a playback user interface object (e.g., 618a) (e.g., in response to the computer system detecting and / or in response to the computer system receiving an indication that another computer system detected user input selecting a playback user interface object), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (714) via the one or more light sources a light having a brightness greater than the first brightness (e.g., Figure 6C The brightness shown at the second brightness (for example, Fig.6D and / or Fig. 6E Outputting light at a first brightness before receiving an indication of a user input corresponding to a playback user interface object and outputting light at a second brightness in response to receiving an indication of a user input corresponding to the playback user interface object provides confirmation to the user as to whether the user input was received, thereby providing improved visual feedback.

[0271] In some embodiments, the content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802) includes content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) from currently playing media files (e.g., a current and / or active movie being played back, displayed, and / or output, album art corresponding to a current and / or active audio output (e.g., a song, podcast, audiobook, and / or streaming audio), a current and / or active sporting event being played back, displayed, and / or output, and / or a current and / or active television program being played back, displayed, and / or output). In some embodiments, the currently playing media file does not include a paused and / or suspended media file that includes a static image corresponding to a portion of the media file displayed on the display generation component (e.g., 602 and / or 802). The inclusion of content from the currently playing media file improves the user's experience of viewing the content without requiring additional user input, thereby reducing the number of inputs required to perform operations.

[0272] In some embodiments, at a first playback time of a currently playing media file (e.g., a first time within a duration associated with the media file and / or a first time measured from a start point and / or end point of the media file), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (716) via one or more light sources a signal having one or more first properties (e.g., Fig.6D At a second playback time of a currently playing media file that is different from the first playback time (e.g., a second time within a duration associated with the media file and / or a second time measured from a start point and / or end point of the media file), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (718) via one or more light sources a light having one or more second properties (e.g., a first color, intensity, brightness, color temperature, contrast, hardness, and / or direction) different from the one or more first properties. Fig. 6EOutputting light having one or more first attributes at a first playback time and outputting light having one or more second attributes different from the first attributes at a second playback time different from the first playback time improves the user's experience of viewing content without the need for additional user input, thereby reducing the number of inputs required to perform operations.

[0273] In some embodiments, the output (e.g., 622a-622j) associated with the currently playing media file (e.g., album and / or playlist) (e.g., audio content associated with the currently playing media file) includes audio output (e.g., 622a-622j) associated with a first song (e.g., audio 622g and / or audio 622h) (e.g., the first track and / or first part of the album and / or playlist) and a second song (e.g., audio 622i and / or 622j) different from the first song (e.g., the second track and / or second part of the album and / or playlist). When the audio output (e.g., 622a-622j) corresponding to the first song (e.g., audio 622g and / or 622h) is output, and when the audio having one or more third attributes (e.g., Figure 6L When the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs light (e.g., 616a-616ab) having one or more third properties (e.g., intensity, brightness, color temperature, contrast, hardness, and / or direction) shown at the first light source, the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) transitions (720) to outputting audio output (e.g., 622a-622j) corresponding to the second song (e.g., audio 622i and / or audio 622j). While transitioning to outputting audio output (e.g., 622a-622j) corresponding to the second song (e.g., audio 622i and / or audio 622j), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (722) having one or more third properties (e.g., intensity, brightness, color temperature, contrast, hardness, and / or direction) different from the one or more third properties (e.g., Figure 6L One or more fourth properties (e.g., properties of light 616s and / or light 616t as shown in FIG. Figure 6K The properties of light 616u and / or light 616v shown at (e.g., a fourth color, intensity, brightness, color temperature, contrast, hardness and / or direction) of light (e.g., 616a-616ab).

[0274] Outputting light having one or more fourth attributes different from one or more third attributes while transitioning to outputting audio output corresponding to the second song improves the user's experience of viewing content without requiring additional user input, thereby reducing the number of inputs required to perform operations.

[0275] In some embodiments, a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) includes a smart speaker (e.g., 608 and / or 610) (e.g., an audio output device configured to communicate with one or more external devices and / or the Internet such that the audio output device can perform functions other than outputting audio (e.g., via voice commands and / or other user input)). In response to receiving voice input (e.g., 650e) (e.g., speech recognized via voice recognition (e.g., speech including one or more keywords and / or key phrases)) via one or more input devices (e.g., one or more audio detection devices, such as one or more microphones) in communication with a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) , the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs (724) a second light (e.g., 616ab) different from the light (e.g., 616a-616ab) (e.g., the light includes one or more first attributes different from one or more second attributes of the second light, such as a different color) via one or more light sources. Outputting a second light different from the light in response to receiving the voice input allows a user to visually confirm that the computer system recognizes, receives, and / or detects the voice input, thereby improving visual feedback.

[0276] In some embodiments, when information associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display generating component (e.g., 602 and / or 802) is received, the display generating component (e.g., 602 and / or 802) is displayed at a third brightness (e.g., Figure 6KWhen light (e.g., 616a-616ab) is outputted at a level greater than or equal to the brightness of light 616u and / or light 616v shown at the light source) (e.g., a third lumen output and / or a third wattage and / or power amount supplied to one or more light sources), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) receives (726) an indication of an increase in volume (in some embodiments, an input requesting an increase in volume (e.g., 650c)) of audio (e.g., 622a-622j) associated with the output (e.g., 622a-622j). In response to receiving an indication of an increase in volume of audio (e.g., 622a-622j) associated with an output (e.g., 622a-622j), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) generates, via the one or more light sources, an audio signal at a brightness greater than a third brightness (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802) based on the received information associated with the content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802). Figure 6K A fourth brightness (eg, the brightness of light 616u and / or light 616v shown at Figure 6L The computer system may output (728) light (e.g., 616a-616ab) at a fourth brightness greater than the third brightness in response to receiving an indication of an increase in the volume of the audio associated with the output) (e.g., a fourth lumen output amount and / or a fourth wattage and / or power amount supplied to one or more light sources) (e.g., the computer system outputs light at an increased brightness when the computer system outputs audio associated with the output at an increased volume). Outputting light at a fourth brightness greater than the third brightness in response to receiving an indication of an increase in the volume of the audio associated with the output allows the user to visually confirm that the computer system has increased the volume in response to receiving the user input, thereby improving visual feedback.

[0277] In some embodiments, a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) communicates with a second computer system (e.g., 608 and / or 610) (e.g., an electronic device; a smart device, such as a smart phone, smart watch, and / or smart speaker; a mobile device; a wearable device; and / or a group of devices that communicate with each other (e.g., two or more smart speakers that operate independently of each other or together as a stereo pair)), which is of the same type as the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) (e.g., the same type of device and / or computer system, such as a smart speaker). The computer system communicating with a second computer system of the same type as the computer system improves the user's experience of viewing content without the need for additional user input, thereby reducing the number of inputs required to perform operations.

[0278] In some embodiments, the light (e.g., 616a-616ab) includes one or more fifth properties (e.g., properties of light 616i) (e.g., fifth color, intensity, brightness, color temperature, contrast, hardness, and / or direction) that are different from one or more sixth properties (e.g., properties of light 616j) (e.g., fourth color, intensity, brightness, color temperature, contrast, hardness, and / or direction) of a third light (e.g., 616a-616ab) output by a second computer system (e.g., 608 and / or 610) (e.g., the second computer system is configured to output a third light via one or more second light sources based on information received associated with content displayed on the display generating component, and the third light is different from the light). The light including one or more fifth properties that are different from the one or more sixth properties of the third light output by the second computer system improves the user's experience of viewing content without the need for additional user input, thereby reducing the number of inputs required to perform operations.

[0279] In some embodiments, the light (e.g., 616a-616ab) corresponds to a first portion (e.g., 618d, 618e, 618f, 618g, 624a, and / or 624b) of the received information associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802) (e.g., a first portion of the display area of ​​the display generating component, a first portion of the color scheme and / or arrangement of the content, and / or a first portion of an image, video, symbol, and / or icon of the content), and the second computer system (e.g., 608 and / or 610) is configured to output a fourth light (e.g., 616a-616ab) (e.g., the second computer system The system causes light to be emitted via one or more second light sources in communication with the second computer system), the fourth light corresponding to a second portion (e.g., 618d, 618e, 618f, 618g, 624a, and / or 624b) of the received information associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on the display generating component (e.g., 602 and / or 802) (e.g., a second portion of the display area of ​​the display generating component, a second portion of the color scheme and / or arrangement of the content, and / or a second portion of the image, video, symbol, and / or icon of the content), the second portion being different from the first portion (e.g., 618d, 618e, 618f, 618g, 624a, and / or 624b). The light output by the computer system corresponds to a first portion of the received information associated with the content displayed on the display generating component, and the fourth light output by the second computer system corresponds to a second portion of the received information associated with the content displayed on the display generating component, improving the user experience of viewing content without the need for additional user input, thereby reducing the number of inputs required to perform operations.

[0280] In some embodiments, the light (e.g., 616a-616ab) includes a fifth color (e.g., the color of light 616c and / or 616e) (e.g., a fifth group of one or more of the one or more light sources outputs light having the fifth color) and a sixth color (e.g., the color of light 616d and / or 616f) that is different from the fifth color (e.g., the color of light 616c and / or 616e) (e.g., a sixth group of one or more of the one or more light sources outputs light having the sixth color), and the second computer system (e.g., 608 and / or 610) is configured to output the fifth light (e.g., 616a-616ab) (e.g., the second computer system causes the light to be emitted via one or more second light sources in communication with the second computer system), the sixth color The fifth light includes a seventh color (e.g., the color of light 616c and / or 616e) (e.g., a seventh group of one or more light sources among the one or more light sources outputs light having the seventh color) and an eighth color (e.g., the color of light 616d and / or 616f) different from the seventh color (e.g., the color of light 616c and / or 616e) (e.g., an eighth group of one or more light sources among the one or more light sources outputs light having the eighth color). The light output by the computer system includes a fifth color and a sixth color different from the fifth color, and the fifth light output by the second computer system includes a seventh color and an eighth color different from the seventh color improves the user's experience of viewing content without requiring additional user input, thereby reducing the number of inputs required to perform operations.

[0281] In some embodiments, outputting light (e.g., 616a-616ab) via one or more light sources based on received information associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display generating component (e.g., 602 and / or 802) includes outputting light based on determining that a predetermined function (e.g., one or more selectable options for displaying a settings menu and / or display control output and / or output of content) is available (e.g., one or more conditions that enable the predetermined function to be performed by the computer system and / or a different computer system are satisfied (e.g., one or more indications of detected user inputs are received, an indication of a notification and / or event is received). In some embodiments, the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs light (e.g., 616a-616ab) having one or more seventh attributes (e.g., attributes of light 616y) (e.g., a seventh color, intensity, brightness, color temperature, contrast, hardness, and / or direction) (in some embodiments, one or more seventh attributes guide and / or prompt a user to provide user input that causes a predetermined function to be performed) (in some embodiments, the light having one or more seventh attributes is output in a direction, color, and / or brightness that signals and / or provides guidance to the user that user input will cause a predetermined function to be performed). Outputting light (e.g., 616a-616ab) via one or more light sources based on received information associated with content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) displayed on a display generating component (e.g., 602 and / or 802) includes outputting light based on determining that a predetermined function is not available (e.g., one or more conditions that enable the predetermined function to be performed by the computer system and / or a different computer system are not satisfied (e.g., one or more detected user requests are not received) based on determining that a predetermined function is not available (e.g., one or more conditions that enable the predetermined function to be performed by the computer system and / or a different computer system are not satisfied) based on determining that one or more detected user requests are not received In response to an indication of user input, an indication that a notification and / or event was not received, and / or an indication that a detected external device was not received), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) outputs light (e.g., 616a-616ab) having one or more eighth properties (e.g., properties of light 616x) (e.g., an eighth color, intensity, brightness, color temperature, contrast, hardness, and / or direction) that are different from one or more seventh properties (e.g., properties of light 616y).

[0282] Outputting light having one or more seventh attributes based on determining that a predetermined function is available, and outputting light having one or more eighth attributes different from the one or more seventh attributes based on determining that the predetermined function is not available allows the user to quickly provide user input to cause the third computer system to perform the predetermined function, thereby improving visual feedback.

[0283] It should be noted that the above reference method 700 (eg, 7A to 7C ) also apply in a similar manner to the methods described below. For example, method 900 optionally includes one or more of the characteristics of the various methods described above with reference to method 700. For example, a computer system executing method 900 may group the computer system executing method 900 with other accessory devices. For the sake of brevity, these details are not repeated below.

[0284] Figures 8A to 8V Examples of techniques for managing controllable accessories according to some embodiments are illustrated. The user interfaces in these figures are used to illustrate the following description including 9A to 9D The process of the process.

[0285] In some embodiments, any of the inputs described herein (e.g., inputs 850a, 850b, 850c, 850d, 850e, 850f, 850g, 850h, 850i, 850j, 850k, 850l, 850m, 850n, 850o, 850p, 850q, 850r, and / or 850s) is or includes touch input (e.g., a tap gesture and / or a swipe gesture). In some embodiments, any of the inputs described herein (e.g., inputs 850a, 850b, 850c, 850d, 850e, 850f, 850g, 850h, 850i, 850j, 850k, 850l, 850m, 850n, 850o, 850p, 850q, 850r, and / or 850s) is or includes voice input (e.g., voice commands for selecting a user interface element or for activating a feature or performing a function, such as a feature or function associated with a user interface element). In some embodiments, any of the inputs described herein (e.g., inputs 850a, 850b, 850c, 850d, 850e, 850f, 850g, 850h, 850i, 850j, 850k, 850l, 850m, 850n, 850o, 850p, 850q, 850r, and / or 850s) is or includes a gesture (e.g., a hand gesture, a facial gesture, and / or an air gesture for selecting a user interface element or for activating a feature or performing a function, such as a feature or function associated with a user interface element) and / or is or includes a voice command. In some embodiments, any of the inputs described herein (e.g., inputs 850a, 850b, 850c, 850d, 850e, 850f, 850g, 850h, 850i, 850j, 850k, 850l, 850m, 850n, 850o, 850p, 850q, 850r, and / or 850s) is or includes activation (e.g., pressing, rotating, and / or moving) of a hardware device (e.g., a button, a rotatable input mechanism, a rotatable depressible input mechanism, a mouse button, a remote control button, and / or a joystick). In some embodiments, any of the user interface elements described herein as being selected (e.g., an icon, an affordance, a button, and / or a selectable option) is selected by activating a hardware device while the user interface element is in focus (e.g., highlighted, bolded, outlined, visually distinguished from other user interface elements, and / or located at or near a cursor).

[0286] Fig. 8AThe electronic device 800 is illustrated as displaying a home user interface 804 associated with a home automation system via a display 802. In some embodiments, the home automation system includes one or more accessory devices (e.g., devices configured to communicate with and / or be controlled by the electronic device 800) associated with a location (such as a building, a home, an office, and / or an apartment). Fig. 8A At , the electronic device 800 is configured to adjust and / or control the state and / or settings of one or more accessory devices of the home automation system via the home user interface 804. In some embodiments, the home user interface 804 is a user interface that is first displayed by the electronic device 800 in response to launching an application associated with the home automation system.

[0287] exist Fig. 8A At , the home user interface 804 includes a home indicator 804a (e.g., "123 Main Street") that provides an indication of a location (e.g., a physical address) associated with the home automation system. In addition, the home user interface 804 includes a category area 806, a camera area 808, a scene area 810, and a first room area 812. In some embodiments, the home user interface 804 is scrollable so that one or more additional areas corresponding to corresponding rooms, scenes, types or categories of accessory devices, and / or designated accessory devices (e.g., favorites) of the home automation system can be displayed in response to a swipe and / or scroll gesture on the home user interface 804.

[0288] In response to receiving an indication of an event, electronic device 800 is configured to display a designated area (e.g., a disk) on home user interface 804 that enables electronic device 800 to group accessory devices and / or associate accessory devices with each other. Displaying the designated area facilitates the user's ability to group accessory devices and / or associate accessory devices with each other by reducing the number of inputs required to create an accessory group. For example, by displaying the designated area on home user interface 804, electronic device 800 enables the user to group accessory devices and / or associate accessory devices with each other without having to navigate to an additional user interface to search for a particular accessory device. FIG. 8A to FIG. 8Q The electronic device 800 is illustrated as displaying a designated area (e.g., designated area 814) in response to receiving an indication that an accessory device is to initiate playback of a media item. FIG. 8A to FIG. 8Q At , electronic device 800 enables accessory devices to be grouped and / or associated with each other so that the grouped accessory devices output content associated with the media item in conjunction with each other. FIG. 8A to FIG. 8QThe electronic device 800 is related to displaying the designated area in response to receiving an indication that an accessory device initiates playback of a media item, but the electronic device 800 is also configured to display the designated area (e.g., designated area 842 and / or designated area 848) in response to receiving an indication and / or request to create a new scene and / or a new room of the home automation system, as described below with reference to Figure 8R to Figure 8V As explained.

[0289] exist Fig. 8A At the time, the electronic device 800 receives an indication from a home automation accessory device to initiate (e.g., start and / or begin) outputting and / or playing back content. In some embodiments, the electronic device 800 receives the indication from the accessory device (e.g., the electronic device 800 communicates with the accessory device). In some embodiments, the electronic device 800 receives the indication from a different electronic device (e.g., a server). In response to receiving an indication from a home automation system accessory device to initiate outputting and / or playing back content, the electronic device 800 displays a home user interface 804, such as Figure 8B as shown.

[0290] exist Figure 8B , the home user interface 804 includes a home indicator 804a, a designated area 814 (e.g., a disk), and an accessories area 816. In some embodiments, the home user interface 804 is scrollable so that the electronic device 800 displays the category area 806, the camera area 808, the scene area 810, and / or the first room area 812 in response to receiving user input. In some embodiments, in response to receiving an indication that an accessory device of the home automation system initiates output and / or playback of content, the electronic device 800 displays the designated area 814 and the accessories area 816 at a location above the category area 806, the camera area 808, the scene area 810, and / or the first room area 812. Thus, in some embodiments, the home user interface 804 is scrollable so that the electronic device 800 displays the category area 806, the camera area 808, the scene area 810, and / or the first room area 812. Fig. 8A In contrast, category area 806, camera area 808, scene area 810, and / or first room area 812 are still included and / or are part of home user interface 804, but are displayed at different locations and / or positions on home user interface 804. In some embodiments, electronic device 800 responds to receiving an indication that an accessory device initiates output and / or playback of content without displaying accessory area 816. In some embodiments, electronic device 800 responds to receiving an indication that an accessory device initiates output and / or playback of content and displays category area 806, camera area 808, scene area 810, and / or first room area 812 below designated area 814.

[0291] exist Figure 8B At , the designated area 814 includes a first user interface object 814a corresponding to an accessory device of the home automation system that initiates output and / or playback of content. Figure 8BAt , the first user interface object 814a corresponds to a television accessory device. In some embodiments, the television accessory device provides information to the electronic device 800 that the television accessory device has been turned on, activated, and / or has initiated output of content. In some embodiments, the electronic device 800 displays a designated area 814 including the first user interface object 814a based on receiving information from the television accessory device. The designated area 814 provides a visual indication to the user of the electronic device 800 that the accessory device of the home automation system is currently outputting content. The designated area 814 includes a content indicator 814b (e.g., "TV Show") that provides a visual indication of the content being output by the television accessory device. For example, at Figure 8B At 814b, the content indicator 814b indicates that the television accessory device is configured to and / or is outputting a television program. In some embodiments, the content indicator 814b includes information about the television program, such as an episode number, episode name, series name, season number, and / or air date of the television program. As described below, in response to receiving one or more user inputs requesting that one or more additional accessory devices be associated with content, the electronic device 800 can associate one or more additional accessory devices with the content being output by the television accessory device.

[0292] For example, the accessory area 816 of the home user interface 804 includes accessory user interface objects 816a-816i corresponding to corresponding accessory devices of the home automation system. The electronic device 800 is configured to control and / or adjust the settings of the accessory device in response to detecting a user input corresponding to the corresponding accessory user interface object 816a-816i. As described below, in some embodiments, when the accessory device corresponding to the corresponding accessory device user interface object 816a-816i is compatible with one or more accessory devices (e.g., a television accessory device) associated with the content and displayed in the designated area 814, the electronic device 800 can associate the accessory device with the content initiated by the television accessory device. In some embodiments, the electronic device 800 determines whether the accessory device is compatible with one or more accessory devices associated with the content based on the function of the accessory device corresponding to the corresponding accessory user interface object 816a-816i. For example, when the accessory device corresponding to the corresponding accessory user interface object 816a-816i is configured to perform a function and / or operation consistent with the current content being output and / or played back by the television accessory device, the electronic device 800 can associate the accessory device with the content.

[0293] In some embodiments, when the electronic device 800 associates the accessory device corresponding to the respective accessory user interface object 816-816i with content, the additional accessory device is configured to output second content based on the content currently being output by the television accessory device. For example, in some embodiments, the television accessory device is outputting a video (e.g., a television program), and when the electronic device 800 associates the speaker accessory device with the content being output by the television accessory device, the speaker accessory device outputs audio corresponding to the video.

[0294] exist Figure 8B At 800, the electronic device 800 detects a user input 850a (e.g., a tap gesture, a tap gesture including a duration exceeding a predefined duration, or other selection / navigation input) corresponding to a selection of the accessory user interface object 816h. In response to detecting the user input 850a, the electronic device 800 displays the accessory user interface 818, such as Figure 8C as shown.

[0295] exist Figure 8C 816h, the accessory user interface 818 includes control user interface objects 818a-818g that enable the electronic device 800 to adjust the operation and / or settings of the light accessory device associated with the accessory user interface object 816h. For example, the accessory user interface 818 enables control of the brightness, color temperature, and / or operating state (e.g., whether the light accessory device is turned on or off) of the light accessory device. Figure 8C At 800, the electronic device 800 detects a user input 850b (e.g., a tap gesture or other selection / navigation input) corresponding to the exit user interface object 818h of the accessory user interface 818. In response to detecting the user input 850b, the electronic device 800 displays (e.g., redisplays) the home user interface 804, such as Fig.8D as shown.

[0296] exist Fig.8D At 800, electronic device 800 detects user input 850c (e.g., a tap and / or press gesture or another selection / navigation input having a duration exceeding a predefined duration followed by movement) on accessory user interface object 816c requesting that a speaker accessory device (e.g., a kitchen speaker) corresponding to accessory user interface object 816c be associated with content being (e.g., configured to be) output and / or played back by a television accessory device. Fig.8DAt , accessory user interface object 816 c includes a first appearance (e.g., a first color, font, shading, emphasis, size, and / or shape, such as a substantially square and / or box shape). As described below, electronic device 800 is configured to change the appearance of accessory user interface object 816 c based on a determination as to whether the speaker accessory device may be associated with content being output by (e.g., configured to be output by) the television accessory device.

[0297] exist Fig.8D At , the electronic device 800 determines whether the speaker accessory device satisfies a set of criteria indicating whether the speaker accessory device may be associated with content being output and / or played back by the television accessory device. For example, as described above, the electronic device 800 determines that the speaker accessory device may be associated with content being output and / or played back by the television accessory device based on the functionality of the speaker accessory device. In some embodiments, the electronic device 800 determines that the speaker accessory device includes a functionality (such as a primary functionality) that includes outputting audio. In some embodiments, the electronic device 800 determines that the functionality of the speaker accessory device is compatible with the content being output and / or played back by the television accessory device. For example, the speaker accessory device may output audio associated with a television program that is configured to be output by the television accessory device and / or is currently being output by the television accessory device.

[0298] As described below, in some embodiments, when a corresponding accessory device associated with one of accessory user interface objects 816a-816i does not satisfy a set of one or more criteria, the electronic device 800 determines that the corresponding accessory device cannot be associated with the content being output and / or played back by the television accessory device. For example, in some embodiments, the corresponding accessory device does not include functionality (e.g., any functionality) that is compatible with content that is configured to be output by the television accessory device and / or is currently being output by the television accessory device. In some embodiments, when the electronic device 800 determines that the corresponding accessory device does not satisfy a set of one or more criteria, the electronic device 800 does not enable the corresponding accessory device to be associated with the content, as described below with reference to Figure 8H and Figure 8I As explained.

[0299] exist Fig.8D At 850c, electronic device 800 determines that the speaker accessory device may be associated with the content being output and / or played back by the television accessory device. Based on determining that the speaker accessory device may be associated with the content, electronic device 800 changes the appearance of accessory user interface object 816c after detecting user input 850c, such as Fig. 8E as shown.

[0300] exist Fig. 8EAt , accessory user interface object 816 c includes a second appearance (e.g., a second color, font, shading, emphasis, size, and / or shape, such as a substantially circular and / or rounded shape) that is different from the first appearance. The second appearance provides a visual indication to a user of electronic device 800 that the speaker accessory device corresponding to accessory user interface object 816 c may be associated with content. Fig. 8E At , while the user input 850c is maintained, the electronic device 800 detects a moving component 850d of the user input 850c. For example, the electronic device 800 detects a request to move the accessory user interface object 816c from the accessory area 816 toward the designated area 814. Fig. 8E At the location, the electronic device 800 displays the accessory user interface object 816c at a first position 820a in the accessory area 816, and in response to detecting a moving component 850d of the user input 850c, the electronic device 800 is configured to move the accessory user interface object 816c toward a second position 820b in the designated area 814.

[0301] exist Fig.8F At 814, electronic device 800 displays movement of accessory user interface object 816c toward designated area 814 in response to detecting movement component 850d of user input 850c. Fig.8F 814, accessory user interface object 816c is displayed at a third position 820c that is between first position 820a in accessory zone 816 and second position 820b in designated zone 814. Thus, electronic device 800 animates and / or moves accessory user interface object 816c in response to detecting movement component 850d of user input 850c (and in accordance with determining that the speaker accessory device corresponding to accessory user interface object 816c can be associated with content).

[0302] exist Figure 8G At the second location 820b in the designated area 814, the electronic device 800 displays the accessory user interface object 816c. In some embodiments, when the electronic device 800 displays the accessory user interface object 816c within the designated area 814 (e.g., at the second location 820b and / or another location), the electronic device 800 causes the speaker accessory device corresponding to the accessory user interface object 816c to output content (e.g., audio output) based on the content being output by the television accessory device. In other words, the accessory device that includes the corresponding accessory user interface object within the designated area 814 is configured to operate in conjunction with another accessory device such that the accessory devices output content based on the same media file (e.g., a television program). For example, at Figure 8GAt , the television accessory device causes a display (eg, a display different from display 802) to display one or more images associated with a television program, and the speaker accessory device to output audio output associated with the television program.

[0303] exist Figure 8G In some embodiments, the electronic device 800 displays the user interface object corresponding to the master device at a larger size within the designated area 814 than the non-master device. In some embodiments, the electronic device 800 determines that the television accessory device corresponding to the first user interface object 814a is the master device because the television accessory device initiated the output of the content. In some embodiments, the electronic device 800 determines that the television accessory device is the master device because the television accessory device is the first accessory device to include the corresponding user interface object in the designated area 814.

[0304] exist Figure 8G At 800, electronic device 800 displays suggestion indicators 822 next to accessory user interface objects 816e, 816g, 816h, and 816i in accessory area 816. Suggestion indicator 822 provides that one or more accessory devices are suggested to be used with the content (e.g., being used by Figure 8G 816a, 816b, and 816d-816i that satisfy a set of one or more criteria that enable the corresponding accessory devices to be associated with the content. In some embodiments, the electronic device 800 determines which accessory devices can be associated with the content before receiving user input selecting the corresponding accessory user interface object. In some embodiments, the electronic device 800 does not display a suggestion indicator 822 next to accessory user interface objects corresponding to accessory devices that the electronic device 800 determines do not satisfy the set of one or more criteria and / or cannot be associated with the content.

[0305] exist Figure 8GAt , electronic device 800 detects user input 850e, user input 850f, and user input 850g corresponding to accessory user interface object 816g, accessory user interface object 816h, and accessory user interface object 816i, respectively. User input 850e, user input 850f, and user input 850g each include a contact component (e.g., a tap and / or press gesture including a duration exceeding a predefined duration) and a movement component. The movement components of user input 850e, user input 850f, and user input 850g include movement from accessory area 816 toward designated area 814. Thus, user input 850e includes a request to associate a second speaker accessory device (e.g., a living room speaker) corresponding to accessory user interface object 816g with content, user input 850f includes a request to associate a first light accessory device (e.g., a living room lamp) corresponding to accessory user interface object 816h with content, and user input 850g includes a request to associate a second light accessory device (e.g., a kitchen lamp) corresponding to accessory user interface object 816i with content.

[0306] As described above, in some embodiments, the electronic device 800 determines that the second speaker accessory device, the first light accessory device, and the second light accessory device satisfy a set of one or more criteria before detecting user input 850e, 850f, and / or 850g. In some embodiments, in response to detecting user input 850e, user input 850f, and user input 850g, the electronic device 800 determines that the second speaker accessory device, the first light accessory device, and the second light accessory device satisfy a set of one or more criteria and / or can be associated with content. Based on determining that the second speaker accessory device, the first light accessory device, and the second light accessory device satisfy a set of one or more criteria and / or can be associated with content, the electronic device 800 displays accessory user interface objects 816g-816i in the designated area 814, such as Figure 8H In some embodiments, the electronic device 800 displays the movement of the accessory user interface objects 816g-816i from the accessory area 816 toward the designated area 814, as described above with reference to FIG. 8E to FIG. 8G As explained.

[0307] exist Figure 8HAt , the electronic device 800 displays a designated area 814 including a first user interface object 814a, an accessory user interface object 816c, and accessory user interface objects 816g-816i. When the electronic device 800 displays the accessory user interface objects 816g-816i in the designated area 814, the electronic device 800 causes the second speaker accessory device, the first light accessory device, and the second light accessory device to output content (e.g., audio and / or light output) that is combined with content output by the television accessory device and / or the speaker accessory device. In some embodiments, the second speaker accessory device outputs an audio output based on the audio of the television program. In some embodiments, the first light accessory device and / or the second light accessory device outputs light including one or more attributes (e.g., one or more colors, brightness, and / or contrast) that is based on one or more visual elements of the television program displayed on the display (e.g., the one or more visual elements that the television accessory device causes to be displayed on the display).

[0308] exist Figure 8H At , electronic device 800 detects user input 850h corresponding to accessory user interface object 816d in accessory area 816 (e.g., including a tap and / or press gesture or another selection / navigation input of a duration exceeding a predefined duration). As described above, in some embodiments, when electronic device 800 determines that the corresponding accessory device does not satisfy a set of one or more criteria, electronic device 800 does not associate the corresponding accessory device with the content. For example, at Figure 8H At , the electronic device 800 determines that the lock accessory device corresponding to the accessory user interface object 816d does not meet the set of one or more criteria and cannot be associated with the content. In some embodiments, the electronic device 800 determines that the lock accessory device does not include functionality (e.g., any functionality) that is compatible with outputting content (such as a television program). In some embodiments, the electronic device 800 determines that the lock accessory device includes functionality that enables a lock of a door to be changed between a locked position and an unlocked position. In some embodiments, the electronic device 800 determines that the lock accessory device does not include additional functionality and / or determines that the lock accessory device does not include any functionality that enables the lock accessory device to output content that is consistent with and / or compatible with a television program.

[0309] Based on determining that the lock accessory device does not satisfy the set of one or more criteria (and upon detecting user input 850h), the electronic device 800 displays an accessory user interface object 816d corresponding to the lock accessory device in appearance 824, such as Figure 8I As shown here. Figure 8I814 , appearance 824 includes a dimmed, blurred, and / or grayed appearance that indicates that the lock accessory device corresponding to accessory user interface object 816 d cannot be associated with the content (and / or cannot be moved toward designated area 814). Thus, when a user of electronic device 800 attempts to associate a non-compatible accessory device with content, electronic device 800 provides a visual indication that the non-compatible accessory device cannot be associated with the content. Figure 8T and Figure 8V As stated, in some embodiments, electronic device 800 can associate any accessory device with other accessory devices. In some embodiments, electronic device 800 uses a different set of one or more criteria that are not based on the functionality of the respective accessory device to determine whether the respective accessory device can be associated with content.

[0310] Electronic device 800 is configured to arrange user interface objects within designated area 814 to provide an indication of the configuration of corresponding accessory devices relative to each other. Figure 8J At 826a, the electronic device 800 displays a designated area 814 having a first user interface object 814a at a first location 826a, an accessory user interface object 816c at a second location 826b, an accessory user interface object 816g at a third location 826c, an accessory user interface object 816h at a fourth location 826d, and an accessory user interface object 816i at a fifth location 826e. Figure 8J , accessory user interface object 816c is at a second location 826b, which is to the right of first user interface object 814a, and accessory user interface object 816g is at a third location 826c, which is to the left of first user interface object 814a. As described above, accessory user interface object 816c corresponds to a speaker accessory device, and accessory user interface object 816g corresponds to a second speaker accessory device. In some embodiments, the electronic device 800 causes the speaker accessory device to output a right channel of audio output corresponding to the audio of the television program, and causes the second speaker accessory device to output a left channel of audio output corresponding to the audio of the television program. In some embodiments, the electronic device 800 is configured to adjust the configuration of the speaker accessory device and the second speaker accessory device (e.g., which speaker accessory device outputs the left channel of audio output, and which speaker accessory device outputs the right channel of audio output) in response to user input.

[0311] For example, in Figure 8JAt 816a, electronic device 800 detects user input 850i (e.g., a swipe gesture, a slide gesture, a drag gesture, or another selection / navigation input) corresponding to accessory user interface object 816g. In response to detecting user input 850i, electronic device 800 switches the respective positioning of accessory user interface object 816g and accessory user interface object 816c in designated area 814, such as Figure 8K as shown.

[0312] exist Figure 8K At the designated area 814, the electronic device 800 displays the accessory user interface object 816c at the third position 826c, and displays the accessory user interface object 816g at the second position 826b. In some embodiments, when the electronic device 800 displays the accessory user interface object 816c at the third position 826c, the speaker accessory device is configured to output the left channel of the audio output associated with the television program. Similarly, in some embodiments, when the electronic device 800 displays the accessory user interface object 816g at the second position 826b, the second speaker accessory device is configured to output the right channel of the audio output associated with the television program. Therefore, the electronic device 800 can change and / or modify the configuration of the accessory device associated with the content in response to detecting a user input requesting to move the corresponding user interface object within the designated area 814. In some embodiments, the electronic device 800 can cause the first light accessory device and the second light accessory device to output light with different properties based on the corresponding positions of the accessory user interface object 816h and the accessory user interface object 816i within the designated area 814.

[0313] exist Figure 8K At , the electronic device 800 receives an indication that the output of content by the accessory device that includes the corresponding user interface object in the designated area 614 has been suspended, paused, and / or stopped. For example, in some embodiments, the electronic device 800 receives the indication from a television accessory device. In some embodiments, the electronic device 800 receives the indication from a different accessory device and / or an external device (e.g., a server). In response to receiving the indication that the output of content has been suspended, paused, and / or stopped, the electronic device 800 stops displaying the designated area 814 and / or the accessory area 816 and displays the home user interface 804, such as Figure 8L as shown.

[0314] exist Figure 8L At this time, the electronic device 800 does not display the designated area 814 on the home user interface 804 to visually indicate that the output of the content has been suspended, paused, and / or stopped. Therefore, the user of the electronic device 800 can quickly determine whether one or more accessory devices of the home automation system are outputting content based on whether the home user interface 804 includes the designated area 814. Figure 8LAt the same time, the electronic device 800 receives an indication that the output of the content has been resumed, initiated, and / or otherwise started. For example, in some embodiments, the electronic device 800 receives the indication from a television accessory device. In some embodiments, the electronic device 800 receives the indication from a different accessory device and / or an external device (e.g., a server). In response to receiving the indication that the output of the content has been resumed, initiated, and / or otherwise started, the electronic device 800 displays a home user interface 804 including a designated area 814 and an accessory area 816, such as Figure 8M as shown.

[0315] exist Figure 8M At the designated area 814, the first user interface object 814a, the accessory user interface object 816c and the accessory user interface objects 816g-816i are included. The electronic device 800 is in the designated area 814 and Figure 8K 816i are displayed at the same respective locations as shown in FIG. 814. Thus, even if the content is suspended, paused, and / or stopped, when the electronic device 800 receives an indication that the output of the content has been resumed, initiated, and / or otherwise started, the electronic device 800 maintains the respective locations of the first user interface object 814a, the accessory user interface object 816c, and the accessory user interface objects 816g-816i.

[0316] exist Figure 8N At 800, electronic device 800 detects user input 850j (e.g., comprising a tap and / or press gesture or another selection / navigation input of a duration exceeding a predefined duration) corresponding to accessory user interface object 816c in designated area 814. User input 850j includes a movement component 850k that requests to disassociate and / or remove an association of the speaker accessory device with the content. Figure 8N In response to detecting user input 850j including a movement component 850k, electronic device 800 displays an animation of accessory user interface object 816c moving from third location 826c in designated area 814 toward location 828 in accessory area 816, as shown in FIG. Fig.8O as shown.

[0317] exist Fig.8O At 816c, the electronic device 800 displays the accessory user interface object 816c at a position 830 between the third position 826c and the position 828. Fig.8OAt , as electronic device 800 continues to detect user input 850j that includes movement component 850k, electronic device 800 displays movement of accessory user interface object 816c toward location 828.

[0318] exist Figure 8P At position 828, electronic device 800 displays accessory user interface object 816c at position 828 in accessory area 816. When electronic device 800 displays accessory user interface object 816c in accessory area 816, electronic device 800 does not cause the speaker accessory device to output content (e.g., audio output) based on the content (such as a television program). In other words, at position 828, electronic device 800 displays accessory user interface object 816c in accessory area 816. Figure 8P At this time, the speaker accessory device is no longer associated with the content and therefore does not output audio based on the content (e.g., audio associated with a television program). Therefore, electronic device 800 can quickly associate and / or disassociate the corresponding accessory device with the content via user input.

[0319] exist Figure 8P At the designated area 814, the electronic device 800 detects a user input 8501 (eg, a tap gesture or other selection / navigation input) corresponding to the designated area 814. In response to detecting the user input 8501, the electronic device 800 displays the content user interface 832, such as Figure 8Q Additionally or alternatively, the electronic device 800 detects a user input 850m (e.g., a tap gesture or other selection / navigation input) corresponding to the add user interface object 804b of the home user interface 804. In response to detecting the user input 850m, the electronic device 800 displays a menu 834, such as Figure 8R as shown.

[0320] exist Figure 8Q At , content user interface 832 includes accessory user interface objects 832a-832d corresponding to respective accessory devices associated with the content. Figure 8Q At the same time, the electronic device 800 organizes and / or categorizes the accessory user interface objects 832a-832d based on the rooms and corresponding accessory devices. For example, in some embodiments, the corresponding accessory devices are associated with different rooms of the location associated with the home automation system (e.g., programmatically mapped to different rooms). Thus, the user of the electronic device 800 can quickly view and / or identify which accessory devices are associated with the content and / or where these accessory devices are located within the location.

[0321] In some embodiments, in response to detecting a user input corresponding to one or more of accessory user interface objects 832a-832d, electronic device 800 displays an accessory user interface, such as a user interface similar to accessory user interface 818. The accessory user interface enables electronic device 800 to adjust and / or modify one or more settings and / or operating states of a corresponding accessory device. Thus, a user can quickly access and / or control the accessory user interface of a corresponding accessory device associated with the content via content user interface 832.

[0322] exist Figure 8R At , menu 834 includes selectable options 834a-834f that enable electronic device 800 to add accessory devices, scenes, automations, rooms, authorized user accounts, and / or additional home automation systems. Figure 8R At 840, the electronic device 800 detects a user input 850n (e.g., a tap gesture or other selection / navigation input) corresponding to a selectable option 834b of the menu 834. In response to detecting the user input 850n, the electronic device 800 displays an add scene user interface 836, such as Figure 8S Additionally or alternatively, the electronic device 800 detects a user input 850o (e.g., a tap gesture or other selection / navigation input) corresponding to a selectable option 834d of the menu 834. In response to detecting the user input 850o, the electronic device 800 displays an add room user interface 838, such as Figure 8U as shown.

[0323] exist Figure 8S At, adding a scene user interface 836 enables the electronic device 800 to create a new scene for the home automation system. In some embodiments, the scene enables the electronic device 800 to control and / or adjust the state of one or more accessory devices of the home automation system via user input based on the location of the electronic device 800 and / or based on satisfying a condition (e.g., the current time of the day is a predetermined time associated with activating and / or deactivating a scene). In some embodiments, the scene is user-defined because the user selects which accessory devices of the home automation system are included in the scene and how to control the selected accessory devices when the scene is activated. In some embodiments, the scene is predetermined and / or suggested by the electronic device 800 based on user habits. In some embodiments, the electronic device 800 controls and / or adjusts the state of multiple accessory devices of the home automation system in response to a single user input selecting a scene user interface object (e.g., a user interface object displayed in the scene area 810 of the home user interface 804). Therefore, the scene allows the user to easily control the accessory devices and / or accessory device groups of the home automation system, and reduces the number of user inputs required to control and / or adjust the accessory devices and / or accessory device groups.

[0324] exist Figure 8S At 836, the add scene user interface 836 includes a scene name user interface object 836a and an add accessories user interface object 836b. Figure 8S At , the electronic device 800 detects user input 850p (e.g., a tap gesture or other selection / navigation input) corresponding to a selection of the scene name user interface object 836a. In some embodiments, in response to detecting the user input 850p, the electronic device 800 displays a keyboard (e.g., a virtual keyboard) that enables a user of the electronic device 800 to provide and / or enter a name for the new scene. In some embodiments, the electronic device 800 detects one or more user inputs associated with naming the new scene. Figure 8S At 836b, the electronic device 800 detects a user input 850q (e.g., a tap gesture or other selection / navigation input) corresponding to the add accessory user interface object 836b. In response to detecting the user input 850q, the electronic device 800 displays the accessory user interface 840, such as Figure 8T as shown.

[0325] exist Figure 8T At , the accessory user interface 840 includes a second designated area 842 and an accessory area 844. The accessory area 844 includes accessory user interface objects 844a-844i corresponding to corresponding accessory devices of the home automation system. In some embodiments, the accessory user interface 840 is scrollable so that the electronic device 800 is configured to display additional accessory user interface objects in the accessory area 844 in response to detecting user input (e.g., a swipe gesture or other selection / navigation input). The second designated area 842 corresponds to the accessory devices of the home automation system included in the new scene. Figure 8T At , the second designated area 842 is blank, and therefore no accessory device is added to the new scene. In some embodiments, the electronic device 800 is configured to add the corresponding accessory device to the new scene in response to detecting a user input corresponding to one or more of the accessory user interface objects 844a-844i of the accessory area 844.

[0326] The second designated area 842 and the accessory area 844 are configured as described above with reference to FIG. 8A to FIG. 8RThe described designated area 814 and accessory area 816 operate and / or function similarly. In some embodiments, the electronic device 800 is configured to add any corresponding accessory device to the new scene. In some embodiments, the electronic device 800 is configured to add the corresponding accessory device to the new scene when the corresponding accessory device meets a set of one or more second criteria. In some embodiments, the set of one or more second criteria includes determining whether the corresponding accessory device corresponds to the overall function, context and / or theme of the new scene, whether it is consistent with the overall function, context and / or theme of the new scene, and / or whether it can be configured with the overall function, context and / or theme of the new scene. For example, in some embodiments, the electronic device 800 determines that a garage door accessory device cannot be included in a new scene for entertainment and / or associated with the new scene.

[0327] exist Figure 8U At , add room user interface 838 enables electronic device 800 to create a new room for the home automation system. As described above, in some embodiments, electronic device 800 associates (e.g., programmatically maps) a corresponding accessory device with a room of a location of the home automation system. Associating an accessory device with a particular room of the location enables a user to quickly find various accessory devices and / or determine where a corresponding accessory device is located within a location of the home automation system.

[0328] exist Figure 8U At , the add room user interface 838 includes a room name user interface object 838a and a continue user interface object 838b. Figure 8U At , the electronic device 800 detects user input 850r (e.g., a tap gesture or other selection / navigation input) corresponding to a selection of the room name user interface object 838a. In some embodiments, in response to detecting the user input 850r, the electronic device 800 displays a keyboard (e.g., a virtual keyboard) that enables a user of the electronic device 800 to provide and / or enter a name for the new room. In some embodiments, the electronic device 800 detects one or more user inputs associated with naming the new room. Figure 8U At 850 s, the electronic device 800 detects a user input 850 s (eg, a tap gesture or other selection / navigation input) corresponding to the continue user interface object 838 b. In response to detecting the user input 850 s, the electronic device 800 displays the accessory user interface 846, such as Figure 8V as shown.

[0329] exist Figure 8V At 846, the accessory user interface 846 includes a third designated area 848 and an accessory area 852. The accessory area 852 includes accessory user interface objects 852a-852i corresponding to respective accessory devices of the home automation system. Figure 8VAt , accessory user interface objects 852a-852i correspond to respective accessory devices of the home automation system that are not yet associated with a room of the location of the home automation system. Figure 8V At the location, the accessory user interface objects 852a-852i do not include room designations and / or indicators because the corresponding accessory devices have not yet been associated with a room at the location. In some embodiments, the accessory area 852 includes accessory user interface objects corresponding to accessory devices that have been associated with a room at the location, thereby enabling a user of the electronic device 800 to change the room designation of the corresponding accessory devices. In some embodiments, the accessory user interface 846 is scrollable, such that the electronic device 800 is configured to display additional accessory user interface objects in the accessory area 852 in response to detecting user input (e.g., a swipe gesture or other selection / navigation input). The third designated area 848 corresponds to accessory devices of the home automation system that are included in the new room. Figure 8V , the third designated area 848 is blank, and therefore no accessory device is added to the new room. In some embodiments, the electronic device 800 is configured to add corresponding accessory devices to the new room in response to detecting user input corresponding to one or more of the accessory user interface objects 852a-852i of the accessory area 852.

[0330] The third designated area 848 and the accessory area 852 are configured as described above with reference to FIG. 8A to FIG. 8R The described designated area 814 and accessory area 816 operate and / or function similarly. In some embodiments, the electronic device 800 is configured to add any corresponding accessory devices to the new room that are not already associated with a room at the location. In some embodiments, the electronic device 800 is configured to add the corresponding accessory device to the new room when the corresponding accessory device meets a set of one or more third criteria. In some embodiments, the set of one or more third criteria includes determining whether the corresponding accessory device corresponds to a description and / or room name of the new room, and / or is consistent with the description and / or room name of the new room. For example, in some embodiments, the electronic device 800 determines that a garage door accessory device cannot be included in and / or associated with a new room named and / or designated as an office.

[0331] 9A to 9D900 is a flowchart illustrating a method for managing a controllable device using a computer system according to some embodiments. Method 900 is performed at a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) (e.g., an electronic device; a smart device, such as a smart phone or smart watch; a mobile device; and / or a wearable device), which communicates with one or more input devices and display generation components (e.g., 602 and / or 802) (e.g., a display controller, a touch-sensitive display system, a projector, a display screen, a display monitor, and / or a holographic display). Some operations in method 900 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0332] As described below, method 900 provides an intuitive way to manage controllable devices. The method reduces the cognitive burden on users to manage controllable devices, thereby creating a more efficient human-computer interface. For battery-powered computing devices, enabling users to manage controllable devices faster and more efficiently saves power and increases the time interval between battery charges.

[0333] When a user interface (e.g., 804) (e.g., a user interface for a home automation system, the user interface being configured to enable control of a home automation system (e.g., one or more devices and / or accessories of the home automation system)) including a user interface object (e.g., 814a) (e.g., an affordance displayed in a predefined area of ​​the user interface) is displayed via a display generating component (e.g., 602 and / or 802), the user interface object, when selected, provides an option for controlling a first remotely controllable external device (e.g., a remotely controllable external device associated with the first user interface object 814a) (e.g., a first accessory of the home automation system, such as a television, a light, a socket, and / or a speaker (e.g., a smart speaker)), wherein the first remotely controllable external device is associated with a context (e.g., currently playing content (e.g., music, movies, television programs, and / or media including images, videos, light data, and / or audio), recently played content, a scene (e.g., a set of defined settings for one or more remotely controllable accessories), one or more currently active (e.g., a remotely controllable external device associated with the first user interface object 814a) (e.g., a first accessory of the home automation system, such as a television, a light, a socket, and / or a speaker (e.g., a smart speaker)). The computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) detects (902) via one or more input devices that a second remotely controllable external device (e.g., a remotely controllable external device associated with one of the accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) (e.g., a home automation system) is associated with an accessory, and / or one or more recently controlled accessories, such as in an active state and / or otherwise. a request to associate a second accessory of the system, such as a television, a lamp, a socket, and / or a speaker (e.g., a smart speaker), that is different from the first remotely controllable external device, with the context (e.g., adding the second remotely controllable external device to a device group that includes the first remotely controllable external device, such that the first remotely controllable external device and the second remotely controllable external device are configured to function based on the context and / or output content (e.g., images, videos, lights, and / or audio) corresponding to the context (e.g., media files, videos, movies, songs, television shows, and / or podcasts);

[0334] In response to detecting a request to associate a second remotely controllable external device (e.g., a remotely controllable external device associated with one of accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) with the context (904) and based on determining that the second remotely controllable external device (e.g., a remotely controllable external device associated with one of accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) satisfies a set of one or more criteria (e.g., the second remotely controllable external device includes functionality that is compatible with the context, consistent with the context, and / or configurable with the context), causing the second remotely controllable external device (e.g., a remotely controllable external device associated with one of accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) to associate with the context In some embodiments, the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) associates a second remotely controllable external device (e.g., a remotely controllable external device associated with one of accessory user interface objects 816a-816i, 844a-44i, and / or 852a-852i) with the context (906) (e.g., associating a remotely controllable external device associated with accessory user interface object 816c with the context). FIG. 8D to FIG. 8G In some embodiments, the context includes a currently playing media file, and associating the second remotely controllable external device with the context enables the second remotely controllable external device to output content based on the media file (e.g., output audio, output visual elements via a display, and / or output lights). In some embodiments, the second remotely controllable external device acts based on the context and / or outputs content based on the context after being associated with the context. In some embodiments, associating the second remotely controllable external device with the context includes displaying (e.g., updating and / or changing) a user interface to indicate (e.g., displaying a visual indication on the user interface) that the second remotely controllable external device is associated with the context.

[0335] In response to detecting a request to associate a second remotely controllable external device (e.g., a remotely controllable external device associated with one of accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) with the context (904) and based on determining that the second remotely controllable external device (e.g., a remotely controllable external device associated with one of accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) does not satisfy a set of one or more criteria (e.g., the second remotely controllable external device does not include a functionality that is compatible with the context, consistent with the context, and / or configurable with the context (e.g., a primary functionality), such that the second remotely controllable external device cannot be associated with the context based on the context. 816a-816i, 844a-844i, and / or 852a-852i) includes (e.g., is configured to perform) a first function that does not correspond to (e.g., is incompatible with, inconsistent with, not configurable with, and / or does not match) a second function of the first remotely-controllable external device (e.g., a remotely-controllable external device associated with one of the first user interface objects 814a-816i). configured to perform an action, operation, and / or task, such as displaying content, causing content to be displayed on a display generating component, outputting audio content, outputting light, and / or outputting another sensory experience) (in some embodiments, the function is the primary function of the second remotely controllable external device, such as the function that the second remotely controllable external device is primarily configured to perform) (for example, the first function of the second remotely controllable external device is incompatible with, inconsistent with, and / or cannot be configured with the second function of the first remotely controllable external device, such that the second remotely controllable external device cannot be combined with and / or coordinated with the first remotely controllable external device to operate (for example, the first remotely controllable external device and the second remotely controllable external device cannot output content related to the same media file). Associated content)) (in some embodiments, when the first function is an action (such as locking a door) that is unrelated to the action of the second function (such as outputting audio and / or media content), the first function does not correspond to the second function) (in some embodiments, when the second remotely controllable external device includes at least one function that corresponds to, is compatible with, is consistent with, and / or can be configured with at least one function of the first remotely controllable external device, the second remotely controllable external device is associated with the context), the computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) abandons (908) associating the second remotely controllable external device with the context (e.g.,Discarding the remotely controllable external device associated with the accessory user interface object 816d and, Figure 8H and Figure 8I The second remotely controllable external device is not grouped with the first remotely controllable external device, so that the second remotely controllable external device does not function based on the context and / or output content based on the context.

[0336] Associating the second remotely controllable external device with the context when the second remotely controllable external device satisfies a set of one or more criteria, and abandoning associating the second remotely controllable external device with the context when the second remotely controllable external device does not satisfy the set of one or more criteria allows a user to easily group devices that are compatible with each other without having to navigate to another user interface, thereby reducing the number of inputs required to perform operations.

[0337] In some embodiments, the context includes content (e.g., 614, 618, 620, 624, 626, 630, 634, 636, 638, and / or 640) (e.g., images, videos, lights, and / or audio) currently being output by a first remotely controllable external device (e.g., a remotely controllable external device associated with the first user interface object 814a) (e.g., the first remotely controllable external device is actively displaying an image, displaying a video, outputting light, and / or outputting audio). The context including the content currently being output by the first remotely controllable external device allows the user to quickly cause another remotely controllable external device to coordinate with the first remotely controllable external device to output content without having to navigate to another user interface, thereby reducing the number of inputs required to perform an operation.

[0338] In some embodiments, the user interface (e.g., 804) includes an indication (e.g., 814b) of the content currently being output by the first remotely controllable external device (e.g., text, a symbol, an icon, an image, and / or another visual element that provides information about the content (such as a title, a track title, an artist name, an episode number, a season number, an album title, a podcast title, and / or a playlist name). The user interface including an indication of the content currently being output by the first remotely controllable external device allows the user to easily understand that associating the second remotely controllable device with the context enables the second remotely controllable external device to coordinate with the first remotely controllable external device to output content, thereby providing improved visual feedback.

[0339] In some embodiments, the context includes a first remotely controllable external device being associated with a location (e.g., Figure 8U and Figure 8VAs shown herein) (e.g., a first remotely controllable external device is programmatically mapped to a location, area, and / or zone (e.g., a physical location, area, and / or zone)) (in some embodiments, the location is an area and / or zone of a structure associated with an automation system (e.g., a home automation system) that is configured to be controlled by a computer system). The scenario includes associating a first remotely controllable external device with a location allowing a user to quickly associate another remotely controllable external device with the location without having to navigate to another user interface, thereby reducing the number of inputs required to perform an operation.

[0340] In some embodiments, the context includes a first remotely controllable external device being associated with a scene (e.g., Figure 8S and Figure 8T Associating a scenario with a first remotely controllable external device with a location allows a user to quickly associate another remotely controllable external device with the scenario without having to navigate to another user interface, thereby reducing the number of inputs required to perform an operation.

[0341] In some embodiments, a user interface (e.g., 804) includes a device area (e.g., 816, 844, and / or 852) (e.g., an area of ​​the user interface that is separate and / or visually distinct from a designated area that includes user interface objects). The device area (e.g., 816, 844 and / or 852) includes a first device user interface object (e.g., 816a-816i, 844a-844i and / or 852a-852i) that is different from the user interface object (e.g., 814a) (e.g., a first selectable user interface object and / or enable representation that enables control and / or adjustment of settings of a third remotely controllable external device), which first device user interface object corresponds to a third remotely controllable external device (e.g., a remotely controllable external device associated with corresponding accessory user interface objects 816-816i, 844a-844i and / or 852a-852i) of an automation system (e.g., a home automation system including one or more remotely controllable external devices configured to be controlled by a computer system) that includes the first remotely controllable external device (e.g., a third accessory of the home automation system that is different from the first remotely controllable external device, such as a television, a light, a socket and / or a speaker (e.g., a smart speaker)). The device area (e.g., 816, 844 and / or 852) includes a second device user interface object (e.g., 816a-816i, 844a-844i and / or 852a-852i) that is different from the user interface object (e.g., 814a) (e.g., a second selectable user interface object and / or enable representation that enables control and / or adjustment of settings of a fourth remotely controllable external device), which second device user interface object corresponds to a fourth remotely controllable external device of the automation system that includes the first remotely controllable external device (e.g., a remotely controllable external device associated with corresponding accessory user interface objects 816a-816i, 844a-844i and / or 852a-852i) (e.g., a fourth accessory of the home automation system that is different from the first remotely controllable external device, such as a television, a light, a socket and / or a speaker (e.g., a smart speaker)).

[0342] The user interface including a device area having a first device user interface object and a second device user interface object allows a user to easily select and / or choose which remotely controllable external devices are associated with a context without having to navigate to another user interface, thereby reducing the number of inputs required to perform operations.

[0343] In some embodiments, in response to detecting that a third remotely controllable external device (e.g., a remotely controllable external device associated with corresponding accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) is associated with a context (e.g., adding the third remotely controllable external device to a device group that includes the first remotely controllable external device, such that the first remotely controllable external device and the third remotely controllable external device are configured to function based on the context and / or output information consistent with the context (e.g., media The controller 100 receives a request (e.g., 850c, 850d, 850e, 850f, 850g, and / or 850h) for content (e.g., images, video, lighting, and / or audio) corresponding to a third remotely controllable external device (e.g., a remotely controllable external device associated with the corresponding accessory user interface object 816a-816i, 844a-844i, and / or 852a-852i) (910), and based on determining that the third remotely controllable external device (e.g., the remotely controllable external device associated with the corresponding accessory user interface object 816a-816i, 844a-844i, and / or 852a-852i) satisfies one or more of the above requirements. A computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) associates (912) a third remotely controllable external device (e.g., a remotely controllable external device associated with corresponding accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) with the context (e.g., grouping the third remotely controllable external device with the first remotely controllable external device such that the first remotely controllable external device is configured to function based on the context and / or output first content based on the context, and the third remotely controllable external device is configured to function based on the context and / or output second content based on the context).

[0344] Associating a third remotely controllable external device with the context in response to detecting a request to associate a third remotely controllable external device with the context and based on determining that the third remotely controllable external device satisfies a set of one or more criteria allows a user to easily select and / or pick which remotely controllable external devices to associate with the context without having to navigate to another user interface, thereby reducing the number of inputs required to perform the operation.

[0345] In some embodiments, based on determining that the context is the first context, the device area (e.g., 816, 844, and / or 852) includes a first suggested device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) (e.g., a selectable user interface object and / or an enable representation that enables control and / or adjustment of settings of a fifth remotely controllable external device), which first suggested device user interface object corresponds to the fifth remotely controllable external device associated with the first context (e.g., with the corresponding accessory user interface object 816a-816i). 816i, 844a-844i and / or 852a-852i associated) (e.g., a fifth accessory of the home automation system that is different from the first remotely controllable external device, such as a television, a light, a socket and / or a speaker (e.g., a smart speaker)) (e.g., the first suggested device user interface object includes an appearance and / or is displayed at a location that indicates to a user of the computer system that the fifth remotely controllable external device associated with the first suggested device user interface object satisfies a set of one or more criteria and / or is otherwise capable of being associated with the first situation). Based on determining that the context is a second context different from the first context, the device area (e.g., 816, 844, and / or 852) includes a second suggested device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) (e.g., a selectable user interface object and / or an affordance that enables controlling and / or adjusting settings of a sixth remotely-controllable external device) that corresponds to a remotely-controllable external device that is different from the fifth remotely-controllable external device (e.g., the remotely-controllable device associated with the corresponding accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) that is associated with the second context. external device) (e.g., a remotely controllable external device associated with corresponding accessory user interface objects 816a-816i, 844a-844i and / or 852a-852i) (e.g., a sixth accessory of the home automation system that is different from the first remotely controllable external device, such as a television, a light, a socket and / or a speaker (e.g., a smart speaker)) (e.g., the second suggested device user interface object includes an appearance and / or is displayed at a location that indicates to a user of the computer system that the sixth remotely controllable external device associated with the second suggested device user interface object satisfies a set of one or more criteria and / or is otherwise capable of being associated with the second context).

[0346] Based on determining that the context is a first context, the device area includes a first suggested device user interface object, and based on determining that the context is a second context different from the first context, the device area includes a second suggested device user interface object allowing the user to easily understand that a fifth remotely controllable external device can be associated with the context, thereby providing improved visual feedback.

[0347] In some embodiments, a first suggested device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) includes a suggestion indicator (e.g., 822) (e.g., text, symbol, icon, and / or image, such as a dot located next to and / or within the suggested device user interface object). In some embodiments, a device area (e.g., 816, 844, and / or 852) includes a device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i), and when the remotely controllable external device corresponding to the device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) is not a suggested device for associating with the context, the device user interface object does not include the suggestion indicator (e.g., 822). Including the suggestion indicator with the first suggestion device user interface object allows the user to easily understand that the fifth remotely controllable external device may be associated with the context, thereby providing improved visual feedback.

[0348] In some embodiments, the device area (e.g., 816, 844, and / or 852) includes a third device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) corresponding to a second remotely controllable external device (e.g., a remotely controllable external device associated with the corresponding accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) (e.g., a selectable user interface object and / or an enable representation that enables control and / or adjustment of settings of the second remotely controllable external device), and the second remotely controllable external device (e.g., a remotely controllable external device associated with the corresponding accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) is associated with the second remotely controllable external device. The request to associate a second remotely controllable external device with the context (e.g., 850c, 850d, 850e, 850f, 850g, and / or 850h) includes a user input (e.g., 850c, 850d, 850e, 850f, 850g, and / or 850h) corresponding to a selection of a third device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) (e.g., a user input including a continuous contact component and a movement component of the continuous contact (e.g., movement of the continuous contact toward a designated area of ​​the user interface including the user interface object), a swipe gesture, and / or a slide gesture). The request to associate a second remotely controllable external device with the context including the user input corresponding to the selection of the third device user interface object allows the user to request that the second remotely controllable external device be associated with the context without having to navigate to another user interface, thereby reducing the number of inputs required to perform the operation.

[0349] In some embodiments, user input (e.g., 850c, 850d, 850e, 850f, 850g, and / or 850h) corresponding to selection of a third device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) includes a continuous contact component (e.g., 850c) (e.g., a tap gesture, a touch gesture, and / or a press gesture that is detected and / or maintained for at least a predetermined amount of time (e.g., one second, two seconds, three seconds, or five seconds)) and a movement component (e.g., 850d) (e.g., movement of the continuous contact component), which The movement component (e.g., 850) includes movement (e.g., detected movement from a first position and / or location on the display generating component toward a second position and / or location on the display generating component) from a device area (e.g., 816, 844, and / or 852) toward a designated area (e.g., 814, 842, and / or 848) of a user interface (e.g., 804) that includes a user interface object (e.g., 814a) (e.g., an area of ​​the user interface that is separate (e.g., non-overlapping) and / or visually distinct from the device area) (e.g., displaying the user interface object in a designated area of ​​the user interface). The user input corresponding to the third device user interface object including the sustained contact component and the movement component allows the user to request that the second remotely controllable external device be associated with the context without having to navigate to another user interface, thereby reducing the number of inputs required to perform the operation.

[0350] In some embodiments, based on determining that a second remotely controllable external device (e.g., a remotely controllable external device associated with corresponding accessory user interface objects 816a-816i, 844a-844i, and / or 852a-852i) satisfies a set of one or more criteria, a computer system (e.g., 100, 300, 500, 604, 606, 608, 610, and / or 800) displays (914) a third device user interface object (e.g., 816a-816i, 844a-844i, and / or 852a-852i) via a display generation component (e.g., 602 and / or 802). 4a-844i and / or 852a-852i) may be used to animate a third device user interface object (e.g., 816, 844, and / or 852) from a device area (e.g., 816, 844, and / or 852) toward a designated area (e.g., 814, 842, and / or 848) of a user interface (e.g., 804) that includes a user interface object (e.g., 814a) (e.g., an area of ​​the user interface that is separate (e.g., not overlapping) and / or visually distinct from the device area) (e.g., displaying the user interface object in a designated area of ​​the user interface). Displaying the animation of the third device user interface object moving from the device area toward the designated area of ​​the user interface that includes the user interface object provides a visual confirmation to a user of the computer system that the second remotely controllable external device may be associated with the context, thereby providing improved visual feedback.

[0351] In some embodiments, based on determining that a second remotely controllable external device (e.g., a remotely controllable external device associated with corresponding accessory user interfa...

Claims

1. A method, comprising: At a computer system in communication with one or more light sources: When the computer system is configured to provide output associated with content displayed on a display generating component, receiving information associated with the content displayed on the display generating component; as well as In response to receiving the information associated with the content displayed on the display generating component, light is output via the one or more light sources in accordance with the received information associated with the content displayed on the display generating component. The method of claim 1 , wherein the light comprises a first color and a second color different from the first color.

3. The method according to any one of claims 1 to 2, wherein outputting the light via the one or more light sources according to the received information associated with the content displayed on the display generating component comprises: One or more properties of the light are adjusted over time based on timing information received regarding information associated with the content displayed on the display generating component.

4. The method of any one of claims 1 to 3, wherein the light comprises a third color based on a fourth color of at least a portion of the content displayed on the display generation component.

5. The method according to any one of claims 1 to 4, wherein: The content displayed on the display generating component includes sports events, The received information associated with the content displayed on the display generating component includes the status of the sporting event, and The light includes one or more attributes based on the status of the sporting event.

6. The method according to any one of claims 1 to 5, wherein the content displayed on the display generation component comprises a trailer page.

7. The method according to claim 6, further comprising: When the trailer page is displayed, the trailer page includes a playback user interface object: outputting the light as dynamic light via the one or more light sources based on determining that a user input corresponding to the playback user interface object has been received; as well as Based on determining that user input corresponding to the playback user interface object has not been received, the light is output as static light via the one or more light sources.

8. The method according to any one of claims 6 to 7, further comprising: prior to receiving an indication of a user input corresponding to the playback user interface object, outputting the light having a first brightness via the one or more light sources; as well as In response to receiving the indication of a user input corresponding to the playback user interface object, outputting the light having a second brightness greater than the first brightness via the one or more light sources.

9. The method of any one of claims 1 to 8, wherein the content displayed on the display generation component includes content from a currently playing media file.

10. The method according to claim 9, further comprising: At a first playback time of the currently playing media file, outputting the light having one or more first properties via the one or more light sources; as well as At a second playback time of the currently playing media file that is different from the first playback time, the light having one or more second properties different from the one or more first properties is output via the one or more light sources.

11. The method of any one of claims 9 to 10, wherein the output associated with the currently playing media file comprises audio output associated with a first song and a second song different from the first song, the method further comprising: while outputting the audio output corresponding to the first song, and while outputting the light having one or more third properties via the one or more light sources, transitioning to outputting the audio output corresponding to the second song; as well as The light having one or more fourth attributes different from the one or more third attributes is output via the one or more light sources simultaneously with the transition to outputting the audio output corresponding to the second song.

12. The method of any one of claims 1 to 11, wherein the computer system comprises a smart speaker, the method further comprising: In response to receiving voice input via one or more input devices in communication with the computer system, a second light different from the light is output via the one or more light sources.

13. The method according to any one of claims 1 to 12, further comprising: receiving an indication of an increase in volume of audio associated with the output when the light is output at a third brightness via the one or more light sources in accordance with the received information associated with the content displayed on the display generating component; as well as In response to receiving the indication of the volume increase of the audio associated with the output, the light is output via the one or more light sources at a fourth brightness greater than the third brightness according to the received information associated with the content displayed on the display generation component.

14. The method of any one of claims 1 to 13, wherein the computer system communicates with a second computer system, the second computer system being of the same type as the computer system.

15. The method of claim 14, wherein the light comprises one or more fifth properties that are different from one or more sixth properties of third light output by the second computer system.

16. The method of claim 14, wherein: The light corresponds to a first portion of the received information associated with the content displayed on the display generating component, and The second computer system is configured to output a fourth light corresponding to a second portion of the received information associated with the content displayed on the display generating component, the second portion being different from the first portion.

17. The method of claim 14, wherein: The light includes a fifth color and a sixth color different from the fifth color, and the second computer system is configured to output a fifth light including a seventh color and an eighth color different from the seventh color.

18. The method of any one of claims 1 to 17, wherein outputting the light via the one or more light sources in accordance with received information associated with the content displayed on the display generating component comprises: outputting the light having one or more seventh properties in response to determining that the predetermined function is available; as well as Based on determining that the predetermined function is not available, outputting the light having one or more eighth properties different from the one or more seventh properties.

19. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system communicating with one or more light sources, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 18.

20. A computer system configured to communicate with one or more light sources, the computer system comprising: one or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 1 to 18.

21. A computer system configured to communicate with one or more light sources, the computer system comprising: A component for carrying out the method according to any one of claims 1 to 18.

22. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with one or more light sources, the one or more programs comprising instructions for performing the method according to any one of claims 1 to 18.

23. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with one or more light sources, the one or more programs comprising instructions for: When the computer system is configured to provide output associated with content displayed on a display generating component, receiving information associated with the content displayed on the display generating component; and In response to receiving the information associated with the content displayed on the display generating component, light is output via the one or more light sources in accordance with the received information associated with the content displayed on the display generating component.

24. A computer system configured to communicate with one or more light sources, the computer system comprising: one or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: When the computer system is configured to provide output associated with content displayed on a display generating component, receiving information associated with the content displayed on the display generating component; as well as In response to receiving the information associated with the content displayed on the display generating component, light is output via the one or more light sources in accordance with the received information associated with the content displayed on the display generating component.

25. A computer system configured to communicate with one or more light sources, the computer system comprising: means for receiving information associated with content displayed on the display generating component when the computer system is configured to provide output associated with the content displayed on the display generating component; as well as means for, in response to receiving the information associated with the content displayed on the display generating component, outputting light via the one or more light sources in accordance with the received information associated with the content displayed on the display generating component.

26. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with one or more light sources, the one or more programs comprising instructions for: When the computer system is configured to provide output associated with content displayed on a display generating component, receiving information associated with the content displayed on the display generating component; and In response to receiving the information associated with the content displayed on the display generating component, light is output via the one or more light sources in accordance with the received information associated with the content displayed on the display generating component.

27. A method comprising: At a computer system in communication with one or more input devices and a display generating component: detecting, via the one or more input devices, a request to associate a second remotely controllable external device with a context while displaying, via the display generating component, a user interface including a user interface object that, when selected, provides an option for controlling a first remotely controllable external device, wherein the first remotely controllable external device is associated with the context; as well as In response to detecting the request to associate the second remotely controllable external device with the context: associating the second remotely controllable external device with the context based on determining that the second remotely controllable external device satisfies a set of one or more criteria; and Based on determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function, abandoning associating the second remotely controllable external device with the context, and the first function does not correspond to the second function of the first remotely controllable external device.

28. The method of claim 27, wherein the context comprises content currently being output by the first remotely controllable external device.

29. The method of claim 28, wherein the user interface includes an indication of the content currently being output by the first remotely controllable external device.

30. The method of claim 27, wherein the context includes the first remotely controllable external device being associated with a location.

31. The method of claim 27, wherein the context includes the first remotely controllable external device being associated with a scene.

32. The method according to any one of claims 27 to 31, wherein the user interface comprises a device area, the device area comprising: a first device user interface object different from the user interface object, the first device user interface object corresponding to a third remotely controllable external device of the automation system including the first remotely controllable external device; as well as A second device user interface object different from the user interface object, the second device user interface object corresponds to a fourth remotely controllable external device of the automation system including the first remotely controllable external device.

33. The method according to claim 32, further comprising: In response to detecting a request to associate the third remotely controllable external device with the context: Based on determining that the third remotely controllable external device satisfies the set of one or more criteria, the third remotely controllable external device is associated with the context.

34. The method of any one of claims 32 to 33, wherein the equipment area comprises: Based on determining that the context is a first context, corresponding to a first suggested-device user interface object for a fifth remotely controllable external device associated with the first context; and Based on determining that the context is a second context different from the first context, a second suggested-device user interface object corresponds to a sixth remotely controllable external device different from the fifth remotely controllable external device associated with the second context.

35. The method of claim 34, wherein the first suggestion device user interface object comprises a suggestion indicator.

36. A method according to any one of claims 32 to 35, wherein: The device area includes a third device user interface object corresponding to the second remotely controllable external device, and The request to associate the second remotely controllable external device with the context comprises a user input corresponding to a selection of the third device user interface object.

37. A method according to claim 36, wherein the user input corresponding to the selection of the third device user interface object includes a continuous contact component and a movement component, wherein the movement component includes movement from the device area toward a designated area of ​​the user interface including the user interface object.

38. The method according to any one of claims 36 to 37, further comprising: Based on determining that the second remotely controllable external device satisfies the set of one or more criteria, an animation of the third device user interface object moving from the device area toward a designated area of ​​the user interface including the user interface object is displayed via the display generation component.

39. The method according to any one of claims 36 to 38, further comprising: Based on determining that the second remotely controllable external device satisfies the set of one or more criteria: abandoning displaying the third device user interface object in the device area; and A second user interface object is displayed via the display generation component in a second designated area of ​​the user interface including the user interface object, wherein the second user interface object corresponds to the second remotely controllable external device.

40. The method of claim 39, wherein the device area and the second designated area of ​​the user interface do not overlap with each other.

41. The method according to any one of claims 36 to 40, further comprising: Based on determining that the second remotely controllable external device does not satisfy the set of one or more criteria, displaying, via the display generation component, an animation of the third device user interface object indicating that the second remotely controllable external device does not satisfy the set of one or more criteria.

42. The method according to any one of claims 27 to 41, wherein the set of one or more criteria comprises a first criterion which is satisfied when the second remotely controllable external device comprises a primary functionality corresponding to the context.

43. The method according to any one of claims 27 to 42, wherein the set of one or more criteria includes a second criterion that is satisfied when the second remotely controllable external device has a first configuration and the first remotely controllable external device has the first configuration.

44. The method according to any one of claims 27 to 43, further comprising: Based on determining that the second remotely controllable external device satisfies the set of one or more criteria, displaying, via the display generating component, a third user interface object corresponding to the second remotely controllable external device in a third designated area, wherein: The third designated area includes the user interface object, The first remotely controllable external device is a master device associated with the context, and The user interface object corresponding to the first remotely controllable external device is displayed at a first size that is larger than a second size of the third user interface object corresponding to the second remotely controllable external device.

45. The method of claim 44, further comprising: displaying the third user interface object at a first position relative to the user interface object within the third designated area based on determining that the first remotely controllable external device and the second remotely controllable external device include a first configuration; as well as Based on determining that the first remotely controllable external device and the second remotely controllable external device comprise a second configuration different from the first configuration, the third user interface object is at a second position different from the first position relative to the user interface object within the third designated area.

46. ​​The method according to any one of claims 27 to 45, further comprising: Based on determining that the second remotely controllable external device satisfies the set of one or more criteria, providing the second remotely controllable external device with a capability to act according to the context.

47. The method according to any one of claims 27 to 46, further comprising: detecting, via the one or more input devices, a request to disassociate the second remotely controllable external device from the context when the second remotely controllable external device is associated with the context; as well as In response to detecting the request to disassociate the second remotely controllable external device from the context, the second remotely controllable external device is disassociated from the context.

48. A method according to claim 47, wherein the request to disassociate the second remotely controllable external device from the context includes a user input corresponding to a selection of a fourth user interface object in a designated area of ​​the user interface that includes the user interface object, wherein the user input includes a continuous contact component and a movement component, and wherein the movement component includes movement away from the designated area.

49. The method according to any one of claims 27 to 48, further comprising: while displaying the user interface including the user interface object via the display generating component, detecting a user input corresponding to a selection of the user interface object; as well as In response to detecting the user input corresponding to selection of the user interface object, displaying, via the display generation component, a second user interface including a fourth user interface object that, when selected, controls one or more functions of the first remotely controllable external device.

50. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that communicates with one or more input devices and a display generating component, the one or more programs comprising instructions for executing a method according to any one of claims 27 to 49.

51. A computer system configured to communicate with one or more input devices and a display generation component, the computer system comprising: one or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs comprising instructions for executing the method according to any one of claims 27 to 49.

52. A computer system configured to communicate with one or more input devices and a display generation component, the computer system comprising: A component for carrying out the method according to any one of claims 27 to 49.

53. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with one or more input devices and a display generating component, the one or more programs comprising instructions for performing the method according to any one of claims 27 to 49.

54. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system in communication with one or more input devices and a display generation component, the one or more programs comprising instructions for: detecting, via the one or more input devices, a request to associate a second remotely controllable external device with a context while displaying, via the display generating component, a user interface including a user interface object that, when selected, provides an option for controlling a first remotely controllable external device, wherein the first remotely controllable external device is associated with the context; as well as In response to detecting the request to associate the second remotely controllable external device with the context: associating the second remotely controllable external device with the context based on determining that the second remotely controllable external device satisfies a set of one or more criteria; as well as Based on determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function, abandoning associating the second remotely controllable external device with the context, and the first function does not correspond to the second function of the first remotely controllable external device.

55. A computer system configured to communicate with one or more input devices and a display generation component, the computer system comprising: one or more processors; and A memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for: detecting, via the one or more input devices, a request to associate a second remotely controllable external device with a context while displaying, via the display generating component, a user interface including a user interface object that, when selected, provides an option for controlling a first remotely controllable external device, wherein the first remotely controllable external device is associated with the context; as well as In response to detecting the request to associate the second remotely controllable external device with the context: associating the second remotely controllable external device with the context based on determining that the second remotely controllable external device satisfies a set of one or more criteria; and Based on determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function, abandoning associating the second remotely controllable external device with the context, and the first function does not correspond to the second function of the first remotely controllable external device.

56. A computer system configured to communicate with one or more input devices and a display generation component, the computer system comprising: means for detecting, via the one or more input devices, a request to associate a second remotely controllable external device with a context when displaying, via the display generating component, a user interface including a user interface object that, when selected, provides an option for controlling a first remotely controllable external device, wherein the first remotely controllable external device is associated with the context; as well as means for, in response to detecting the request to associate the second remotely controllable external device with the context: associating the second remotely controllable external device with the context based on determining that the second remotely controllable external device satisfies a set of one or more criteria; as well as Based on determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function, abandoning associating the second remotely controllable external device with the context, and the first function does not correspond to the second function of the first remotely controllable external device.

57. A computer program product comprising one or more programs configured to be executed by one or more processors of a computer system in communication with one or more input devices and a display generating component, the one or more programs comprising instructions for: detecting, via the one or more input devices, a request to associate a second remotely controllable external device with a context while displaying, via the display generating component, a user interface including a user interface object that, when selected, provides an option for controlling a first remotely controllable external device, wherein the first remotely controllable external device is associated with the context; as well as In response to detecting the request to associate the second remotely controllable external device with the context: associating the second remotely controllable external device with the context based on determining that the second remotely controllable external device satisfies a set of one or more criteria; and Based on determining that the second remotely controllable external device does not satisfy the set of one or more criteria, wherein the set of one or more criteria is not satisfied when the second remotely controllable external device includes a first function, abandoning associating the second remotely controllable external device with the context, and the first function does not correspond to the second function of the first remotely controllable external device.

Images