Method and User Interface for Audio Synchronization

Abstract

The present disclosure generally relates to audio synchronization. Exemplary methods include, at a first device having a communication device: performing an audio timing synchronization process, the audio timing synchronization process including: transmitting, via the communication device, a request for the second device to participate in the audio timing synchronization process; after transmitting the request to the second device, causing an output audio tone to be output; after causing the output audio tone to be output, receiving data from the second device based on the audio tone; and adjusting, at least in part, an audio timing synchronization setting of a third device based on the data received from the second device.

Description

[0001] Cross - Reference to Related Applications

[0002] This patent application claims the benefit of U.S. Provisional Application No. 62 / 855,911, filed May 31, 2019, and U.S. Patent Application No. 16 / 667,684, filed Oct. 29, 2019, the contents of which are hereby incorporated by reference in their entireties for all purposes. Technical Field

[0003] The present disclosure generally relates to computer user interfaces, and more particularly, to techniques for audio synchronization. Background Art

[0004] The playback of media that includes audio, including movies, music, and audiobooks, can be enhanced by using multiple output devices, which can provide a more immersive audio experience. Summary of the Invention

[0005] Audio output via multiple devices can be improved by synchronizing the audio output timing. Some techniques for audio synchronization using electronic devices are generally cumbersome and inefficient. For example, some prior art uses complex and time-consuming user interfaces, which may include multiple button presses or keystrokes. The prior art takes more time than necessary, which results in wasted user time and device energy. This latter consideration is particularly important in battery-powered devices.

[0006] Accordingly, the present technology provides faster and more efficient methods and interfaces for audio synchronization in electronic devices. Such methods and interfaces optionally supplement or replace other methods for audio synchronization. Such methods and interfaces reduce the cognitive burden imposed on the user and result in a more effective human-machine interface. For battery-powered computing devices, such methods and interfaces save power and increase the time between battery charges.

[0007] Example methods are disclosed herein. An exemplary method includes performing an audio timing synchronization process at an electronic device that includes a first device having a communication device, the audio timing synchronization process including: transmitting, via the communication device, a request for a second device to participate in the audio timing synchronization process; causing an output audio tone after transmitting the request to the second device; receiving data from the second device based on the audio tone after causing the output audio tone; and adjusting, at least in part, an audio timing synchronization setting of a third device based on the data received from the second device.

[0008] This document describes an example non-transitory computer-readable storage medium. An exemplary non-transitory computer-readable storage medium stores one or more programs, the one or more programs being configured to be executed by one or more processors of an electronic device having a communication device, the one or more programs including instructions for performing the following operations: performing an audio timing synchronization process, the audio timing synchronization process including: transmitting, via the communication device, a request for the second device to participate in the audio timing synchronization process to a second device; after transmitting the request to the second device, causing an audio tone to be output; after causing the audio tone to be output, receiving data from the second device based on the audio tone; and adjusting, at least in part based on the data received from the second device, the audio timing synchronization settings of a third device.

[0009] This document describes an example transitory computer-readable storage medium. An exemplary transitory computer-readable storage medium stores one or more programs, the one or more programs being configured to be executed by one or more processors of an electronic device having a communication device, the one or more programs including instructions for performing the following operations: performing an audio timing synchronization process, the audio timing synchronization process including: transmitting, via the communication device, a request for the second device to participate in the audio timing synchronization process to a second device; after transmitting the request to the second device, causing an audio tone to be output; after causing the audio tone to be output, receiving data from the second device based on the audio tone; and adjusting, at least in part based on the data received from the second device, the audio timing synchronization settings of a third device.

[0010] This document describes an example electronic device. An exemplary electronic device includes: a communication device; one or more processors; and a memory that stores one or more programs, the one or more programs being configured to be executed by the one or more processors, the one or more programs including instructions for performing the following operations: performing an audio timing synchronization process, the audio timing synchronization process including: transmitting, via the communication device, a request for the second device to participate in the audio timing synchronization process to a second device; after transmitting the request to the second device, causing an audio tone to be output; after causing the audio tone to be output, receiving data from the second device based on the audio tone; and adjusting, at least in part based on the data received from the second device, the audio timing synchronization settings of a third device.

[0011] An exemplary electronic device includes: a communication device; and means for performing an audio timing synchronization process, the audio timing synchronization process including: transmitting, via the communication device, a request for a second device to participate in the audio timing synchronization process; after transmitting the request to the second device, causing an audio tone to be output; after causing the audio tone to be output, receiving data from the second device based on the audio tone; and adjusting, at least in part, an audio timing synchronization setting of a third device based on the data received from the second device.

[0012] Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors. Executable instructions for performing these functions are optionally included in a transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors.

[0013] Accordingly, a faster and more efficient method and interface for audio synchronization are provided for the device, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces may supplement or replace other methods for audio synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] To better understand the various described embodiments, reference should be made to the following detailed description in conjunction with the accompanying drawings, in which like reference numerals indicate corresponding parts in all the drawings.

[0015] Figure 1A is a block diagram showing a portable multifunctional device having a touch-sensitive display in accordance with some embodiments.

[0016] Figure 1B is a block diagram showing exemplary components for event handling in accordance with some embodiments.

[0017] Figure 2 shows a portable multifunctional device having a touch screen in accordance with some embodiments.

[0018] Figure 3 is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface in accordance with some embodiments.

[0019] Figure 4A shows an exemplary user interface for a menu of an application on a portable multifunctional device in accordance with some embodiments.

[0020] Figure 4B shows an exemplary user interface for a multifunctional device having a touch-sensitive surface separate from the display in accordance with some embodiments.

[0021] Figure 5AIllustrates a personal electronic device according to some embodiments.

[0022] Figure 5B Is a block diagram showing a personal electronic device according to some embodiments.

[0023] Figure 6 Illustrates an exemplary arrangement of exemplary electronic devices in an exemplary home media system according to some embodiments.

[0024] Figures 7A to 7F Illustrates an exemplary user interface for performing an audio timing synchronization process according to some embodiments.

[0025] Figures 8A to 8F Illustrates an exemplary user interface for performing an audio timing synchronization process according to some embodiments.

[0026] Figure 9 Is a flowchart showing a method for performing an audio timing synchronization process according to some embodiments. Detailed Description

[0027] The following description sets forth exemplary methods, parameters, etc. However, it should be recognized that such description is not intended to limit the scope of the present disclosure, but rather is provided as a description of exemplary embodiments.

[0028] There is a need to provide electronic devices with efficient methods and interfaces for audio synchronization. Such techniques can reduce the cognitive burden on users performing audio synchronization, thereby increasing productivity. Additionally, such techniques can reduce processor power and battery power otherwise wasted on redundant user input.

[0029] Below Figures 1A to 1B 、 Figure 2 、 Figure 3 、 Figures 4A to 4B And Figures 5A to 5B Provides a description of exemplary devices for techniques for performing management event notifications. Figure 6 、 Figures 7A to 7F 、And Figures 8A to 8F Illustrates an exemplary user interface for performing an audio timing synchronization process according to some embodiments. Figure 9 Is a flowchart showing a method for performing an audio timing synchronization process according to some embodiments. Figure 6 The exemplary arrangement of exemplary electronic devices in an exemplary home media system in Figures 7A to 7F And Figures 8A to 8F The user interfaces in are used to illustrate the processes described below, including Figure 9 The processes in.

[0030] Although the following description uses terms such as "first" and "second" to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, the first touch could be named the second touch and similarly the second touch could be named the first touch, without departing from the scope of the various described embodiments. Both the first touch and the second touch are touches, but they are not the same touch.

[0031] The terms used in the description of the various described embodiments herein are for the purpose of describing particular embodiments only and are not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will also be understood that the term "comprises" ("includes", "including", "comprises" and / or "comprising") when used in this specification is specifying the presence of the stated features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0032] Depending on the context, the term "if" is optionally interpreted to mean "when", "upon", or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined..." or "if [the stated condition or event] is detected" is optionally interpreted to mean "when it is determined..." or "in response to determining..." or "when [the stated condition or event] is detected" or "in response to detecting [the stated condition or event]".

[0033] Embodiments of electronic devices, user interfaces for such devices, and related processes of using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions such as PDA and / or music player functions. Exemplary embodiments of portable multifunctional devices include, but are not limited to, devices from Apple Inc. (Cupertino, California), devices, iPod devices, and Device. Optionally, other portable electronic devices are used, such as a laptop or a tablet computer having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer having a touch-sensitive surface (e.g., a touch screen display and / or a touchpad).

[0034] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, a mouse, and / or a joystick.

[0035] The device generally supports various applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, website creation applications, disk editing applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, fitness support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0036] The various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device are optionally adjusted and / or varied for different applications, and / or within the respective applications. Thus, a common physical architecture of the device, such as a touch-sensitive surface, optionally supports various applications with a user interface that is intuitive and clear to the user.

[0037] Attention is now turned to embodiments of a portable device having a touch-sensitive display. Figure 1AFIG. 0 is a block diagram of a portable multifunctional device 100 having a touch-sensitive display system 112 in accordance with some embodiments. The touch-sensitive display 112 is sometimes called a “touch screen” for convenience and is sometimes known as or called a “touch-sensitive display system”. Device 100 includes memory 102 (which optionally includes one or more computer-readable storage media), memory controller 122, one or more processing units (CPUs) 120, peripheral device interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input control devices 116, and external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more contact intensity sensors 165 for detecting the intensity of contacts on device 100 (e.g., a touch-sensitive surface such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touchpad 355 of device 300). These components optionally communicate over one or more communication buses or signal lines 103.

[0038] As used in this specification and the claims, the "intensity" of a contact on a touch-sensitive surface refers to the force or pressure of a contact (e.g., a finger contact) on the touch-sensitive surface (force per unit area), or to a surrogate for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values that includes at least four different values and more typically includes hundreds of different values (e.g., at least 256). The intensity of a contact is optionally determined (or measured) using a variety of methods and a variety of sensors or combinations of sensors. For example, one or more force sensors beneath or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some embodiments, force measurements from multiple force sensors are combined (e.g., weighted average) to determine the estimated contact force. Similarly, a pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as surrogates for the force or pressure of a contact on the touch-sensitive surface. In some embodiments, the surrogate measurements of contact force or pressure are used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measurements). In some embodiments, the surrogate measurements of contact force or pressure are converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of a contact as an attribute of a user input allows a user to access additional device functions that would otherwise be inaccessible to the user on a smaller device with limited footprint, which is used to display affordances and / or receive user input (e.g., via a touch-sensitive display, a touch-sensitive surface, or physical / mechanical controls such as knobs or buttons).

[0039] As used in this specification and the claims, the term "haptic output" refers to a physical displacement of the device relative to a previous position of the device detected by a user using the user's sense of touch, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., the housing), or a displacement of a component relative to the center of mass of the device. For example, in the case of contact between the device or a component of the device and a surface of the user sensitive to touch (e.g., a finger, palm, or other part of the user's hand), the haptic output generated by the physical displacement will be interpreted by the user as a sense of touch corresponding to a perceived change in a physical characteristic of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or a touchpad) is optionally interpreted by the user as a "press click" or "release click" of a physical actuation button. In some cases, the user will feel a sense of touch, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the movement of the user does not move. As another example, even when there is no change in the smoothness of the touch-sensitive surface, movement of the touch-sensitive surface will optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface. Although such interpretations of touch by the user will be limited by the user's individual sensory perception, many sensory perceptions of touch are common to most users. Thus, when a haptic output is described as corresponding to a particular sensory perception of the user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated haptic output corresponds to a physical displacement of the device or a component thereof that would generate the described sensory perception of a typical (or ordinary) user.

[0040] It should be understood that device 100 is merely an example of a portable multifunctional device, and device 100 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of these components. Figure 1A The various components shown are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0041] Memory 102 optionally includes high-speed random access memory and also optionally includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid state memory devices. Memory controller 122 optionally controls access to memory 102 by other components of device 100.

[0042] The peripheral device interface 118 can be used to couple the input and output peripheral devices of the device to the CPU 120 and the memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions of the device 100 and process data. In some embodiments, the peripheral device interface 118, the CPU 120, and the memory controller 122 are optionally implemented on a single chip such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0043] The RF (Radio Frequency) circuit 108 receives and transmits RF signals, which are also referred to as electromagnetic signals. The RF circuit 108 converts electrical signals into electromagnetic signals / converts electromagnetic signals into electrical signals, and communicates with a communication network and other communication devices via electromagnetic signals. The RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, a memory, and so on. The RF circuit 108 optionally communicates with the network and other devices via wireless communication, and these networks are such as the Internet (also known as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). The RF circuit 108 optionally includes well-known circuits for detecting a near field communication (NFC) field, such as via a short-range communication radio component. The wireless communication optionally uses any one of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-Speed Downlink Packet Access (HSDPA), High-Speed Uplink Packet Access (HSUPA), Evolution-Data Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPDA), Long-Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and / or IEEE 802.11ac), Voice over Internet Protocol (VoIP), WiMAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including communication protocols not yet developed as of the date of submission of this document.

[0044] The audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuitry 110 receives audio data from the peripheral interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into sound waves audible to humans. The audio circuitry 110 also receives the electrical signal converted from sound waves by the microphone 113. The audio circuitry 110 converts the electrical signal into audio data and transmits the audio data to the peripheral interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuitry 108 by the peripheral interface 118. In some embodiments, the audio circuitry 110 also includes an earphone jack (e.g., Figure 2 212 in

[0045] ). The earphone jack provides an interface between the audio circuitry 110 and a removable audio input / output peripheral device, which is such as an output-only headset or an earphone with both output (e.g., a mono or stereo earphone) and input (e.g., a microphone). Figure 2 208 in Figure 2 ). One or more buttons (e.g.,

[0046] Quickly pressing the depress button optionally disengages the lock of the touch screen 112 or optionally starts the process of unlocking the device using gestures on the touch screen, as described in U.S. Patent Application No. 11 / 322,549, filed on December 23, 2005, entitled "Unlocking a Device by Performing Gestures on an Unlock Image" (i.e., U.S. Patent No. 7,657,849), which is hereby incorporated by reference in its entirety. Long pressing the depress button (e.g., 206) optionally powers on or powers off the device 100. The functions of one or more buttons are optionally user-customizable. The touch screen 112 is used to implement virtual buttons or soft buttons and one or more soft keyboards.

[0047] The touch-sensitive display 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch screen 112 and / or sends electrical signals to the touch screen 112. The touch screen 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

[0048] The touch screen 112 has a touch-sensitive surface, sensor, or group of sensors that accepts input from the user based on tactile and / or haptic contact. The touch screen 112 and the display controller 156 (along with any associated modules and / or instruction sets in the memory 102) detect contact (and any movement or interruption of the contact) on the touch screen 112 and convert the detected contact into an interaction with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch screen 112. In an exemplary embodiment, the point of contact between the touch screen 112 and the user corresponds to the user's finger.

[0049] The touch screen 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, but uses other display technologies in other embodiments. The touch screen 112 and the display controller 156 optionally use any of a variety of touch sensing technologies now known or later to be developed, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch screen 112 to detect contact and any movement or interruption thereof, the variety of touch sensing technologies including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies. In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as in and the iPod the technology used in

[0050] In some embodiments of the touch-sensitive display of the touch screen 112 is optionally similar to the multi-touch sensitive touchpad described in the following U.S. patents: 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.) and / or 6,677,932 (Westerman et al.) and / or U.S. Patent Publication 2002 / 0015024A1, each of which is hereby incorporated by reference in its entirety. However, the touch screen 112 displays the visual output from the device 100, while the touch-sensitive touchpad does not provide a visual output.

[0051] In some embodiments, the touch-sensitive display of the touch screen 112 is as described in the following patent applications: (1) U.S. Patent Application No. 11 / 381,313, entitled "Multipoint Touch Surface Controller," filed on May 2, 2006; (2) U.S. Patent Application No. 10 / 840,862, entitled "Multipoint Touchscreen," filed on May 6, 2004; (3) U.S. Patent Application No. 10 / 903,964, entitled "Gestures For Touch Sensitive Input Devices," filed on July 30, 2004; (4) U.S. Patent Application No. 11 / 048,264, entitled "Gestures For Touch Sensitive Input Devices," filed on January 31, 2005; (5) U.S. Patent Application No. 11 / 038,590, entitled "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices," filed on January 18, 2005; (6) U.S. Patent Application No. 11 / 228,758, entitled "Virtual Input Device Placement On A Touch Screen User Interface," filed on September 16, 2005; (7) U.S. Patent Application No. 11 / 228,700, entitled "Operation Of A Computer With A Touch Screen Interface," filed on September 16, 2005; (8) U.S. Patent Application No. 11 / 228,737, entitled "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard," filed on September 16, 2005; and (9) U.S. Patent Application No. 11 / 367,749, entitled "Multi-Functional Hand-Held Device," filed on March 3, 2006. All of these applications are hereby incorporated by reference in their entirety.

[0052] The touch screen 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen has a video resolution of about 160 dpi. The user optionally uses any suitable object or attachment such as a stylus, finger, etc. to contact the touch screen 112. In some embodiments, the user interface is designed to work primarily through finger-based contact and gestures, which may not be as precise as stylus-based input due to the larger contact area of the finger on the touch screen. In some embodiments, the device converts the rough finger-based input into an accurate pointer / cursor position or command for performing the action desired by the user.

[0053] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of the device, which, unlike the touch screen, does not display a visual output. The touchpad is optionally a touch-sensitive surface separate from the touch screen 112 or an extension of the touch-sensitive surface formed by the touch screen.

[0054] The device 100 also includes a power system 162 for powering various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., light-emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0055] The device 100 optionally further includes one or more optical sensors 164. Figure 1AAn optical sensor coupled to the optical sensor controller 158 in the I / O subsystem 106 is shown. The optical sensor 164 optionally includes a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. The optical sensor 164 receives light projected through one or more lenses from the environment and converts the light into data representing an image. In conjunction with the imaging module 143 (also referred to as the camera module), the optical sensor 164 optionally captures still images or videos. In some embodiments, the optical sensor is located on the rear of the device 100, opposite the touch screen display 112 on the front of the device, such that the touch screen display can be used as a viewfinder for still image and / or video image capture. In some embodiments, the optical sensor is located on the front of the device such that an image of the user can optionally be acquired for video conferencing while the user views other video conferencing participants on the touch screen display. In some embodiments, the position of the optical sensor 164 can be changed by the user (e.g., by rotating the lens and sensor in the device housing) such that a single optical sensor 164 is used with the touch screen display for both video conferencing and still image and / or video image capture.

[0056] The device 100 optionally further includes one or more depth camera sensors 175. Figure 1A A depth camera sensor coupled to the depth camera controller 169 in the I / O subsystem 106 is shown. The depth camera sensor 175 receives data from the environment to create a three-dimensional model of an object (e.g., a face) within a scene from a viewpoint (e.g., the depth camera sensor). In some embodiments, in conjunction with the imaging module 143 (also referred to as the camera module), the depth camera sensor 175 is optionally used to determine a depth map of different portions of an image captured by the imaging module 143. In some embodiments, the depth camera sensor is located on the front of the device 100 such that an image of the user with depth information can optionally be acquired for video conferencing while the user views other video conferencing participants on the touch screen display, and a selfie with depth map data can be captured. In some embodiments, the depth camera sensor 175 is located on the rear of the device, or on both the rear and the front of the device 100. In some embodiments, the position of the depth camera sensor 175 can be changed by the user (e.g., by rotating the lens and sensor in the device housing) such that the depth camera sensor 175 is used with the touch screen display for both video conferencing and still image and / or video image capture.

[0057] The device 100 optionally further includes one or more contact intensity sensors 165. Figure 1AA contact intensity sensor coupled to the intensity sensor controller 159 in the I / O subsystem 106 is shown. The contact intensity sensor 165 optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electro-mechanical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). The contact intensity sensor 165 receives contact intensity information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., the touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the rear of the device 100, opposite the touch screen display 112 located on the front of the device 100.

[0058] The device 100 optionally further includes one or more proximity sensors 166. Figure 1A A proximity sensor 166 coupled to the peripheral device interface 118 is shown. Alternatively, the proximity sensor 166 is optionally coupled to the input controller 160 in the I / O subsystem 106. The proximity sensor 166 optionally operates as described in the following U.S. patent applications: No. 11 / 241,839, entitled "Proximity Detector In Handheld Device"; No. 11 / 240,788, entitled "Proximity Detector In Handheld Device"; No. 11 / 620,702, entitled "Using Ambient Light Sensor To Augment Proximity Sensor Output"; No. 11 / 586,862, entitled "Automated Response To And Sensing Of User Activity In Portable Devices"; and No. 11 / 638,251, entitled "Methods And Systems For Automatic Configuration Of Peripherals", which U.S. patent applications are hereby incorporated by reference in their entirety. In some embodiments, when the multifunctional device is placed near the user's ear (e.g., when the user is making a phone call), the proximity sensor turns off and disables the touch screen 112.

[0059] Device 100 optionally further includes one or more haptic output generators 167. Figure 1A A haptic output generator coupled to the haptic feedback controller 161 in the I / O subsystem 106 is shown. The haptic output generator 167 optionally includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components for converting an electrical signal into a haptic output on the device). The contact intensity sensor 165 receives haptic feedback generation instructions from the haptic feedback module 133 and generates a haptic output on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one haptic output generator is juxtaposed or adjacent to a touch-sensitive surface (e.g., the touch-sensitive display system 112), and optionally generates a haptic output by moving the touch-sensitive surface vertically (e.g., into / out of the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one haptic output generator sensor is located on the rear of the device 100, opposite to the touchscreen display 112 located on the front of the device 100.

[0060] Device 100 optionally further includes one or more accelerometers 168. Figure 1AAn accelerometer 168 is shown coupled to the peripheral device interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. The accelerometer 168 optionally operates as described in the following U.S. Patent Publications: U.S. Patent Publication No. 20050190059, titled "Acceleration-based Theft Detection System for Portable Electronic Devices" and U.S. Patent Publication No. 20060017692, titled "Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer", both of which are hereby incorporated by reference in their entireties. In some embodiments, information is displayed in a portrait or landscape view on the touch screen display based on an analysis of data received from one or more accelerometers. Device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver in addition to the accelerometer 168 for obtaining information about the location and orientation (e.g., portrait or landscape) of device 100.

[0061] In some embodiments, the software components stored in the memory 102 include an operating system 126, a communication module (or set of instructions) 128, a contact / motion module (or set of instructions) 130, a graphics module (or set of instructions) 132, a text input module (or set of instructions) 134, a Global Positioning System (GPS) module (or set of instructions) 135, and an application program (or set of instructions) 136. Additionally, in some embodiments, the memory 102 ( Figure 1A ) or 370 ( Figure 3 ) stores the device / global internal state 157, as Figure 1A and Figure 3 shown. The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy the various regions of the touch screen display 112; a sensor state, including information obtained from the various sensors and input control devices 116 of the device; and location information regarding the location and / or attitude of the device.

[0062] The operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware components and software components.

[0063] The communication module 128 facilitates communication with other devices via one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) is adapted to be directly coupled to other devices or indirectly coupled via a network (e.g., the Internet, a wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, similar to, and / or compatible with the 30-pin connector used on (a trademark of Apple Inc.) devices.

[0064] The contact / motion module 130 optionally detects contact with the touch screen 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or a physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection, such as determining whether contact has occurred (e.g., detecting a finger press event), determining the intensity of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement on the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point optionally includes determining the rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of the contact point being represented by a series of contact data. These operations are optionally applied to single-point contact (e.g., single-finger contact) or multi-point simultaneous contact (e.g., "multi-touch" / multiple finger contact). In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

[0065] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an operation has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds is determined according to software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators and can be adjusted without changing the physical hardware of device 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of the touchpad or touchscreen can be set to any threshold in a wide range of predefined thresholds. Additionally, in some implementations, software settings are provided to a user of the device for adjusting one or more of the intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by making a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0066] The contact / motion module 130 optionally detects gesture inputs made by a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different motions, timing, and / or intensities of the detected contacts). Thus, gestures are optionally detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger down event and then detecting a finger lift (lift-off) event at the same position (or substantially the same position) as the finger down event (e.g., at the position of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger down event, then detecting one or more finger drag events, and subsequently detecting a finger lift (lift-off) event.

[0067] The graphics module 132 includes various known software components for presenting and displaying graphics on the touchscreen 112 or other display, including components for changing the visual impact of the displayed graphics (e.g., brightness, transparency, saturation, contrast, or other visual attributes). As used herein, the term "graphics" includes any object that can be displayed to a user, which non-limitingly includes text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0068] In some embodiments, the graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives from an application, etc., one or more codes specifying the graphics to be displayed, and also receives coordinate data and other graphic attribute data as necessary, and then generates screen image data for output to the display controller 156.

[0069] The haptic feedback module 133 includes various software components for generating instructions that are used by the haptic output generator 167 to generate haptic output at one or more locations on the device 100 in response to user interaction with the device 100.

[0070] The text input module 134, which is optionally a component of the graphics module 132, provides a soft keyboard for entering text in various applications (such as contacts 137, email 140, IM 141, browser 147, and any other application that requires text input).

[0071] The GPS module 135 determines the location of the device and provides this information for use in various applications (such as providing it to the phone 138 for location-based dialing; providing it to the camera 143 as picture / video metadata; and providing it to applications that offer location-based services, such as weather widgets, local yellow pages widgets, and map / navigation widgets).

[0072] The application 136 optionally includes the following modules (or instruction sets) or subsets or supersets thereof:

[0073] · The contacts module 137 (sometimes referred to as an address book or contact list);

[0074] · The phone module 138;

[0075] · The video conferencing module 139;

[0076] · The email client module 140;

[0077] · The instant messaging (IM) module 141;

[0078] · The fitness support module 142;

[0079] · The camera module 143 for still images and / or video images;

[0080] · The image management module 144;

[0081] · The video player module;

[0082] · The music player module;

[0083] · The browser module 147;

[0084] · The calendar module 148;

[0085] · The desktop applet module 149, which optionally includes one or more of the following: weather desktop applet 149-1, stock market desktop applet 149-2, calculator desktop applet 149-3, alarm clock desktop applet 149-4, dictionary desktop applet 149-5, and other desktop applets obtained by the user, as well as user-created desktop applet 149-6;

[0086] · The desktop applet creator module 150 for forming the user-created desktop applet 149-6;

[0087] · The search module 151;

[0088] · The video and music player module 152, which combines a video player module and a music player module;

[0089] · The notepad module 153;

[0090] · The map module 154; and / or

[0091] · The online video module 155.

[0092] Examples of other application programs 136 optionally stored in the memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-supported applications, encryption, digital rights management, speech recognition, and speech replication.

[0093] In combination with the touch screen 112, the display controller 156, the contact / motion module 130, the graphics module 132, and the text input module 134, the contacts module 137 is optionally used to manage an address book or a contacts list (e.g., in the application internal state 192 of the contacts module 137 stored in the memory 102 or the memory 370), including: adding one or more names to the address book; deleting names from the address book; associating a phone number, an email address, a physical address, or other information with a name; associating an image with a name; categorizing and classifying names; providing a phone number or an email address to initiate and / or facilitate communication via the phone 138, the video conferencing module 139, the email 140, or the IM 141; and so on.

[0094] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the telephone module 138 is optionally used to input a character sequence corresponding to a telephone number, access one or more telephone numbers in the contact module 137, modify a telephone number that has been input, dial the corresponding telephone number, conduct a session, and disconnect or hang up when the session is complete. As described above, wireless communication optionally uses any one of a variety of communication standards, protocols, and technologies.

[0095] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, text input module 134, contact module 137, and telephone module 138, the video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between the user and one or more other participants in accordance with user instructions.

[0096] In combination with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the e-mail client module 140 includes executable instructions for creating, sending, receiving, and managing e-mail in response to user instructions. In combination with the image management module 144, the e-mail client module 140 makes it very easy to create and send e-mail with a still image or video image captured by the camera module 143.

[0097] In combination with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the instant messaging module 141 includes executable instructions for: inputting a character sequence corresponding to an instant message, modifying previously input characters, transmitting the corresponding instant message (e.g., using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the instant messages transmitted and / or received optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant message" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, or IMPS).

[0098] In combination with the RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module, the fitness support module 142 includes executable instructions for creating a fitness (e.g., having time, distance, and / or calorie burn goals); communicating with fitness sensors (exercise equipment); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for fitness; and displaying, storing, and transmitting fitness data.

[0099] In combination with the touch screen 112, display controller 156, optical sensor 164, optical sensor controller 158, contact / motion module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions for: capturing still images or videos (including video streams) and storing them in the memory 102, modifying the characteristics of still images or videos, or deleting still images or videos from the memory 102.

[0100] In combination with the touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, tagging, deleting, presenting (e.g., in a digital slide show or album), and storing still images and / or video images.

[0101] In combination with the RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the browser module 147 includes executable instructions for browsing the Internet according to user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, and linking to attachments and other files of web pages.

[0102] In combination with the RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying, and storing calendars and data associated with the calendars (e.g., calendar entries, to-do items, etc.) according to user instructions.

[0103] In combination with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by a user (e.g., weather desktop widget 149-1, stock market desktop widget 149-2, calculator desktop widget 149-3, alarm clock desktop widget 149-4, and dictionary desktop widget 149-5) or a mini-application created by the user (e.g., user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (HyperText Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! desktop widget).

[0104] In combination with RF circuitry 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, and browser module 147, the desktop widget creator module 150 is optionally used by a user to create a desktop widget (e.g., transforming a user-specified portion of a web page into a desktop widget).

[0105] In combination with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions for searching memory 102 for text, music, sound, images, video, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) according to a user instruction.

[0106] In combination with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, the video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats such as MP3 or AAC files, and executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch screen 112 or on an external display connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (a trademark of Apple Inc.).

[0107] In combination with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notepads, to-do lists, etc. according to user instructions.

[0108] In combination with RF circuit 108, touch screen 112, display controller 156, contact / motion module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, the map module 154 is optionally used to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions, data related to stores and other points of interest at or near a particular location, and other location-based data) according to user instructions.

[0109] In combination with touch screen 112, display controller 156, contact / motion module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, the online video module 155 includes instructions for performing the following operations: allowing a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on the touch screen or on an external display connected via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, the instant message module 141 is used instead of the email client module 140 to send a link to a particular online video. Other descriptions of the online video application can be found in U.S. Provisional Patent Application No. 60 / 936,562, filed on June 20, 2007, and entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos" and U.S. Patent Application No. 11 / 968,067, filed on December 31, 2007, and entitled "Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos", the contents of which are hereby incorporated by reference in their entirety.

[0110] Each of the above modules and applications corresponds to a set of executable instructions for performing one or more of the above functions and the methods described in this patent application (e.g., the computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) need not be implemented as separate software programs, procedures, or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. For example, the video player module is optionally combined with the music player module into a single module (e.g., Figure 1A the video and music player module 152 in

[0111] In some embodiments, device 100 is a device in which the operation of a predefined set of functions on the device is performed uniquely by a touchscreen and / or a touchpad. By using the touchscreen and / or the touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on device 100 is optionally reduced.

[0112] The predefined set of functions performed uniquely by the touchscreen and / or the touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates device 100 from any user interface displayed on device 100 to the main menu, home menu, or root menu. In such embodiments, the touchpad is used to implement the "menu button". In some other embodiments, the menu button is a physical push button or other physical input control device rather than the touchpad.

[0113] Figure 1B is a block diagram showing exemplary components for event handling according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 ) includes an event classifier 170 (e.g., in operating system 126) and corresponding applications 136-1 (e.g., any one of the foregoing applications 137 to 151, 155, 380 to 390).

[0114] The event classifier 170 receives event information and determines the application 136-1 to which the event information is to be delivered and the application view 191 of the application 136-1. The event classifier 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, the application 136-1 includes an application internal state 192 that indicates one or more current application views that are displayed on the touch-sensitive display 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application(s) is / are currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0115] In some embodiments, the application internal state 192 includes additional information such as one or more of the following: recovery information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or is ready for display by the application 136-1, a state queue for enabling the user to return to a previous state or view of the application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0116] The event monitor 171 receives event information from the peripheral device interface 118. The event information includes information about sub-events (e.g., a user touch on the touch-sensitive display 112 as part of a multi-touch gesture). The peripheral device interface 118 transmits the information it receives from the I / O subsystem 106 or sensors such as a proximity sensor 166, one or more accelerometers 168, and / or a microphone 113 (via the audio circuit 110). The information received by the peripheral device interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display 112 or a touch-sensitive surface.

[0117] In some embodiments, the event monitor 171 sends requests to the peripheral device interface 118 at predetermined intervals. In response, the peripheral device interface 118 transmits event information. In other embodiments, the peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for a duration exceeding a predetermined duration).

[0118] In some embodiments, the event classifier 170 further includes a hit view determination module 172 and / or an active event recognizer determination module 173.

[0119] When the touch-sensitive display 112 displays more than one view, the hit view determination module 172 provides a software process for determining where within one or more of the views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0120] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the corresponding application) in which a touch is detected optionally corresponds to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest-level view in which a touch is detected is optionally referred to as the hit view, and the set of events identified as correct inputs is optionally determined at least in part based on the hit view of the initial touch that begins the touch-based gesture.

[0121] The hit view determination module 172 receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, the hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should process the sub-event. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once the hit view is identified by the hit view determination module 172, the hit view generally receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0122] The active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views and thus determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with a particular view, higher views in the hierarchy will still remain as actively participating views.

[0123] The event dispatcher module 174 distributes event information to event recognizers (e.g., event recognizer 180). In embodiments that include the active event recognizer determination module 173, the event dispatcher module 174 delivers the event information to the event recognizer determined by the active event recognizer determination module 173. In some embodiments, the event dispatcher module 174 stores the event information in an event queue, which is retrieved by the corresponding event receiver 182.

[0124] In some embodiments, the operating system 126 includes an event classifier 170. Alternatively, the application 136-1 includes an event classifier 170. In yet another embodiment, the event classifier 170 is a stand-alone module or is part of another module (such as, the touch / motion module 130) stored in the memory 102.

[0125] In some embodiments, the application 136-1 includes a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring within a corresponding view of the user interface of the application. Each application view 191 of the application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a stand-alone module that is a higher-level object such as a user interface toolkit or from which the application 136-1 inherits methods and other properties. In some embodiments, the corresponding event handlers 190 include one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from the event classifier 170. The event handlers 190 optionally utilize or call the data updater 176, the object updater 177, or the GUI updater 178 to update the internal state 192 of the application. Alternatively, one or more of the application views 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of the data updater 176, the object updater 177, and the GUI updater 178 are included in the corresponding application view 191.

[0126] The corresponding event recognizer 180 receives event information (e.g., event data 179) from the event classifier 170 and identifies events based on the event information. The event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, the event recognizer 180 also includes at least a subset of metadata 183 and event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0127] The event receiver 182 receives event information from the event classifier 170. The event information includes information about sub-events such as a touch or a touch movement. Depending on the sub-event, the event information further includes additional information such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information optionally further includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device pose).

[0128] The event comparator 184 compares the event information with predefined event or sub-event definitions and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, the event comparator 184 includes an event definition 186. The event definition 186 contains definitions of events (e.g., predefined sequences of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, the sub-events in an event (187) include, for example, touch start, touch end, touch movement, touch cancellation, and multi-touch. In one example, the definition of event 1 (187-1) is a double-tap on a displayed object. For example, a double-tap includes a first touch (touch start) of a predetermined duration on the displayed object, a first lift-off (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on the displayed object, and a second lift-off (touch end) of a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, a drag includes a touch (or contact) of a predetermined duration on the displayed object, movement of the touch on the touch-sensitive display 112, and lift-off of the touch (touch end). In some embodiments, the event further includes information for one or more associated event handlers 190.

[0129] In some embodiments, the event definition 187 includes definitions of events for corresponding user interface objects. In some embodiments, the event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view that displays three user interface objects on the touch-sensitive display 112, when a touch is detected on the touch-sensitive display 112, the event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, the event comparator 184 selects the event handler associated with the sub-event and the object that triggered the hit test.

[0130] In some embodiments, the definition of a corresponding event (187) also includes a delay action that delays the delivery of event information until it has been determined that the sub - event sequence does in fact correspond to, or does not correspond to, the event type of the event recognizer.

[0131] When the corresponding event recognizer 180 determines that the sub - event sequence does not match any of the events in the event definition 186, the corresponding event recognizer 180 enters an event - impossible, event - failed, or event - ended state, after which subsequent sub - events of the touch - based gesture are ignored. In such a case, any other event recognizers (if any) for which the hit view remains active continue to track and process the ongoing sub - events of the touch - based gesture.

[0132] In some embodiments, the corresponding event recognizer 180 includes metadata 183 having configurable attributes, flags, and / or lists that indicate how the event delivery system should perform sub - event delivery to the active participating event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists that indicate how event recognizers interact with each other or can interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists that indicate whether sub - events are delivered to different levels in a view or a programmatic hierarchy.

[0133] In some embodiments, when one or more specific sub - events of an event are recognized, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferring) sub - events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a token associated with the recognized event, and the event handler 190 associated with that token retrieves the token and executes a predefined process.

[0134] In some embodiments, the event delivery instruction 188 includes a sub - event delivery instruction that delivers event information about a sub - event without activating the event handler. Instead, the sub - event delivery instruction delivers the event information to an event handler associated with the sub - event sequence or to an actively participating view. The event handler associated with the sub - event sequence or with the actively participating view receives the event information and executes a predetermined process.

[0135] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates a telephone number used in contact module 137, or stores a video file used in a video player module. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates a new user interface object or updates the position of a user interface object. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on the touch-sensitive display.

[0136] In some embodiments, event handler 190 includes data updater 176, object updater 177, and GUI updater 178, or has access to the data updater, the object updater, and the GUI updater. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the respective application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0137] It should be understood that the above discussion of event handling for user touches on a touch-sensitive display also applies to other forms of user input for operating multifunctional device 100 using an input device, and not all user input is initiated on a touchscreen. For example, mouse movement and mouse button presses optionally in cooperation with single or multiple keyboard presses or holds; contact movement on a touchpad, such as tapping, dragging, scrolling, etc.; stylus input; movement of the device; verbal instructions; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to sub-events that define the event to be recognized.

[0138] Figure 2FIG. 0 shows a portable multifunctional device 100 having a touch screen 112, according to some embodiments. The touch screen optionally displays one or more graphics within a user interface (UI) 200. In this and other embodiments described below, the user is able to select one or more of these graphics by making gestures on the graphics, such as by using one or more fingers 202 (not drawn to scale in the figures) or one or more styli 203 (not drawn to scale in the figures). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gestures optionally include one or more taps, one or more swipes (from left to right, right to left, up, and / or down), and / or rolling of a finger that has made contact with the device 100 (from right to left, left to right, up, and / or down). In some implementations or in some cases, inadvertently contacting a graphic does not select the graphic. For example, when the gesture corresponding to selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

[0139] The device 100 optionally further includes one or more physical buttons, such as a "home" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any of a set of applications 136 that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI that is displayed on the touch screen 112.

[0140] In some embodiments, the device 100 includes a touch screen 112, a menu button 204, a depressible button 206 for powering on / off the device and for locking the device, one or more volume adjustment buttons 208, a subscriber identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The depressible button 206 is optionally used to power on / off the device by depressing the button and holding the button in the depressed state for a predefined time interval; to lock the device by depressing the button and releasing the button before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In an alternative embodiment, the device 100 also receives voice input for activating or deactivating certain functions via a microphone 113. The device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of contact on the touch screen 112, and / or one or more tactile output generators 167 for generating a tactile output for a user of the device 100.

[0141] Figure 3is a block diagram of an exemplary multifunctional device having a display and a touch-sensitive surface, in accordance with some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, desktop computer, tablet computer, multimedia player device, navigation device, educational device (such as a children's learning toy), game system, or control device (e.g., a home or industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuitry (sometimes called a chipset) that interconnects system components and controls the communication between them. Device 300 includes an input / output (I / O) interface 330 having a display 340, which is typically a touch screen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a haptic output generator 357 for generating haptic output on device 300 (e.g., similar to haptic output generator 167 described above with reference to Figure 1A ), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch-sensitive sensors, and / or contact intensity sensors (similar to contact intensity sensor 165 described above with reference to Figure 1A ). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid state memory devices; and optionally includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores programs, modules, and data structures similar to or a subset of those stored in memory 102 of portable multifunctional device 100 ( Figure 1A ). Additionally, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunctional device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while memory 102 of portable multifunctional device 100 ( Figure 1A ) optionally does not store these modules.

[0142] Figure 3Each of the above elements in [element name] is optionally stored in one or more of the previously mentioned memory devices of the memory device. Each of the above modules corresponds to a set of instructions for performing the above functions. The above modules or programs (e.g., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, the memory 370 optionally stores a subgroup of the above modules and data structures. Additionally, the memory 370 optionally stores additional modules and data structures not described above.

[0143] Attention is now turned to an embodiment of a user interface optionally implemented on, for example, the portable multifunctional device 100.

[0144] Figure 4A An exemplary user interface of an application menu on the portable multifunctional device 100 according to some embodiments is shown. A similar user interface is optionally implemented on the device 300. In some embodiments, the user interface 400 includes the following elements or a subset or superset thereof:

[0145] · A signal strength indicator 402 for wireless communications such as cellular signals and Wi-Fi signals;

[0146] · Time 404;

[0147] · A Bluetooth indicator 405;

[0148] · A battery status indicator 406;

[0149] · A tray 408 with icons for commonly used applications, such as:

[0150] ο An icon 416 labeled "Phone" for the phone module 138, which icon 416 optionally includes an indicator 414 of the number of missed calls or voicemails;

[0151] ο An icon 418 labeled "Mail" for the email client module 140, which icon 418 optionally includes an indicator 410 of the number of unread emails;

[0152] ο An icon 420 labeled "Browser" for the browser module 147; and

[0153] ο An icon 422 labeled "iPod" for the video and music player module 152 (also referred to as the iPod (trademark of Apple Inc.) module 152); and

[0154] · Icons for other applications, such as:

[0155] The icon 424 marked as "Message" of the οIM module 141;

[0156] The icon 426 marked as "Calendar" of the οCalendar module 148;

[0157] The icon 428 marked as "Photo" of the οImage Management module 144;

[0158] The icon 430 marked as "Camera" of the οCamera module 143;

[0159] The icon 432 marked as "Online Video" of the οOnline Video module 155;

[0160] The icon 434 marked as "Stock Market" of the οStock Market Desktop Applet 149-2;

[0161] The icon 436 marked as "Map" of the οMap module 154;

[0162] The icon 438 marked as "Weather" of the οWeather Desktop Applet 149-1;

[0163] The icon 440 marked as "Clock" of the οAlarm Desktop Applet 149-4;

[0164] The icon 442 marked as "Fitness Support" of the οFitness Support module 142;

[0165] The icon 444 marked as "Notepad" of the οNotepad module 153; and

[0166] The icon 446 marked as "Settings" of the οSettings Application or Module, which provides access to the settings of the device 100 and its various applications 136.

[0167] It should be noted that Figure 4A The icon labels shown in

[0168] Figure 4B are merely exemplary. For example, the icon 422 of the Video and Music Player module 152 is marked "Music" or "Music Player". Other labels may optionally be used for the various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon. Figure 3 A device (e.g., Figure 3Exemplary user interface on device 300). Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of contacts on the touch-sensitive surface 451 and / or one or more tactile output generators 357 for generating tactile output for a user of device 300.

[0169] Although some of the following examples will be given with reference to input on a touch screen display 112 (where a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface separate from the display, as Figure 4B shown. In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451 in ) has a major axis (e.g., Figure 4B 452 in ) corresponding to the major axis (e.g., Figure 4B 453 in ) on the display (e.g., 450). According to these embodiments, the device detects contact (e.g., Figure 4B 460 and 462 in ) with the touch-sensitive surface 451 at a location corresponding to a respective location on the display (e.g., in Figure 4B 460 corresponds to 468 and 462 corresponds to 470). Thus, when the touch-sensitive surface (e.g., Figure 4B 451 in ) is separate from the display of the multifunctional device (e.g., Figure 4B 450 in ), user input detected by the device on that touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0170] Additionally, although the following examples are mainly given with reference to finger input (e.g., finger contact, single-finger tap gesture, finger swipe gesture), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of a contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of a contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is above the location of the tap gesture (e.g., instead of detecting a contact, followed by stopping detection of the contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or a mouse and a finger contact are optionally used simultaneously.

[0171] Figure 5AAn exemplary personal electronic device 500 is shown. The device 500 includes a body 502. In some embodiments, the device 500 may include some or all of the features described with respect to devices 100 and 300 (e.g., Figures 1A to 4B ). In some embodiments, the device 500 has a touch-sensitive display screen 504 hereinafter referred to as a touch screen 504. As an alternative or addition to the touch screen 504, the device 500 has a display and a touch-sensitive surface. As in the case of devices 100 and 300, in some embodiments, the touch screen 504 (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of an applied contact (e.g., a touch). One or more intensity sensors of the touch screen 504 (or touch-sensitive surface) may provide output data representing the intensity of the touch. The user interface of the device 500 may respond to the touch based on the intensity of the touch, meaning that touches of different intensities may invoke different user interface operations on the device 500.

[0172] Exemplary techniques for detecting and processing touch intensity are found, for example, in the following related patent applications: International Patent Application Serial No. PCT / US2013 / 040061, filed May 8, 2013, entitled "Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application," published as WIPO Patent Publication No. WO / 2013 / 169849; and International Patent Application Serial No. PCT / US2013 / 069483, filed November 11, 2013, entitled "Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships," published as WIPO Patent Publication No. WO / 2014 / 105276, each of which is hereby incorporated by reference in its entirety.

[0173] In some embodiments, device 500 has one or more input mechanisms 506 and 508. Input mechanisms 506 and 508 (if included) can be in physical form. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, device 500 has one or more attachment mechanisms. Such attachment mechanisms (if included) may allow device 500 to be attached to, for example, hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watchbands, bracelets, pants, belts, shoes, wallets, backpacks, etc. These attachment mechanisms allow the user to wear device 500.

[0174] Figure 5B An exemplary personal electronic device 500 is shown. In some embodiments, device 500 may include some or all of the components described with reference Figure 1A , Figure 1B and Figure 3 Device 500 has a bus 512 that operably couples the I / O section 514 to one or more computer processors 516 and a memory 518. The I / O section 514 may be connected to a display 504 that may have a touch-sensitive component 522 and optionally a strength sensor 524 (e.g., a contact strength sensor). Additionally, the I / O section 514 may be connected to a communication unit 530 for receiving application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular, and / or other wireless communication technologies. Device 500 may include input mechanisms 506 and / or 508. For example, input mechanism 506 is optionally a rotatable input device or a pressable input device and a rotatable input device. In some examples, input mechanism 508 is optionally a button.

[0175] In some examples, input mechanism 508 is optionally a microphone. Personal electronic device 500 optionally includes various sensors such as a GPS sensor 532, an accelerometer 534, an orientation sensor 540 (e.g., a compass), a gyroscope 536, a motion sensor 538, and / or combinations thereof, all of which are operably connected to the I / O section 514.

[0176] The memory 518 of personal electronic device 500 may include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by one or more computer processors 516, may cause the computer processors to perform the techniques described below, including process 900 ( Figure 9)。A computer-readable storage medium can be any medium that tangibly contains or stores computer-executable instructions for use by or in connection with an instruction execution system, apparatus, and device. In some examples, the storage medium is a transient computer-readable storage medium. In some examples, the storage medium is a non-transient computer-readable storage medium. Non-transient computer-readable storage media can include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state memories such as flash memory, solid-state drives, etc. The personal electronic device 500 is not limited to Figure 5B the components and configurations thereof, but may include other components or additional components in a variety of configurations.

[0177] As used herein, the term "indicative representation" refers to a user-interactive graphical user interface object optionally displayed on the display screen of the devices 100, 300, and / or 500 ( Figure 1A 、 Figure 3 and Figures 5A to 5B ). For example, images (e.g., icons), buttons, and text (e.g., hyperlinks) each optionally constitute an indicative representation.

[0178] As used herein, the term "focus selector" refers to an input element for indicating the current portion of the user interface with which the user is interacting. In some specific implementations including a cursor or other position marker, the cursor acts as the "focus selector" such that when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., Figure 3 the touchpad 355 in Figure 4B or Figure 1A the touch-sensitive display system 112 in Figure 4AIn some specific implementations of the touch screen 112), the detected contact on the touch screen serves as a "focus selector", such that when an input (e.g., a press input made by a contact) is detected at the position of a specific user interface element (e.g., a button, a window, a slider, or other user interface element) on the touch screen display, the specific user interface element is adjusted according to the detected input. In some specific implementations, the focus moves from one area of the user interface to another area of the user interface without a corresponding movement of the cursor or a movement of the contact on the touch screen display (e.g., moving the focus from one button to another button by using the tab key or arrow keys); in these specific implementations, the focus selector moves according to the movement of the focus between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is generally a user interface element (or a contact on the touch screen display) that is controlled by the user in order to deliver the user-expected interaction with the user interface (e.g., by indicating to the device the element of the user interface that the user desires to interact with). For example, when a press input is detected on a touch-sensitive surface (e.g., a touchpad or a touch screen), the position of the focus selector (e.g., a cursor, a contact, or a selection box) above the corresponding button will indicate that the user desires to activate the corresponding button (rather than other user interface elements shown on the device display).

[0179] As used in the specification and claims, the term "feature intensity" of a contact refers to a feature of the contact based on one or more intensities of the contact. In some embodiments, the feature intensity is based on a plurality of intensity samples. The feature intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predefined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after detecting the contact, before detecting the lift-off of the contact, before or after detecting the start of movement of the contact, before detecting the end of the contact, before or after detecting an increase in the intensity of the contact, and / or before or after detecting a decrease in the intensity of the contact). The feature intensity of the contact is optionally based on one or more of the following: the maximum value of the intensity of the contact, the mean value of the intensity of the contact, the average value of the intensity of the contact, the value at the 10% front of the intensity of the contact, the half maximum value of the intensity of the contact, the 90% maximum value of the intensity of the contact, etc. In some embodiments, the duration of the contact is used in determining the feature intensity (e.g., when the feature intensity is the average value of the intensity of the contact over time). In some embodiments, the feature intensity is compared with a set of one or more intensity thresholds to determine whether the user has performed an operation. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a contact whose feature intensity does not exceed the first threshold results in a first operation, a contact whose feature intensity exceeds the first intensity threshold but does not exceed the second intensity threshold results in a second operation, and a contact whose feature intensity exceeds the second threshold results in a third operation. In some embodiments, the comparison between the feature intensity and one or more thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding operation or to forgo performing the corresponding operation) rather than for determining whether to perform a first operation or a second operation.

[0180] In some embodiments, a portion of the gesture is identified for determining the feature intensity. For example, the touch-sensitive surface optionally receives a continuous swipe contact that transitions from a starting position to an ending position where the contact intensity increases. In this example, the feature intensity of the contact at the ending position is optionally based on only a portion of the continuous swipe contact rather than the entire swipe contact (e.g., only the portion of the swipe contact at the ending position). In some embodiments, a smoothing algorithm is optionally applied to the intensity of the swipe contact before determining the feature intensity of the contact. For example, the smoothing algorithm optionally includes one or more of the following: an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow spikes or dips in the intensity of the swipe contact for the purpose of determining the feature intensity.

[0181] Optionally, characterize the contact intensity on the touch-sensitive surface relative to one or more intensity thresholds such as a contact detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and / or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform an operation different from an operation typically associated with clicking a button of a physical mouse or touchpad. In some embodiments, when a contact is detected with a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold, contacts below the nominal contact detection intensity threshold are no longer detected), the device will move a focus selector based on the movement of the contact on the touch-sensitive surface without performing an operation associated with the light press intensity threshold or the deep press intensity threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different sets of user interface figures.

[0182] An increase in the contact characteristic intensity from an intensity below the light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in the contact characteristic intensity from an intensity below the deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in the contact characteristic intensity from an intensity below the contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in the contact characteristic intensity from an intensity above the contact detection intensity threshold to an intensity below the contact detection intensity threshold is sometimes referred to as detecting a lift-off of the contact from the touch surface. In some embodiments, the contact detection intensity threshold is zero. In some embodiments, the contact detection intensity threshold is greater than zero.

[0183] In some embodiments described herein, one or more operations are performed in response to detecting a gesture including a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or contacts), where the corresponding press input is detected at least in part based on the intensity of the detected contact (or contacts) increasing above a press input intensity threshold. In some embodiments, a corresponding operation is performed in response to detecting the intensity of the corresponding contact increasing above the press input intensity threshold (e.g., the "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the intensity of the corresponding contact above the press input intensity threshold and a subsequent decrease in the intensity of the contact below the press input intensity threshold, and a corresponding operation is performed in response to detecting the subsequent decrease in the intensity of the corresponding contact below the press input threshold (e.g., the "up stroke" of the corresponding press input).

[0184] In some embodiments, the device employs strength hysteresis to avoid unexpected inputs sometimes referred to as "jitter", where the device defines or selects a hysteresis strength threshold having a predefined relationship to a press input strength threshold (e.g., the hysteresis strength threshold is X strength units lower than the press input strength threshold, or the hysteresis strength threshold is 75%, 90%, or some reasonable proportion of the press input strength threshold). Thus, in some embodiments, a press input includes the strength of a corresponding contact increasing above the press input strength threshold and the strength of that contact subsequently decreasing below the hysteresis strength threshold corresponding to the press input strength threshold, and a corresponding operation is performed in response to detecting that the strength of the corresponding contact subsequently decreases below the hysteresis strength threshold (e.g., the "upstroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the contact strength increases from a strength equal to or below the hysteresis strength threshold to a strength equal to or above the press input strength threshold and optionally the contact strength subsequently decreases to a strength equal to or below the hysteresis strength, and a corresponding operation is performed in response to detecting the press input (e.g., depending on the context, the contact strength increases or the contact strength decreases).

[0185] For ease of explanation, optionally, a description of an operation performed in response to a press input associated with a press input strength threshold or in response to a gesture including a press input is triggered in response to detecting any one of the following various situations: the contact strength increases above the press input strength threshold, the contact strength increases from a strength below the hysteresis strength threshold to a strength above the press input strength threshold, the contact strength decreases below the press input strength threshold, and / or the contact strength decreases below the hysteresis strength threshold corresponding to the press input strength threshold. Additionally, in an example where an operation is described as being performed in response to detecting that the strength of a contact decreases below the press input strength threshold, the operation is optionally performed in response to detecting that the strength of the contact decreases below the hysteresis strength threshold corresponding to and less than the press input strength threshold.

[0186] As used herein, an "installed application" refers to a software application that has been downloaded to an electronic device (e.g., devices 100, 300, and / or 500) and is ready to be launched (e.g., made open) on the device. In some embodiments, a downloaded application becomes an installed application using an installer that extracts program portions from the downloaded software package and integrates the extracted portions with the operating system of the computer system.

[0187] As used herein, the terms "open application" or "executing application" refer to a software application that maintains state information (e.g., as part of the device / global internal state 157 and / or the application internal state 192). An open or executing application is optionally any one of the following types of applications:

[0188] · The active application currently displayed on the display screen of the device on which the application is being used;

[0189] · Background applications (or background processes), which are not currently displayed but one or more processes of the application are being processed by one or more processors; and

[0190] · Paused or dormant applications that are not running but have state information stored in memory (volatile and non-volatile respectively) and can be used to resume the execution of the application.

[0191] As used herein, the term "closed application" refers to a software application that does not maintain state information (e.g., the state information of a closed application is not stored in the memory of the device). Thus, closing an application includes stopping and / or removing the application process of the application and removing the state information of the application from the memory of the device. Generally speaking, when in a first application, opening a second application does not close the first application. When the second application is being displayed and the first application stops being displayed, the first application becomes a background application.

[0192] Attention is now turned to embodiments of a user interface ("UI") implemented on an electronic device (such as the portable multifunctional device 100, device 300, or device 500) and associated processes.

[0193] Figure 6 、 Figures 7A to 7F 、and Figures 8A to 8F illustrates an exemplary user interface for performing an audio timing synchronization process according to some embodiments. The user interfaces in these figures are used to illustrate processes including the processes described below in Figure 9 .

[0194] Figure 6 illustrates an exemplary arrangement of exemplary electronic devices in an exemplary home media system 600. In some embodiments, a home media system (e.g., 600) includes one or more electronic devices (e.g., 602, 606, 608A, 608B). In this example, the home media system 600 includes a home media center 602, a display device 606 (e.g., a television), and audio output devices 608A - 608B. In some embodiments, device 602, device 606, and device 608 each include one or more features of devices 100, 300, 500, and 580.

[0195] As Figure 6As shown, the home media system 600 includes a home media center 602. In some embodiments, the home media center (e.g., 602) is a device that includes one or more of the following features: media content (e.g., video and / or audio) storage, media content streaming (e.g., via a network connection), video output (e.g., via HDMI and / or other wired or wireless connections), audio output (e.g., via HDMI, optical interface, and / or other wired or wireless connections), one or more display devices (e.g., screens), output to one or more external display devices (e.g., televisions), a user interface (e.g., a graphical user interface), and support for user input and control via an external device (e.g., a remote control) (e.g., via infrared communication, Bluetooth communication, etc.). An example of a home media center is the "Apple TV" device produced by Apple Inc. (Cupertino, California).

[0196] The exemplary home media system 600 also includes a display device 606 (e.g., a television). In this example, the display device 606 is connected to the home media center 602 (e.g., via HDMI), and the home media center 602 causes content (e.g., a graphical user interface, media content) to be displayed on the display device 606 (e.g., because the home media center 602 does not include a display).

[0197] The exemplary home media system 600 also includes audio output devices 608A and 608B, which are smart speakers in this example. In some embodiments, the audio output devices (e.g., 608A, 608B) are connected to one or more of the following: one or more home media centers (e.g., 602) (e.g., via HDMI, optical interface, and / or other wired or wireless connections), one or more display devices (e.g., 606) (e.g., via HDMI, optical interface, and / or other wired or wireless connections), and one or more audio amplifiers (e.g., when the audio output device is a regular speaker) (e.g., which is connected to one or more of 602 and 606 via HDMI, optical interface, and / or other wired or wireless connections). In this example, the audio output devices 608A and 608B are connected to the home media center 602 via a wireless connection (e.g., Wi-Fi). The audio output devices 608A and 608B can be configured to output the audio (e.g., music, dialogue, tones, etc.) of content (e.g., media, applications, etc.) synchronously with each other and with the audio output by other devices (e.g., the display device 606). For example, the smart speakers 608A and 608B can be used as audio output devices that provide an audio output of the content in the direction of (e.g., output by, received from) the home media center 602 and / or the display device 606 (e.g., as a supplement to or replacement for the speakers of the display device 606). For example, the interface of the home media center 602 can be used to select music or a movie for playback, and the home media center 602 can then cause the audio output associated with the selected media content to be output at the speakers of the display device 606 and at the audio output devices 608A and 608B (e.g., by transmitting data to those external devices 606, 608A, and 608B). Causing the audio content to be output (e.g., at the external device) can include one or more of the following: providing data representing the content (e.g., a stream of the audio track), and providing data on where such content can be accessed (e.g., a location on a local area network, a location on a wide area network (e.g., the Internet address of a music streaming service)). In such examples of coordinated content output between various devices, the timing synchronization of the audio output between the various devices that are outputting audio is important because a listener can perceive even a slight difference in the arrival times of the audio output from different audio output sources at the listener's location.

[0198] Figure 6Also shown is a user 610 holding an exemplary electronic device, a smart phone 604, and positioned at a listening location. In some embodiments, the smart phone 604 includes one or more features of devices 100, 300, and 500. In this example, the smart phone 604 includes a display device 604A, one or more microphones, and a communication device. The smart phone 604 is connected to a home media center 602 by connecting to a common network (e.g., a home Wi-Fi network). Thus, the home media center 602 and the smart phone 604 can communicate (e.g., exchange data) via the common network. Figure 6 Also included is a tablet 612 near the user 610. In some embodiments, the tablet 612 includes one or more features of devices 100, 300, and 500, including one or more microphones and a communication device.

[0199] If the home media system has not been optimized, using the audio output of the home media system (e.g., 600) can result in unexpected and negative effects. In one example, the use of different devices (e.g., 606, 608A, 608B) with different respective connections (e.g., the display device 606 has a wired connection to the home media center 602, and the audio output devices 608A and 608B have a wireless connection to the home media center 602) can result in the audio output having different time delays between when the audio signal is output by the first device (e.g., the home media center 602) and when the audio output reaches the listener's location. As another example, different distances between the respective audio output devices and the listener can cause the respective audio outputs from each of the audio output devices to reach the listener's location at different times (e.g., because the audio output from a more distant audio output device needs to cover a greater distance to reach the listener). In such examples, the audio outputs from the respective audio output devices will sound out of sync with each other to the listener. In any of the above examples, the experience of the user (e.g., the listener) is negatively affected. However, some techniques for optimizing the home media system may involve complex interfaces, specialized devices that are not easily accessible to users, or are not supported at all. The techniques provided below with respect to Figures 7A to 7F and Figures 8A to 8F show an effective and efficient process for optimizing a home media system according to some embodiments.

[0200] In some embodiments, the home media system (e.g., 600) may include one or more devices other than those described above. In some embodiments, the home media system may include more than one of the above devices. In some embodiments, the home media system may include less than one of the above devices.

[0201] Figures 7A to 7FShows an exemplary interface for performing an audio timing synchronization process using a first device (e.g., home media center 602). In Figures 7A to 7F 's example, the home media center 602 is connected to a display device 606, audio output devices 608A - 608B, and a smart phone 604, as described above with respect to Figure 6 . For simplicity, the home media center 602 and the audio output devices 608A - 608B are not shown in Figures 7A to 7F . In Figures 7A to 7F , the home media center 602 causes the corresponding interfaces shown in each figure to be displayed on the display device 606. Thus, in these examples, the display device 606 serves as an external display for showing the interfaces of the home media center 602. Figures 7A to 7F The user input described in

[0202] Figure 7A is received at the home media center 602 or an external device (e.g., remote control) connected to the home media center 602. Figure 7A Shows an exemplary home media center configuration menu interface 710. In this example, a user who wishes to manually start performing the audio timing synchronization process navigates to the interface 710. The interface 710 includes a plurality of options for configuring and / or switching settings associated with the home media center 602 (e.g., audio and video related settings), including an enabling representation 710A for initiating at least a part of the audio timing synchronization process. In Figure 7A , the home media center 602 receives user input representing a request to initiate the audio timing synchronization process. For example, the home media center 602 receives data representing a user input selection (e.g., from a remote control) of the menu enabling representation 710A, which includes the text "Wireless Audio Sync" and is highlighted for selection in Figures 7B to 7F . In response to receiving a request to initiate the audio timing synchronization process, the device (e.g., 602) performs at least a part of the timing synchronization process (e.g., performs one of the actions described below with respect to

[0203] In some embodiments, the audio timing synchronization process is performed automatically (e.g., in the background) without being started via user input (e.g., selection of the enabling representation 710A). For example, the audio timing synchronization process may be performed in a user - invisible manner (e.g., without displaying an indication interface, and / or without outputting an audible tone) during the initial setup of the home media center 602.

[0204] Figure 7BShows an audio timing synchronization process interface 720 that is optionally displayed when performing an audio timing synchronization process at the home media center 602. As shown, the interface 720 includes the text "Wireless Audio Sync" and "Your home media center will use nearby devices to measure how long it takes for your TV to play audio and video. Tap the notification on the nearby device". Thus, the interface 720 provides instructions to the user on how to proceed with the audio timing synchronization process. In this example, user input (e.g., a tap) at a second device (e.g., the smart phone 604) is required to proceed with the audio timing synchronization process. For example, when the home media center causes the interface 720 to be displayed, the smart phone 604 may display a notification 804 waiting for user input 806 (as Figure 8A shown, which is described in more detail below) before proceeding with the audio timing synchronization process.

[0205] In some embodiments, performing the audio timing synchronization process includes transmitting a request to one or more devices to participate in the audio timing synchronization process. For example, the home media center 602 transmits a request to the smart phone 604 to participate in the audio timing synchronization process (e.g., which receives a notification 804 that causes the smart phone 604 to display, as Figure 8A shown). In this example, the request is transmitted based on receiving a selection of the enablement indication 710A. In some embodiments, the home media center 602 also transmits a request to the tablet 612 to participate in the audio timing synchronization process.

[0206] In some embodiments, performing the audio timing synchronization process includes causing the display of one or more indications (e.g., 720A) of one or more devices that can participate in the audio timing synchronization process. For example, the interface 720 includes an indicator 720A depicting a plurality of devices (e.g., a tablet and a smart phone), indicating that any one of the plurality of devices can be used to perform the audio timing synchronization process. The home media center 602 sends a request to a plurality of nearby devices (e.g., devices connected to the same network (e.g., a Wi-Fi network) and / or within a wireless range (e.g., a Wi-Fi range, a Bluetooth range)) to participate in the audio timing synchronization process.

[0207] Using a second device such as the smart phone 604 as part of the audio timing synchronization process can be beneficial. For example, the use of a dedicated device can be eliminated by using a user device that can easily participate in the audio timing synchronization process (e.g., a user device that includes one or more microphones and is connected to the home media center 602). Additionally, the user may be holding the smart phone 604 in the same position where audio timing needs to be optimized (e.g., the normal listening position, as Figure 6 shown by the position of the user 610 in) (e.g., based on the distance from that position and one or more audio output devices).

[0208] Figures 7C to 7D shows an audio timing synchronization process interface 730, and the home media center 602 causes the display of the audio timing synchronization process interface after transmitting a request to participate in the audio timing synchronization process to one or more devices. For example, the home media center 602 receives data representing one or more of the user inputs 806 ( Figure 8A ) and 810 ( Figure 8B ) received at the smart phone 604, and based on receiving such data, causes the display of the interface 730. The interface 730 includes the text "Play Tone" and "Play Tone from Speaker". Thus, the interface 730 provides an indication that the home media center 602 is currently causing an audio tone to be output (e.g., output by the speaker of the display device 606). The interface 730 also includes a graphical indication 730A that visually indicates that an audio tone is being output. In this example, the indication 730A includes a representation of the display device having concentric circles (e.g., representing sound waves) that emanate outwardly from the depicted display device in an animated manner and increase in size (e.g., in Figure 7D , Figure 7C the circle representing the sound wave has increased to a larger size, and additional smaller circles are shown in Figure 7D ).

[0209] In some embodiments, the interface includes an indication of the device participating in the audio timing synchronization process. For example, the interface 730 also includes an indicator 730B that provides an indication of another device that is participating in the audio timing synchronization process. As shown, the indicator 730B resembles the top of the smart phone 604 (as shown in Figure 6 and Figure 8A ). If instead a tablet computer (as shown in 720A of Figure 7B ) is used in the audio timing synchronization process, then the indicator 730B may be different and provide an indication that the tablet computer is being used in the audio timing synchronization process. Providing an indication such as 730B can reduce confusion regarding which device is currently participating in the audio timing synchronization process, making the process more user-friendly.

[0210] In the Figures 7C to 7D example, the home media center 602 causes the display device 606 (TV) to output audio through the speaker of the device 606. The output audio may be one or more audio tones (e.g., which may or may not be audible to normal human hearing). The audio tones may include one or more audio frequencies. The smart phone 604 listens for one or more audio tones while the audio is being output (e.g., via one or more microphones). For example, the smart phone 604 displays an interface 812 as shown in Figures 8C to 8D while listening for the audio tones, which is described in more detail below.

[0211] Figure 7E Shows the interface 740 displayed as a result of the home media center 602 responding to the successful completion of the audio timing synchronization process. The interface 740 includes the text "Audio synchronization complete" and "Your home media center has synchronized wireless audio to match your TV". The interface 740 also includes an indicator 740A (e.g., indicating the success of the audio timing synchronization process, when it includes a checkmark) and an enabling representation 740B (e.g., for terminating the process interface and returning to the interface 710).

[0212] In some embodiments, successfully completing the audio timing synchronization process includes receiving data from another device (e.g., a second device) (e.g., 604) participating in the audio timing synchronization process. For example, such data may include one or more of the following: a response indicating the detection of an audio tone, timing information about the audio tone (e.g., a timestamp when the audio tone was detected), and other information (e.g., characteristics of the detected audio tone). For example, after causing an audio tone to be output (e.g., when the interface 730 is displayed, as Figures 7C to 7D shown), the home media center 602 (e.g., via Wi-Fi communication) receives data from the smart phone 604 indicating that the smart phone 604 detected the tone and the timestamp associated with the detection of the tone (e.g., the time when the tone started being detected).

[0213] In some embodiments, successfully completing the audio timing synchronization process includes causing an adjustment to one or more audio time synchronization settings. For example, in response to receiving the data from the smart phone 604, the home media center 602 causes an adjustment to at least one audio timing synchronization setting. In this example, the home media center 602 uses the received data and determines the time delay between the timestamp when the home media center 602 outputs a signal representing the audio tone and the timestamp when the smart phone 604 detected the tone. This value gives the time delay between when the signal is output and when a listener (e.g., at the location of the smart phone 604 when the tone was detected, such as Figure 6 the location of the user in) will hear the corresponding audio output caused by the home media center 602. In this example, adjusting the audio timing synchronization setting includes adding a time delay to the audio output from the audio output devices 608A and 608B based on the data received from the smart phone 604. For example, the audio output caused by the speakers of the display device 606 may have a delay of five hundred milliseconds (e.g., from the output of the corresponding signal by the home media center 602 to the audio output reaching Figure 6at the location of the smart phone 604). If the audio output devices 608A and 608B are capable of outputting audio that reaches the smart phone 604 three hundred milliseconds after the home media center 602 outputs the corresponding signal to the display device 606, there is a synchronization difference of two hundred milliseconds (e.g., before performing an audio timing synchronization process to synchronize the time delay) (e.g., the sound from the audio output devices 608A and 608B reaches two hundred milliseconds before the audio is output by the display device 606 at the location of the device 604). Thus, a delay of two hundred milliseconds is introduced into the audio output of the devices 608A and 608B such that the audio output of the same corresponding audio signal sent to the display device 606 occurs five hundred milliseconds after the home media center 602 causes the corresponding audio signal to be output, thereby synchronizing the audio output of the respective audio output devices. Figure 6 at the location of the device 604). Thus, a delay of two hundred milliseconds is introduced into the audio output of the devices 608A and 608B such that the audio output of the same corresponding audio signal sent to the display device 606 occurs five hundred milliseconds after the home media center 602 causes the corresponding audio signal to be output, thereby synchronizing the audio output of the respective audio output devices.

[0214] In some embodiments, the audio output devices (e.g., 608A, 608B) have a known audio output delay. For example, since the audio output devices 608A and 608B are smart speakers (e.g., capable of synchronizing audio output between them and at least one other audio output device), their audio delays can be determined (e.g., by one of the audio output devices 608A or 608B, by the home media center 602). For example, the audio output device 608A can synchronize the timing of its audio output with the audio output device 608B, which involves the device 608A knowing the audio output timing. Thus, the audio output device 608A has the ability to determine a timestamp associated with its own audio output (e.g., which it uses to determine the time delay) or to determine the actual value of the time delay for itself. The time delay of the device's output audio may be due to various factors, including technical limitations. In the example provided above, the three hundred - millisecond delay represents a delay attributable to the transmission of received media content (e.g., from the home media center 602, from an Internet location), the processing of such media content, and other technology - based delays.

[0215] Figure 7FAn interface 750 is shown, which the home media center 602 causes to be displayed based on determining that the audio timing synchronization process has not been successfully completed. For example, after causing an audio tone to be output, the home media center 602 does not receive data from the smart phone 604 indicating that the audio tone has been detected by the smart phone 604 (or alternatively, receives an indication from the smart phone 604 that no tone has been detected or that a tone has been detected but the tone does not meet a certain quality threshold (e.g., detected at too low a volume)), so the home media center 602 causes the interface 750 to be displayed on the display device 606. The interface 750 includes the text "Tone not detected" and "Hold your device closer to your TV and adjust the volume if needed". The interface 750 includes an (optional) indicator 750A that graphically indicates that the process has not been successful (e.g., a depiction of the display device overlaid with the symbol "!"). The interface 750 includes an (optional) enablement representation 750B that, when selected, causes the home media center 602 to repeat the output of the audio tone (e.g., and causes the interface 730 to be displayed again as well).

[0216] In some embodiments, user input at another device that is participating in the audio timing synchronization process can cause the device (e.g., the home media center 602) to repeat causing the output of the audio tone. In some embodiments, user input for causing the repeated tone is received at the first device (e.g., the home media center 602) (e.g., only 602 displays an optional enablement representation for causing the repeated audio tone), at the second device (e.g., the smart phone 604) (e.g., only 604 displays an optional enablement representation for causing the repeated audio tone), or at the first device or the second device (e.g., both 602 and 604 cause optional options to be displayed on the respective display devices).

[0217] Figures 8A to 8F An exemplary interface for performing an audio timing synchronization process using a second device (e.g., 604) that is participating in the audio timing synchronization process with a first device (e.g., 602) is shown. In Figures 8A to 8F the example, the smart phone 604 is connected to the home media center 602, which is connected to the display device 606 and the audio output devices 608A - 608B, as described above with respect to Figure 6 described.

[0218] Figure 8AAn exemplary notification (e.g., 804) is shown that prompts acceptance of a request to participate in an audio timing synchronization process with a first device (e.g., 602). In some embodiments, in response to receiving a request to participate in an audio timing synchronization process (e.g., from a first device such as a home media center 602), a second device (e.g., 604) displays a notification (e.g., 804). In this example, notification 804 is displayed at the lock screen interface 802 on the display device 604A of the smart phone 604. Notification 804 includes the text "TV" and "Wireless Audio Sync" and "Tap to use the phone to measure how long it takes your home media center to play audio and video". As Figure 8A shown, the smart phone 604 receives a user input 806 (a tap on notification 804) indicating acceptance of the request to participate in an audio timing synchronization session with the first device (e.g., home media center 602). In some embodiments, in response to receiving a user input indicating acceptance of the request to participate in an audio timing synchronization session with the first device (e.g., 806), the second device (e.g., 604) transmits a response to the first device (e.g., 602). For example, in response to receiving user input 806, the smart phone 604 transmits an acceptance to the home media center 602 when the interface 720 ( Figure 7B ) is being displayed. In response to receiving the acceptance, the home media center 602 replaces interface 720 with interface 730 ( Figures 7C to 7D ). In some embodiments, in response to receiving a user input indicating acceptance of the request to participate in an audio timing synchronization session with the first device (e.g., 806), the second device (e.g., 604) causes an audio timing synchronization process interface (e.g., 808) to be displayed and, optionally, stops displaying or changes the appearance of the notification (e.g., 804). For example, in response to receiving user input 806, the smart phone 604 displays Figure 8B the audio timing synchronization process interface 808 in

[0219] Figure 8BAn exemplary audio timing synchronization process interface 808 is shown. As shown, the interface 808 is displayed as overlaid on the lock screen interface 802, which is also shown with an altered (e.g., dimmed) appearance (e.g., as indicated by the cross-hatching covering the notification 804). The interface 808 includes the text "Wireless Audio Sync" and "Measure how long it takes for your TV's audio to play." In some embodiments, the audio timing synchronization process interface (e.g., 808) includes an enabling representation (e.g., 808A) for continuing at least a portion of the audio timing synchronization process. For example, the interface 808 includes a continue enabling representation 808A. The interface 808 also includes a cancel enabling representation 808B (with the text "Not now"), which, when selected, causes the smart phone 604 to forego continuing the audio timing synchronization process and optionally exit / cancel the smart phone's participation in the audio timing synchronization process.

[0220] In some embodiments, a first device (e.g., 602) causes an audio tone to be output in accordance with an indication of a user input (e.g., 810) received at a second device (e.g., 604). For example, as described above, an exemplary first device (home media center 602) may wait for a user input at an exemplary second device (smart phone 604) before causing the audio tone to be output, thus giving the user an opportunity to control when the tone is output and the location of the second device (e.g., at a normal listening location in the home media system 600, such as the middle of a couch). Figure 8B Such an input 810 is shown. For example, when the smart phone 604 displays the interface 808, the home media center 602 causes the display of Figure 7B the interface 720, indicating that the user provide a user input (such as 810) at a nearby device (such as the smart phone 604). In response to the user input 810 at the smart phone 604, the home media center causes the display of Figure 7C the interface 730 and causes an audio tone to be output.

[0221] Figure 8C An exemplary audio timing synchronization process interface 812 is shown, which the smart phone 604 causes to be displayed while listening to the audio tone (e.g., output by the first device, which in this example is the home media center 602). For example, the smart phone 604 responds to receiving the user input 810 ( Figure 8B)and display interface 812. Interface 812 includes the text "Listening..." and "Your home media center is playing a tone from your speaker". Thus, interface 812 provides an indication that home media center 602 is currently causing an output audio tone (e.g., output by the speaker of display device 606). Interface 812 also includes graphical indication 812A, which visually indicates that an audio tone is being output (e.g., similar to that described above with respect to interface 730) and / or that smart phone 604 is listening to the audio tone. In this example, indication 812A includes a representation of the display device with concentric circles (e.g., representing sound waves) that emanate outward from the depicted display device in an animated fashion and increase in size (e.g., in Figure 8D which, Figure 8C the circle representing the sound wave has increased to a larger size, and additional smaller circles are shown in Figure 8D ).

[0222] Figure 8E An audio timing synchronization process interface 814 is shown, which smart phone 604 causes to be displayed in response to a successful completion of the audio timing synchronization process. Interface 814 includes the text "Audio synchronization complete". Interface 814 also includes indicator 814A (e.g., indicating that the audio timing synchronization process was successful, when it includes a check mark) and an enabling representation 814B (e.g., for terminating the process interface and returning to interface 802, as Figure 8A shown). In Figure 8E , smart phone receives user input 816 (a tap) representing a selection of the enabling representation 814B (e.g., such that smart phone 604 displays the lock screen interface 802 (without or without changing its appearance)).

[0223] In some embodiments, a successful completion of the audio timing synchronization process includes the transfer of data from a second device (e.g., 604) to a first device (e.g., 602) participating in the audio timing synchronization process. For example, such data may include one or more of the following: a response indicating detection of an audio tone, timing information about the audio tone (e.g., a timestamp when the audio tone was detected), and other information (e.g., characteristics of the detected audio tone). For example, after detecting an audio tone (e.g., when display interface 812 is shown as in Figures 8C to 8D ), smart phone 604 (e.g., via Wi-Fi communication) transfers data to home media center 602 indicating that smart phone 604 detected the tone and the timestamp associated with the detection of the tone (e.g., the time when the tone was first detected). In some embodiments, a successful completion of the audio timing synchronization process includes receiving an acknowledgement that such data was received by the first device (e.g., by 602).

[0224] In some embodiments, successful completion of the audio timing synchronization process includes causing an adjustment to one or more audio time synchronization settings. For example, a first device may cause the adjustment (e.g., the home media center 602, as described above). In some embodiments, a second device that detects an audio tone output causes an adjustment to the audio timing synchronization settings. For example, the smart phone 604 causes an adjustment to at least one audio timing synchronization setting, as described above with respect to the home media center 602 causing an adjustment to the audio timing synchronization settings. For example, the smart phone 604 may use data (e.g., from the audio tone, received from the first device (e.g., the home media center 602)) and determine the time delay between the timestamp when the home media center 602 outputs a signal representing the audio tone and the timestamp when the smart phone 604 detects the tone. In such examples, adjusting the audio timing synchronization settings includes adding the time delay to the audio output from the audio output devices 608A and 608B, similar to as described above with respect to Figure 7E described. In some embodiments, successful completion of the audio timing synchronization process includes receiving confirmation that the audio settings have been adjusted (e.g., adjusted by 602).

[0225] Figure 8F An audio timing synchronization process interface 818 is shown, which the smart phone 604 causes to be displayed based on determining that the audio timing synchronization process has not been successfully completed. For example, after the home media center 602 causes an audio tone to be output, the smart phone 604 fails to successfully detect the audio tone. In such a case, the smart phone 604 may transmit to the home media center 602 an indication that it did not detect the audio tone or that it detected the tone but the tone did not meet a certain quality threshold (e.g., detected at too low a volume). Alternatively, the smart phone 604 may forgo transmitting to the home media center 602 an indication of detecting the tone (e.g., in which case the home media center 602 infers that no tone has been detected after a certain timeout period). In the case where the audio timing synchronization process has not been successfully completed, the smart phone 604 causes the interface 818 to be displayed. The interface 818 includes the text "Tone not detected". The interface 818 includes an (optional) indicator 818A that graphically indicates that the process has not been successful (e.g., a depiction of a display device overlaid with the symbol "!"). The interface 818 includes an (optional) affordance representation 818B that, when selected, causes the home media center 602 to repeat outputting the audio tone (e.g., similar to the selection of the affordance representation 750B as described above with respect to Figure 7F . In some embodiments, a user input (e.g., Figure 7F the selection of the affordance representation 750B) at another device (e.g., the home media center 602) that is participating in the audio timing synchronization process may cause the device (e.g., the home media center 602) to repeat causing the audio tone to be output.

[0226] In the examples discussed above with respect to Figure 6 , Figures 7A to 7F , and Figures 8A to 8F the audio timing synchronization process is described as including at least a first device (e.g., 602) and a second device (e.g., 604). In some embodiments, the audio timing synchronization process includes two or more devices.

[0227] Although the interfaces described above with respect to Figures 7A to 7F are described as being caused to be displayed by the exemplary device home media center 602, one or more of such interfaces may be displayed (or caused to be displayed) by any device (e.g., participating in the audio timing synchronization process). Although the interfaces described above with respect to Figures 8A to 8F are described as being displayed on the exemplary device smart phone 604, one or more of such interfaces may be displayed (or caused to be displayed) by any device (e.g., participating in the audio timing synchronization process).

[0228] Figure 9 is a flowchart showing a method for performing an audio timing synchronization process using an electronic device according to some embodiments. Method 900 is performed at a first device (e.g., home media center, smart device) (e.g., 100, 300, 500) having a communication device (e.g., a transmitter and / or receiver circuit for communicating via wired and / or wireless transmission media (e.g., via Wi-Fi, Bluetooth, etc.)). Some of the operations in method 900 are optionally combined, the order of some operations is optionally changed, and some operations are optionally omitted.

[0229] As described below, method 900 provides an intuitive way to perform the audio timing synchronization process. The method reduces the cognitive burden on the user for performing the audio timing synchronization process, thereby creating a more effective human-machine interface. For battery-powered computing devices, enabling the user to perform the audio timing synchronization process more quickly and efficiently saves power and increases the interval between two battery charges.

[0230] A first device (e.g., 602) performs (902) the audio timing synchronization process.

[0231] During an audio timing synchronization process, a first device transmits (904) a request for a second device (e.g., 604) (e.g., a device connected to the same Wi-Fi (e.g., a smart phone, a tablet); a device within Bluetooth range) to participate in the audio timing synchronization process via a communication device. In some embodiments, the first device receives an acknowledgement (e.g., acceptance) that the second device will participate in the audio timing synchronization process. For example, the first device waits to receive an acknowledgement from the second device as a condition for proceeding with at least a portion of the audio timing synchronization process (e.g., for causing a user interface to be displayed (e.g., the next interface is a set of configuration user interfaces), for causing an audio tone to be output, etc.). In some embodiments, the first device receives a second acknowledgement (e.g., after the first acknowledgement) as a condition for proceeding with at least a portion (e.g., another portion) of the audio timing synchronization process. For example, the first device receives an acknowledgement that the second device will participate and a subsequent acknowledgement of when the second device is ready to listen to an audio tone. For example, a user input at the second device may accept the request for the second device to participate, and a subsequent user input at the second device may indicate that the audio tone should be played (e.g., giving the user an opportunity to locate the second device, reduce ambient noise, etc.).

[0232] In some embodiments, transmitting a request for the second device to participate in an audio timing synchronization process further includes the first device transmitting, via a communication device, a second request for a fourth device (e.g., 612) (e.g., a device outside the first device and outside the second device) to participate in the audio timing synchronization process (e.g., the same request as transmitted to the second device) (e.g., a request sent to multiple individual devices). In some embodiments, an acknowledgement or acceptance of participation in the audio timing synchronization process is received from the second device and / or the fourth device. For example, an acknowledgement / acceptance is received from multiple devices (e.g., selecting one or more of the accepting devices to perform the audio timing synchronization process). In some embodiments, a single device (e.g., the first to respond) is used in the audio timing synchronization process. In some embodiments, multiple devices are used in the audio timing synchronization process. In some embodiments, transmitting a request for multiple (e.g., nearby) devices to participate in the audio timing synchronization process provides an efficient configuration operation that increases the chance of finding devices to participate in the audio timing synchronization process, thereby reducing the chance of failure. Performing an optimization operation when a set of conditions have been met without requiring further user input enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the number of inputs required, and / or by helping the user provide appropriate inputs and reducing user errors when operating the device / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and effectively. In some embodiments, transmitting a request for multiple (e.g., nearby) devices to participate in the audio timing synchronization process may (e.g., to the user) provide an option for selecting which device will participate in the audio timing synchronization process with the first device, and allow selection of a preferred device or multiple devices to participate. Performing an optimization operation when a set of conditions have been met without requiring further user input enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the number of inputs required, and / or by helping the user provide appropriate inputs and reducing user errors when operating the device / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and effectively.

[0233] During the audio timing synchronization process, after the first device transmits the request to the second device (e.g., in response to a user input at the second device; in response to a handshake with the second device), the first device causes (906) an audio tone to be output (e.g., output by the first device, output by one or more other devices different from the first device and the second device, and / or output by some combination of the first device and the one or more other devices) (e.g., by a TV or an audio receiver).

[0234] In some embodiments, the first device is connected to a display device (e.g., 606) (e.g., a display device integrated into the first device, a display device connected to the first device by wire or wirelessly). In some embodiments, when causing an output audio tone, the first device causes a graphical indication (e.g., 730A) (e.g., an animation) regarding the output audio tone to be displayed via the display device (e.g., on the display device of the first device and / or an external device (e.g., TV)).

[0235] In some embodiments, the first device receives data representing a third user input (e.g., a selection enabling 710A, a selection enabling 810) (e.g., a user input at the first device, a user input at an external device (e.g., a second device) (e.g., a remote control, a smart phone) communicating with the first device). In some embodiments, at least a portion of the audio timing synchronization process is performed by the first device in response to receiving the data representing the third user input. In some embodiments, a first input is received when causing a display of an audio timing synchronization configuration interface. For example, when the first device causes an external display (e.g., a TV) to display an audio timing synchronization configuration interface. In some embodiments, the first input represents a selection of an enabling indication displayed at the audio timing synchronization configuration interface. In some embodiments, at least a portion of the audio timing synchronization process (e.g., providing to the user) is performed in response to a user input (e.g., received at the first device or the second device) to provide control for timing the audio timing synchronization process of the first device and may allow (e.g., by the user) selection of a preferred device to participate. Providing additional control options without cluttering the UI due to additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the number of required inputs, and / or by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0236] In some embodiments, in response to receiving data representing a third user input (e.g., in response to a selection of a notification displayed on a second device in response to the second device receiving a request to participate), a request for the second device to participate in the audio timing synchronization process (e.g., a request causing the display of notification 804) is transmitted.

[0237] In some embodiments, an audio tone is output in response to receiving data representing a third user input. In some embodiments, an audio tone used in an audio timing synchronization process is output in response to a user input (e.g., received at a second device) to provide control (e.g., to the user) for controlling the timing of the audio timing synchronization process audio output by the first device. Providing additional control options without cluttering the UI with additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by reducing the number of inputs required, and / or by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0238] In some embodiments, a third user input (e.g., selection of 710A) is received at the first device (e.g., via user interaction with the first device or with a remote control for controlling the first device and communicating with the first device).

[0239] In some embodiments, a third user input (e.g., 806, 810) is received at the second device (e.g., via user interaction with the second device or with a remote control for controlling the second device and communicating with the second device).

[0240] During the audio timing synchronization process, after causing an audio tone to be output, the first device receives (908) data (e.g., a timestamp when the audio tone is detected by the second device) from the second device based on the audio tone. In some embodiments, the data is the difference between two timestamps. For example, the first device provides a timestamp at the time of causing the output to the second device, and the second device determines the timestamp when the audio tone is detected and determines the difference between the two timestamps. In some embodiments, the first device determines the difference between the two timestamps (e.g., after the second device provides the first device with the timestamp when the second device detected the audio tone).

[0241] In some embodiments, a first device determines (e.g., calculates, looks up in a lookup table) a value of a time delay (e.g., a difference) between a first timestamp representing an output time corresponding to an audio tone and a second timestamp corresponding to a time when a second device detects the output audio tone. In some embodiments, the value representing the time delay is used to determine an adjusted audio timing synchronization setting for a third device. In some embodiments, the adjusted audio timing synchronization setting for the third device is a second time delay (e.g., the same as or different from the time delay represented by the value). For example, based on the value representing the time delay (e.g., the time it takes for a TV to output audio), a second time delay is used to delay the audio output of a third device (e.g., a smart speaker). In this example, the second time delay (e.g., at the first device, at the third device) is introduced into the audio output of the third device such that when the devices simultaneously output the same content, the audio outputs from the third device and another device (e.g., the TV) appear synchronized to a listener. In some embodiments, the second time delay is different from the time delay (e.g., the first time delay). For example, the second time delay may be less than the first time delay, reducing a certain amount of time delay (e.g., time delay resulting from technical reasons (including delays due to audio signal processing and transmission between the first device and the third device)). In some embodiments, a device determines a value of a time delay between when the second device hears an audio output (e.g., a tone) from one device and when the second device hears an audio output (e.g., a tone) output by another (different) device. For example, the time delay represents the time difference when audio from two sources arrives at the second device. In this example, such a time delay can be used to delay the audio output of the device whose audio arrives at the second device first.

[0242] During an audio timing synchronization process, a first device adjusts (910) an audio timing synchronization setting (e.g., adjusts or increases the delay of the output of one or more audio signals) of a third device (e.g., the first device, a third device different from the first and second devices) (e.g., a smart speaker not used in the synchronization process) at least in part based on data received from the second device. In some embodiments, adjusting the audio timing synchronization setting at least in part based on data received from the second device based on an audio tone output by the first device provides an effective configuration operation for correcting synchronization errors that requires minimal user input. Performing an optimization operation when a set of conditions has been met without requiring further user input enhances the operability of the device and makes the user-device interface more effective (e.g., by reducing the number of inputs required, and / or by helping the user provide appropriate inputs and reducing user errors when operating the device / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and effectively.

[0243] In some embodiments, adjusting the audio timing synchronization settings of a third device (e.g., 608A, 608B) includes the first electronic device transmitting (914) adjusted audio timing synchronization data (e.g., a new or updated time delay value) to the third device via a communication device. In some embodiments, the adjusted audio timing settings are maintained (e.g., stored thereon) by the first device. For example, the first device then uses the adjusted audio timing synchronization settings to affect the output / delivery of one or more audio signals to one or more devices (e.g., by the first device) (e.g., delaying the delivery of the output audio signal, introducing a shift of the output audio signal relative to a video signal, etc.).

[0244] In some embodiments, after an output audio tone is caused, the first device repeats (912) outputting the audio tone (e.g., one or more times) based on determining that a set of criteria indicating that a second device did not detect the audio tone (e.g., not detected via a microphone, or not detected with sufficient quality (e.g., without excessive noise) or not detected for a required length of time) is satisfied (e.g., when one or more of the following occur: (e.g., at the first device) receiving a user input (e.g., selecting "try again") or an indication of a user input (e.g., from the second device), receiving an indication from the second device that the tone was not detected or that an error occurred (e.g., hearing a partial tone, detecting audio interference), or not receiving data / response from the second device within a predetermined period of time (e.g., a response timeout period)). In some embodiments, if the set of criteria continues to be satisfied (e.g., after each repetition), the audio tone is repeated a predetermined number of times (e.g., three times). In some embodiments, repeating the output audio tone if a set of criteria is satisfied provides an effective configuration operation for handling errors that requires minimal user input. Performing an optimization operation when a set of conditions has been satisfied without the need for further user input enhances the operability of the device and makes the user-device interface more effective (e.g., by reducing the number of inputs required, and / or by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and effectively.

[0245] In some embodiments, the set of criteria includes criteria that are satisfied when a first user input indicating a request for repeating the output audio tone (e.g., selection of the enabling representation 750B) (e.g., an input indicating a selection of an enabling representation (e.g., try again) at a displayed interface (e.g., caused to be displayed by the first device)) is received at the first device.

[0246] In some embodiments, the set of criteria includes criteria that are satisfied when a second user input indicating a request for a repeated output audio tone (e.g., a selection of the enabling representation 818B) (e.g., an input indicating a selection of an enabling representation (e.g., try again) at a displayed interface (e.g., caused to be displayed by the first device)) is received at the second device.

[0247] In some embodiments, a third device (e.g., 608A, 608B, smart speaker) is different from the first device and different from the second device. In some embodiments, the first device causes adjustment of the audio timing synchronization settings of multiple devices. For example, different respective delays may be configured for delivering respective audio signals (e.g., corresponding to different audio channels) to different devices (e.g., speakers) that require audio timing synchronization. In some embodiments, adjusting the audio timing synchronization settings for a third device that is not the first device or the second device participating in the audio timing synchronization process provides an effective configuration operation for correcting synchronization errors that requires minimal (e.g., no) user input at the third device (e.g., which may not have a user-friendly interface). Performing an optimization operation when a set of conditions has been met without requiring further user input (e.g., at the configured devices) enhances the operability of the devices and makes the user-device interface more effective (e.g., by reducing the number of inputs required, and / or by helping the user provide appropriate inputs and reducing user errors when operating / interacting with the devices), which in turn reduces power usage and extends the battery life of the devices by enabling the user to use the devices more quickly and effectively.

[0248] In some embodiments, during a process for configuring multiple settings of the first device, the first device automatically performs an audio timing synchronization process (e.g., without requiring / without directly responding to a user input received at the first device or another device) (e.g., without displaying an audio timing synchronization configuration interface) (e.g., automatically transmits a request for participation to one or more devices during the configuration process and then receives data from such devices based on the output audio tones and adjusts the audio timing synchronization settings).

[0249] For purposes of explanation, the foregoing description has been described by reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of the techniques and their practical applications. Other technicians in the art can thereby best utilize the techniques and various embodiments with various modifications suitable for the particular uses contemplated.

[0250] While the present disclosure and the examples have been described in full detail with reference to the accompanying drawings, it should be noted that various changes and modifications will become apparent to those skilled in the art. It should be understood that such changes and modifications are considered to be included within the scope of the present disclosure and examples as defined by the claims.

[0251] As described above, one aspect of the present technology is the collection and use of data obtained from various sources to improve audio synchronization. The present disclosure contemplates that, in some instances, the collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, telephone numbers, email addresses, Twitter IDs, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0252] The present disclosure recognizes that the use of such personal information data in the present inventive technology can be used to benefit users. For example, personal information data can be used to improve audio synchronization (e.g., for media accessed using a personal account information). In addition, the present disclosure also contemplates other uses of personal information data that are beneficial to users. For example, health and fitness data can be used to provide insights into a user's overall health condition, or can be used as positive feedback for individuals using the technology to pursue health goals.

[0253] The present disclosure contemplates that entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be readily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of those legitimate uses. In addition, such collection / sharing should occur after receiving informed consent from the user. Further, such entities should consider taking any necessary steps to safeguard and secure access to such personal information data and to ensure that others with access to personal information data comply with their privacy policies and procedures. Additionally, such an entity may subject itself to third-party assessments to demonstrate its compliance with widely accepted privacy policies and practices. Further, policies and practices should be adjusted to account for the specific types of personal information data being collected and / or accessed and to apply applicable laws and standards, including specific considerations of the jurisdiction. For example, in the United States, the collection or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Thus, different privacy practices should be maintained for different types of personal data in each country.

[0254] Notwithstanding the foregoing, the present disclosure also anticipates embodiments where users selectively block the use or access of personal information data. That is, the present disclosure anticipates that hardware elements and / or software elements may be provided to prevent or block access to such personal information data. For example, in the case of an advertising delivery service, the techniques of the present invention may be configured to allow a user to select to "opt-in" or "opt-out" of participating in the collection of personal information data during or at any time after registering for the service. As another example, a user may choose not to provide account information for access to a particular audio media. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notice relating to the access or use of personal information. For example, a user may be notified at the time of downloading an application that their personal information data will be accessed and then reminded again just before the personal information data is accessed by the application.

[0255] In addition, it is an object of the present disclosure to manage and process personal information data to minimize the risk of inadvertent or unauthorized access or use. The risk can be minimized by restricting data collection and deleting data once it is no longer needed. Additionally, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of the data stored (e.g., collecting location data at the city level rather than at the address level), controlling how the data is stored (e.g., aggregating data among users), and / or other methods, as appropriate.

[0256] Accordingly, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also anticipates that the various embodiments can also be implemented without access to such personal information data. That is, the various embodiments of the inventive technology will not fail to operate properly due to the lack of all or a portion of such personal information data. For example, preferences can be inferred by relying on non-personal information data or an absolute minimum amount of personal information such as the content requested by a device associated with the user, other non-personal information available to a content delivery service, or publicly available information, and content can be selected and delivered to the user accordingly.

Claims

1. A computer-implemented method, comprising: At a first device having a communication device: Performing an audio timing synchronization process, the audio timing synchronization process comprising: Transmitting, via the communication device, a request for the second device to participate in the audio timing synchronization process to the second device; After transmitting the request to the second device, receiving, from the second device, an acknowledgement of participation in the audio timing synchronization process; After receiving the acknowledgement, causing an audio tone to be output from a reference device different from the first device and the second device; After causing the audio tone to be output from the reference device, receiving, from the second device, data generated based on the output of the audio tone from the reference device; and Adjusting, at least in part, an audio timing synchronization setting of a third device based on the data received from the second device, the data received from the second device including a time delay between a first timestamp and a second timestamp, the first timestamp corresponding to the time at which the second device detected the output audio tone, and the second timestamp corresponding to the time of output of the audio tone by the reference device.

2. The method according to claim 1, further comprising: After causing the audio tone to be output: Repeating the output of the audio tone according to a set of criteria determined to indicate that the second device did not detect the audio tone being satisfied.

3. The method according to claim 2, wherein the set of criteria includes criteria that are satisfied when a first user input indicating a request for repeating the output of the audio tone is received at the first device.

4. The method according to claim 2, wherein the set of criteria includes criteria that are satisfied when a second user input indicating a request for repeating the output of the audio tone is received at the second device.

5. The method according to claim 1, wherein the third device is different from the first device and different from the second device.

6. The method according to claim 1, wherein adjusting the audio timing synchronization setting of the third device comprises: Transmitting, via the communication device, adjusted audio timing synchronization data to the third device.

7. The method according to claim 1, wherein the first device is connected to a display device, the method further comprising: When causing the audio tone to be output, causing a graphical indication that the audio tone is being output to be displayed via the display device.

8. The method according to claim 1, wherein transmitting the request for the second device to participate in the audio timing synchronization process further comprises: Transmitting, via the communication device, a second request for a fourth device to participate in the audio timing synchronization process to the fourth device.

9. The method according to claim 1, further comprising: Receiving data representing a first user input, wherein in response to receiving the data representing the first user input, at least a part of the audio timing synchronization process is performed.

10. The method according to claim 9, wherein in response to receiving the data indicating the first user input, a request for the second device to participate in the audio timing synchronization process is transmitted.

11. The method according to claim 9, wherein in response to receiving the data indicating the first user input, the audio tone is output.

12. The method according to claim 9, wherein the first user input is received at the first device.

13. The method according to claim 9, wherein the first user input is received at the second device.

14. The method according to claim 1, further comprising: Automatically performing the audio timing synchronization process when executing a process for configuring a plurality of settings of the first device.

15. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs being configured to be executed by one or more processors of an electronic device having a communication device, the one or more programs including instructions for performing the method according to any one of claims 1 to 14.

16. An electronic device, comprising: A communication device; One or more processors; And A memory storing one or more programs, the one or more programs being configured to be executed by the one or more processors, the one or more programs including instructions for performing the method according to any one of claims 1 to 14.

17. A computer program product, comprising: A computer program including instructions that, when run, cause the electronic device to perform the method according to any one of claims 1 to 14.

Images