Systems, methods, and user interfaces for headphone fit adjustment and audio output control

Abstract

The disclosure relates to systems, methods, and user interfaces for headphone fit adjustment and audio output control. A wearable audio output device in a physical environment includes an input device and one or more microphones. When ambient sound from a physical environment is being detected by the microphone (s), the wearable audio output device: provides a first audio output when in a first mode, the first audio output comprising one or more pass-through audio components selected to increase audio pass-through of the ambient sound; detecting an input via the input device; transitioning from the first mode to a second mode in response to detecting the input and in accordance with a determination that the input is a first type of gesture; and providing, while in the second mode, a second audio output comprising one or more cancellation audio components selected to increase audio cancellation of the ambient sound.

Description

[0001] This application is a divisional application of the Chinese invention patent application with an international application date of July 7, 2020, national application number 202080049377.X, and invention name “System, method and user interface for headphone fit adjustment and audio output control”. Technical Field

[0002] The present invention generally relates to audio output devices, such as wearable audio output devices, including but not limited to wearable audio output devices in which the fit of the wearable audio output device in a user's ear can be adjusted, and in which audio output control can be performed using input at the wearable audio output device. Background Art

[0003] Audio output devices (including wearable audio output devices, such as earbuds and headphones) are widely used to provide audio output to users. However, conventional methods of providing audio output using audio output devices are cumbersome, inefficient, and limited. In some cases, conventional methods fail to ensure that the wearable audio output device is properly calibrated and fits the user's ear (e.g., to achieve effective active noise control), and that information about the fit of the wearable audio output device is effectively communicated to the user. In some cases, conventional methods fail to ensure that the wearable audio output device continues to fit the user's ear as the user performs various activities over time. In some cases, input provided at the wearable audio output device provides limited control of the audio output; for example, the input may be limited to controlling a single predefined feature of the audio output, such as increasing the output volume or switching a feature on or off. In some cases, this limited control over the audio output interferes with the user's ability to control the amount of sound that the user can hear from the surrounding physical environment when wearing the earbuds or headphones. In other cases, when the wearable audio output device is not placed in both ears, the control of the audio output provided by inputs provided at the wearable audio output device can result in undesirable acoustic effects. Furthermore, in some cases, it would be beneficial to automatically change the manner in which the audio output is provided in response to certain types of events occurring relative to the wearable audio output device, but conventional approaches provide audio output in a static manner, independent of the events occurring relative to the wearable audio output device. In some cases, the user interface for controlling audio output settings provides too few controls, such as providing controls for only one audio output device, thereby requiring the user to provide numerous inputs and navigate through different menus or user interfaces to perform specific actions, or provides too many controls, thereby cluttering the user interface and increasing the likelihood that the user will mistakenly interact with the wrong control, particularly for implementations where display area is limited. Furthermore, conventional approaches take longer than necessary to calibrate these wearable audio output devices and control the audio output, and require more user interaction, thereby wasting energy. This latter consideration is particularly important in battery-powered devices. Summary of the Invention

[0004] Therefore, there is a need for wearable audio output devices and associated computer systems with improved methods and interfaces for determining and adjusting the fit of wearable audio output devices and for controlling audio output using input at the wearable audio output device. Such methods and interfaces optionally supplement or replace conventional methods of calibrating audio output devices and controlling audio output. Such methods and interfaces reduce the number, extent, and / or nature of input from the user and produce a more efficient human-computer interface. For battery-powered systems and devices, such methods and interfaces conserve power and increase the time between battery charges.

[0005] With the disclosed computer system and wearable audio output device, the above-mentioned defects and other problems associated with calibrating audio output devices and controlling audio output are reduced or eliminated. In some embodiments, the computer system includes a desktop computer. In some embodiments, the computer system is portable (e.g., a laptop, tablet computer, or handheld device). In some embodiments, the computer system includes a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the computer system includes a wearable audio output device (e.g., in-ear headphones, earbuds, over-ear headphones, etc.) (and / or communicates with it). In some embodiments, the computer system has a touch-sensitive surface (also referred to as a "touchpad") (and / or communicates with it). In some embodiments, the computer system has a display device, which in some embodiments is a touch-sensitive display (also referred to as a "touch screen" or "touch screen display") (and / or communicates with it). In some embodiments, the computer system has a graphical user interface (GUI), one or more processors, memory, and one or more modules, a program or instruction set stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI primarily through stylus and / or finger contact and gestures on the touch-sensitive surface. In some embodiments, these functions optionally include image editing, drawing, presentations, word processing, spreadsheet creation, playing games, making and receiving calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, audio output device pairing and calibration, digital music / audio playback, note taking and / or digital video playback. Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0006] According to some embodiments, a method is performed at a computer system that includes a display device and a touch-sensitive surface. The method includes establishing a wireless connection with a pair of wearable audio output devices, the pair of wearable audio output devices including: a first wearable audio output device having one or more first microphones; and a second wearable audio output device having one or more second microphones. The method includes: detecting that the first wearable audio output device has been placed in an ear of a user; and detecting that the second wearable audio output device has been placed in an ear of the user. The method includes, after detecting that the first wearable audio output device has been placed in the ear of the user and the second wearable audio output device has been placed in the ear of the user: outputting a first calibration tone via the first wearable audio output device and the second wearable audio output device; detecting first audio via the one or more first microphones of the first wearable audio output device; and detecting second audio via the one or more second microphones of the second wearable audio output device. The method includes, based on determining that the detected first audio does not meet the device fit standard associated with the first calibration tone, displaying an alert via a display device prompting a user to perform an adjustment on a first wearable audio output device; and based on determining that the detected second audio does not meet the device fit standard associated with the first calibration tone, displaying an alert via the display device prompting the user to perform an adjustment on a second wearable audio output device.

[0007] According to some embodiments, a method is performed at a computer system including a display device and in communication with one or more wearable audio output devices. The computer system is configured to execute multiple applications, and the one or more wearable audio output devices include one or more sensors and one or more microphones for detecting placement of the one or more wearable audio output devices. The method includes, when the one or more wearable audio output devices are in one or more corresponding positions relative to a user's ears and a media presentation application on the computer system is being used to play media via the one or more wearable audio output devices without displaying a settings user interface for configuring a fit of the one or more wearable audio output devices: providing, via the one or more wearable audio output devices, audio output based on media from the media presentation application, wherein the media presentation application is separate from the settings user interface; determining, based on the audio output based on the media from the media presentation application, that the one or more wearable audio output devices no longer meet device fit criteria; and, in response to determining that the one or more wearable audio output devices no longer meet the device fit criteria, displaying an alert on the display device corresponding to information regarding the fit of the one or more wearable audio output devices.

[0008] According to some embodiments, a method is performed at a wearable audio output device that includes an input device and one or more microphones and is located in a physical environment. The method includes, while ambient sound from the physical environment is being detected by the one or more microphones: providing a first audio output based at least in part on the ambient sound from the physical environment while the wearable audio output device is in a first audio output mode, wherein the first audio output includes one or more pass-through audio components selected to increase audio pass-through of the ambient sound from the physical environment; detecting a first input via the input device, and in response to detecting the first input and based on determining that the first input is a first type of gesture, transitioning the wearable audio output device from the first audio output mode to a second audio output mode. The method includes, while the wearable audio output device is in the second audio output mode, providing a second audio output based at least in part on the ambient sound from the physical environment, wherein the second audio output includes one or more cancellation audio components selected to increase audio cancellation of the ambient sound from the physical environment.

[0009] According to some embodiments, a method is performed at a computer system including a wearable audio output device and one or more input devices, wherein the wearable audio output device is in a physical environment. The method includes operating the wearable audio output device in a first audio output mode; and while operating the wearable audio output device in the first audio output mode, receiving, via the one or more input devices, a first input corresponding to a request to transition the wearable audio output device from the first audio output mode to a noise cancellation mode. When the wearable audio output device is in the noise cancellation mode, audio output provided via the wearable audio output device includes one or more cancellation audio components selected to at least partially cancel ambient sounds from the physical environment. The method includes, in response to receiving a first input: transitioning the wearable audio output device from a first audio output mode to a noise cancellation mode based on a determination that a first wearable audio output component of the wearable audio output device is in an in-ear position relative to a first ear of a user and a second wearable audio output component of the wearable audio output device is in an in-ear position relative to a second ear of the user; and forgoing transitioning the wearable audio output device from the first audio output mode to the noise cancellation mode based on a determination that one of the first wearable audio output component or the second wearable audio output component is not in the in-ear position relative to the corresponding ear of the user.

[0010] According to some embodiments, a method is performed at a computer system including a wearable audio output device in a physical environment. The method includes operating the wearable audio output device in a first audio output mode while a first wearable audio output component of the wearable audio output device is in a first position relative to a first ear of a user and a second wearable audio output component of the wearable audio output device is in a first position relative to a second ear of the user, and while operating the wearable audio output device in the first audio output mode, detecting a position change of the first wearable audio output component from a first position relative to the first ear of the user to a second position relative to the first ear of the user. The method includes, in response to detecting the position change of the first wearable audio output component from the first position relative to the first ear of the user to the second position relative to the first ear of the user while the second wearable audio output component remains in the first position relative to the second ear of the user: transitioning the wearable audio output device from the first audio output mode to a second audio output mode, the second audio output mode being a pass-through audio output mode different from the first audio output mode. When the wearable audio output device is in the pass-through audio output mode, audio output provided via the wearable audio output device includes one or more pass-through audio components having at least a portion of ambient sound from the physical environment.

[0011] According to some embodiments, a method is performed at a computer system including a display device and a wearable audio output device. The wearable audio output device includes a first wearable audio output component and a second wearable audio output component. The method includes detecting an occurrence of a corresponding event; and in response to detecting the occurrence of the corresponding event, displaying acoustic seal information of the wearable audio output device based on a determination that the first wearable audio output component is at least partially in a first ear of a user and the second wearable audio output component is at least partially in a second ear of the user, including simultaneously displaying via the display device: a first indication of the quality of a first acoustic seal between the first wearable audio output component and the user's first ear; and a second indication of the quality of a second acoustic seal between the second wearable audio output component and the user's second ear that is different from the first indication.

[0012] According to some embodiments, a method is performed at a computer system that includes a display device and is in communication with one or more groups of wearable audio output devices. The method also includes receiving a first input corresponding to a request to display an audio output settings user interface; and in response to receiving the first input, displaying the audio output settings user interface. Based on determining that the computer system is in communication with at least a first group of one or more wearable audio output devices and a second group of one or more wearable audio output devices, the audio output settings user interface includes: a first set of audio output controls corresponding to the first group of wearable audio output devices, including a first volume control indicating a current output volume level of the first group of wearable audio output devices, and a representation of a first audio output mode that is a current audio output mode among a first plurality of audio output modes available at the first group of wearable audio output devices, wherein the representation of the first audio output mode is visually associated with the first volume control; and a second set of audio output controls corresponding to the second group of wearable audio output devices, including a second volume control indicating the current output volume level of the second group of wearable audio output devices.

[0013] According to some embodiments, a method is performed on a wearable audio output device having a rotatable input mechanism. The method includes outputting first audio based on first media via the wearable audio output device. The method includes receiving a first input via the rotatable input mechanism while outputting the first audio. The method includes, in response to receiving the first input: upon determining that the first input is a first type of input to the rotatable input mechanism comprising rotation of the rotatable input mechanism, changing an audio output volume of the first audio based on the rotation of the rotatable input mechanism while continuing to output the first audio; and upon determining that the first input is a second type of input to the rotatable input mechanism, wherein the second type of input is different from the first type of input, ceasing to output the first audio.

[0014] According to some embodiments, a computer system includes one or more wearable audio output devices, a display device, optionally a touch-sensitive surface, one or more processors, and a memory storing or communicating with one or more programs; the one or more programs are configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the execution of the operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium has instructions stored therein that, when executed by the computer system described herein, cause the computer system to perform or cause the execution of the operations of any of the methods described herein. According to some embodiments, a graphical user interface on a computer system described herein (e.g., on an electronic device) includes one or more elements of the elements displayed in any of the methods described herein, which are updated in response to input, as described in any of the methods described herein. According to some embodiments, a computer system described herein includes a device for performing or causing the execution of the operations of any of the methods described herein. According to some embodiments, an information processing device for use in a computer system described herein includes a device for performing or causing the execution of the operations of any of the methods described herein.

[0015] According to some embodiments, a wearable audio output device as described herein includes one or more microphones, optionally an input device (e.g., which may be pressure-sensitive and / or touch-sensitive), optionally one or more accessories (e.g., in-ear earbuds), optionally one or more sensors for detecting placement of the wearable audio output device, one or more processors, and a memory storing one or more programs; the one or more programs are configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the performance of the operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium has instructions stored therein that, when executed by the wearable audio output device described herein, cause the wearable audio output device to perform or cause the performance of the operations of any of the methods described herein. According to some embodiments, the wearable audio output device described herein includes means for performing or causing the performance of the operations of any of the methods described herein. According to some embodiments, an information processing device for use in the wearable audio output device described herein includes means for performing or causing the performance of the operations of any of the methods described herein.

[0016] Thus, a computer system with improved methods and interfaces for adjusting the fit of a wearable audio output device and for controlling audio output using input at the wearable audio output device is provided, the computer system comprising one or more wearable audio output devices, a display device, and optionally a touch-sensitive surface, and a wearable audio output device comprising one or more microphones, optionally an input device (e.g., which may be pressure-sensitive and / or touch-sensitive), optionally one or more accessories (e.g., in-ear eartips), and optionally one or more sensors for detecting placement of the wearable audio output device, or communicating with the same, thereby improving the effectiveness, efficiency, and user satisfaction of such systems and devices. Such methods and interfaces can supplement or replace conventional methods for calibrating audio output devices and controlling audio output. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0018] Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display according to some embodiments.

[0019] Figure 1B is a block diagram illustrating example components for event processing according to some embodiments.

[0020] Figure 2 A portable multifunction device with a touch screen according to some embodiments is shown.

[0021] Figure 3A is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.

[0022] Figure 3B is a block diagram of an exemplary wearable audio output device according to some embodiments.

[0023] Figures 3C to 3D Example audio control by a wearable audio output device according to some embodiments is shown.

[0024] Figure 4A An exemplary user interface for an application menu on a portable multifunction device according to some embodiments is shown.

[0025] Figure 4B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0026] Figure 5A to Figure 5VAn exemplary user interface for pairing and calibrating a wearable audio output device according to some embodiments is shown.

[0027] 6A to 6O An exemplary user interface is shown for alerting a user when a wearable audio output device no longer meets fit criteria during use, according to some embodiments.

[0028] Figures 7A to 7Q An exemplary user interface and user interaction for changing the audio output mode of a wearable audio output device according to some embodiments are shown.

[0029] Figures 8A to 8J Example user interactions with a wearable audio output device for controlling audio output are shown according to some embodiments.

[0030] Figures 9A to 9C An exemplary settings user interface for controlling various features associated with a wearable audio output device and examples of controlling audio output modes are shown according to some embodiments.

[0031] Figures 10A to 10H is a flowchart of a process for pairing and calibrating a wearable audio output device according to some embodiments.

[0032] 11A to 11D is a flow chart of a process for monitoring the fit of a wearable audio output device during use, according to some embodiments.

[0033] 12A to 12E is a flowchart of a process for controlling audio output using input at a wearable audio output device according to some embodiments.

[0034] 13A to 13D An exemplary user interface for pairing a wearable audio output device according to some embodiments is shown.

[0035] Figures 14A to 14Z An exemplary user interface and user interaction for changing the audio output mode of a wearable audio output device according to some embodiments are shown.

[0036] Figures 15A to 15E An exemplary settings user interface for controlling various features associated with a wearable audio output device and examples of controlling audio output modes are shown according to some embodiments.

[0037] Figures 16A to 16G An exemplary user interface for calibrating the acoustic seal quality of a wearable audio output device and displaying information about the same is shown according to some embodiments.

[0038] Figures 17A to 17J Exemplary user interfaces and user interactions are shown for selecting an audio output device, such as a wearable audio output device, and changing the audio output mode of the selected audio output device via the wearable electronic device, according to some embodiments.

[0039] 18A to 18H is a flowchart of a process for controlling an audio output mode of a wearable audio output device according to some embodiments.

[0040] 19A to 19C is a flowchart of a process for transitioning an audio output mode of a wearable audio output device in response to a detected event according to some embodiments.

[0041] FIG. 20A to FIG. 20B is a flowchart of a process for displaying information about the quality of an acoustic seal of a wearable audio output device according to some embodiments.

[0042] Figures 21A to 21J An exemplary user interface and user interaction for changing the audio output mode of a wearable audio output device according to some embodiments are shown.

[0043] 22A to 22D Example user interfaces and user interactions are shown for accessing a settings user interface for controlling various features associated with a wearable audio output device according to some embodiments.

[0044] Figures 23A to 23AA Example user interfaces and user interactions are shown for controlling audio output settings of multiple wearable audio output devices using a single settings user interface according to some embodiments.

[0045] 24A to 24D is a flowchart of a process for controlling audio output settings of multiple wearable audio output devices using a single settings user interface according to some embodiments.

[0046] Figures 25A to 25G Example user interactions for controlling audio output from a wearable audio output device and example audio alerts and visual alerts in response to such user interactions are shown according to some embodiments.

[0047] 26A to 26D is a flowchart of a process for controlling audio output from a wearable audio output device according to some embodiments. DETAILED DESCRIPTION

[0048] As described above, audio output devices, such as wearable audio output devices, are widely used to provide audio output to users. Many computer systems that include or communicate with wearable audio output devices fail to ensure that the wearable audio output device is properly calibrated and remains in place against the user's ear, or give the user only limited control over the audio output in response to input at the wearable audio output device, or provide too few or too many audio output controls for the user interface. The methods, systems, and user interfaces / interactions described herein improve how audio output is provided in a variety of ways. For example, the embodiments disclosed herein describe improved methods for determining and adjusting the fit of a wearable audio output device, controlling audio output using input at the wearable audio output device, and providing an improved user interface for controlling audio output settings.

[0049] under, Figure 1A to Figure 1B 、 Figure 2 as well as Figures 3A to 3D Descriptions of examples of exemplary devices and their operation are provided. Figures 4A to 4B An exemplary user interface of an exemplary device on which embodiments disclosed herein are implemented is shown. Figure 5A to Figure 5V An exemplary user interface for pairing and calibrating a wearable audio output device is shown. 6A to 6O An example user interface is shown for alerting a user when a wearable audio output device no longer meets fit criteria during use. Figures 7A to 7Q Exemplary user interfaces and user interactions for changing the audio output mode of a wearable audio output device are shown. Figures 8A to 8J Exemplary user interactions with a wearable audio output device for controlling audio output are shown. Figures 9A to 9C An exemplary settings user interface for controlling various features associated with a wearable audio output device and examples of controlling audio output modes are shown. Figures 10A to 10H is a flow chart of the process for pairing and calibrating a wearable audio output device. 11A to 11D is a flow chart of a process for monitoring the fit of a wearable audio output device during use. 12A to 12E is a flow chart of a process for controlling audio output using input at a wearable audio output device. 13A to 13D An exemplary user interface for pairing a wearable audio output device is shown. Figures 14A to 14Z Exemplary user interfaces and user interactions for changing the audio output mode of a wearable audio output device are shown. Figures 15A to 15E An exemplary settings user interface for controlling various features associated with a wearable audio output device and examples of controlling audio output modes are shown. Figures 16A to 16G An exemplary user interface for calibrating the acoustic seal quality of a wearable audio output device and displaying information thereabout is shown. Figures 17A to 17J Exemplary user interfaces and user interactions are shown for selecting an audio output device, such as a wearable audio output device, and changing an audio output mode of the selected audio output device via (eg, using) a wearable electronic device. Figures 21A to 21J Exemplary user interfaces and user interactions for changing the audio output mode of a wearable audio output device are shown. 22A to 22D Exemplary user interfaces and user interactions are shown for accessing a settings user interface for controlling various features associated with a wearable audio output device. Figures 23A to 23AA Exemplary user interfaces and user interactions are shown for controlling audio output settings of multiple wearable audio output devices using a single settings user interface. Figures 25A to 25G Exemplary user interactions for controlling audio output from a wearable audio output device and exemplary audio and visual alerts in response to such user interactions are shown. Figure 5A to Figure 5V 、 6A to 6O 、 Figures 7A to 7Q 、 Figures 8A to 8J 、 Figures 9A to 9C 、 13A to 13D 、 Figures 14A to 14Z 、 Figures 15A to 15E 、 Figures 16A to 16G 、 Figures 17A to 17J 、 Figures 21A to 21J 、 22A to 22D 、 Figures 23A to 23AA as well as Figures 25A to 25G The user interface and user interaction in Figures 10A to 10H 、 11A to 11D 、 12A to 12E 、 18A to 18H 、 19A to 19C 、 FIG. 20A to FIG. 20B 、 24A to 24D as well as 26A to 26D in the process.

[0050] Exemplary devices

[0051] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure various aspects of the embodiments.

[0052] It will also be understood that, although in some cases, the terms "first," "second," etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are merely used to distinguish one element from another. For example, a first audio output can be named a second audio output and similarly, a second audio output can be named a first audio output without departing from the scope of the various described embodiments. A first audio output and a second audio output are both audio outputs, but they are not the same audio output unless the context clearly indicates otherwise.

[0053] The terms used in the description of the various embodiments described herein are only for the purpose of describing specific embodiments and are not intended to be limiting. As used in the description of the various embodiments described and in the appended claims, the singular forms "a" and "the" are intended to also include plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "and / or" used herein refer to and encompass any and all possible combinations of one or more items in the associated listed items. It will also be understood that the terms "includes," "including," "comprises," and / or "comprising" when used in this specification specify the presence of stated features, integers, steps, operations, elements, and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.

[0054] As used herein, the term "if" is optionally interpreted to mean "when" followed by "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [stated condition or event] is detected" are optionally interpreted to mean "upon determining" or "in response to determining" or "upon detecting [stated condition or event]" or "in response to detecting [stated condition or event]," depending on the context.

[0055] The present invention describes embodiments of computer systems that include or communicate with wearable audio output devices, user interfaces for such systems, and associated processes for using such systems and devices. In some embodiments, the computer system includes a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple Watch from Apple Inc. (Cupertino, California). iPod and Device. Other portable electronic devices, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or trackpads) are optionally used. It should also be understood that in some embodiments, the computer system does not include a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or trackpads).

[0056] In the following discussion, a computer system comprising an electronic device having a display device and a touch-sensitive surface is described. However, it should be understood that the computer system optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0057] The computer system typically supports a variety of applications, such as one or more of the following: a note-taking application, a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0058] Various applications executed on the computer system optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the computer system are optionally adjusted and / or varied for different applications and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the computer system (such as the touch-sensitive surface) optionally supports the various applications with a user interface that is intuitive and clear to the user.

[0059] Attention is now turned to embodiments of portable devices having touch-sensitive displays. Figure 1A1 is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a "touch screen" for convenience, and sometimes simply as a touch-sensitive display. The device 100 includes memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and external ports 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0060] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of a device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device that will be detected by a user using the user's sense of touch. For example, when a device or a component of the device is in contact with a surface that is touch-sensitive to a user (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" on a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click," even when the physical actuation button associated with the touch-sensitive surface that was physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the touch-sensitive surface does not change, movement of the touch-sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface. Although such interpretation of touch by the user will be limited by the user's individualized sensory perception, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click," "release click," "roughness"), unless otherwise stated, the tactile output generated corresponds to a physical displacement of the device or a component thereof that would generate the sensory perception of a typical (or average) user. Providing tactile feedback to the user using tactile output enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0061] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the motion waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.

[0062] When a device (e.g., one or more tactile output generators that generate tactile output via movement of a movable mass) generates tactile output with different tactile output patterns, the tactile output can produce different tactile sensations in a user holding or touching the device. While a user's senses are based on their perception of the tactile output, most users will be able to discern changes in the waveform, frequency, and amplitude of the tactile output generated by the device. Therefore, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different operations have been performed. Thus, tactile output with tactile output patterns designed, selected, and / or arranged to simulate the characteristics (e.g., size, material, weight, stiffness, smoothness, etc.); behavior (e.g., oscillation, displacement, acceleration, rotation, extension, etc.); and / or interaction (e.g., collision, adhesion, repulsion, attraction, friction, etc.) of objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and / or any combination of the foregoing) can, in some cases, provide helpful feedback to the user, reducing input errors and improving the user's efficiency in operating the device. In addition, tactile output is optionally generated to correspond to feedback that is unrelated to the simulated physical properties (such as input threshold or object selection). Such tactile output will provide helpful feedback to the user in some cases, which reduces input errors and improves the efficiency of the user's operation of the device.

[0063] In some embodiments, a tactile output with a suitable tactile output pattern serves as a prompt for an event of interest in the user interface or behind the screen in the device. Examples of events of interest include activation of an affordance (e.g., a real or virtual button or toggle switch) provided on the device or in the user interface, the success or failure of a requested operation, reaching or crossing a boundary in the user interface, entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detecting or recognizing a type of input or gesture, and the like. In some embodiments, a tactile output is provided to serve as a warning or prompt for an upcoming event or result that would otherwise occur unless a change in direction or interrupt input is detected in a timely manner. Tactile output is also used in other contexts to enrich the user experience, improve the accessibility of the device for users with visual or motor difficulties or other accessibility needs, and / or improve the efficiency and functionality of the user interface and / or device. Optionally, the tactile output is compared with audio input and / or visual user interface changes, which further enhances the user's experience when the user interacts with the user interface and / or device, and facilitates better transmission of information about the state of the user interface and / or device, and reduces input errors and improves the efficiency of the user's operation of the device.

[0064] It should be understood that device 100 is merely one example of a portable multifunction device and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing circuits and / or application specific integrated circuits.

[0065] Memory 102 optionally includes high-speed random access memory and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100 (such as CPU 120 and peripheral device interface 118) is optionally controlled by memory controller 122.

[0066] Peripherals interface 118 may be used to couple the device's input and output peripherals to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0067] In some embodiments, peripherals interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0068] RF (radio frequency) circuitry 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuitry 108 converts electrical signals into / from electromagnetic signals and communicates with a communication network and other communication devices via the electromagnetic signals. RF circuitry 108 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, and the like. RF circuitry 108 optionally communicates with networks and other devices via wireless communications, such as the Internet (also known as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). The wireless communication optionally uses any of a variety of communication standards, protocols and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution-Data Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilizing Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including communication protocols not yet developed on the date of filing of this document.

[0069] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., Figure 2 The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single or dual-ear headset) and input (e.g., a microphone).

[0070] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch-sensitive display system 112 and other input or control devices 116, to a peripherals interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from / to other input or control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, the one or more input controllers 160 are optionally coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. The one or more buttons (e.g., Figure 2 208) optionally includes an up / down button (e.g., or an up button and a separate down button) for volume control of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2 206 in the ).

[0071] The touch-sensitive display system 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch-sensitive display system 112 and / or sends electrical signals to the touch-sensitive display system. In some embodiments, the touch-sensitive display system 112 or the display controller 156, or the combination of the touch-sensitive display 112 and the display controller 156 is referred to as the display generation component of the device 100. The touch-sensitive display system 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some visual outputs or all of the visual outputs correspond to user interface objects. As used herein, the term "indicator" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object configured to respond to input directed to a graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0072] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from the user based on tactile and / or haptic contact. The touch-sensitive display system 112 and the display controller 156 (together with any associated modules and / or instruction sets in the memory 102) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In some embodiments, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0073] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112 to detect contact and any movement or interruption thereof. In some embodiments, projected mutual capacitance sensing technology is used, such as the ATmega 2000 from Apple Inc. (Cupertino, California). iPod and Technology found in.

[0074] The touch-sensitive display system 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touch screen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi, or greater). The user optionally uses any suitable object or appendage, such as a stylus, finger, or the like, to contact the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as precise as stylus-based input due to the larger contact area of ​​a finger on the touch screen. In some embodiments, the device converts rough finger-based input into precise pointer / cursor positions or commands for performing the action desired by the user.

[0075] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface that is separate from the touch-sensitive display system 112 or an extension of the touch-sensitive surface formed by the touch screen.

[0076] Device 100 also includes a power system 162 for powering the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power fault detection circuitry, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0077] Device 100 optionally also includes one or more optical sensors 164 . Figure 1A An optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. One or more optical sensors 164 optionally include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The one or more optical sensors 164 receive light projected from the environment through one or more lenses and convert the light into data representing an image. In conjunction with the imaging module 143 (also called a camera module), the one or more optical sensors 164 optionally capture still images and / or video. In some embodiments, the optical sensor is located on the rear portion of the device 100 opposite the touch-sensitive display system 112 on the front of the device, so that the touch screen can be used as a viewfinder for still image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to obtain an image of the user (e.g., for selfies, for video conferencing when the user views other video conference participants on the touch screen, etc.).

[0078] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1A A contact force sensor is shown coupled to a force sensor controller 159 in the I / O subsystem 106. One or more contact force sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). One or more contact force sensors 165 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the rear portion of the device 100 opposite the touch-sensitive display system 112 located on the front portion of the device 100.

[0079] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0080] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1A A tactile output generator is shown coupled to the tactile feedback controller 161 in the I / O subsystem 106. In some embodiments, the tactile output generator 167 includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generation components (e.g., components for converting electrical signals into tactile outputs on the device). The tactile output generator 167 receives tactile feedback generation instructions from the tactile feedback module 133 and generates tactile outputs on the device 100 that can be felt by the user of the device 100. In some embodiments, at least one tactile output generator is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112) and optionally generates tactile outputs by moving the touch-sensitive surface vertically (e.g., inward / outward toward the surface of the device 100) or laterally (e.g., back and forth in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on the back of the device 100 opposite the touch-sensitive display system 112 located on the front of the device 100.

[0081] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown coupled to the peripherals interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed on the touch screen display in a portrait view or a landscape view based on an analysis of data received from the one or more accelerometers. The device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver in addition to the accelerometer 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0082] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a tactile feedback module (or instruction set) 133, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and applications (or instruction sets) 136. In addition, in some embodiments, memory 102 stores a device / global internal state 157, as shown in Figures 1A and 3. The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state, which includes information obtained from various sensors and other input or control devices 116 of the device; and position and / or orientation information about the position and / or posture of the device.

[0083] The operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0084] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are suitable for coupling directly to other devices or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external ports are compatible with some of the Apple Inc. (Cupertino, California) iPod and In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used in some Apple Inc. (Cupertino, California) iPod and A Lightning connector that is the same as, or similar to, and / or compatible with, the Lightning connector used in the device.

[0085] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there has been movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has ceased (e.g., detecting a finger lift event or contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact or stylus contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, contact / motion module 130 and display controller 156 detect contact on a touchpad.

[0086] The contact / motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event. Similarly, taps, swipes, drags, and other gestures of the stylus are optionally detected by detecting a specific contact pattern of the stylus.

[0087] In some embodiments, detecting a finger tap gesture depends on detecting the length of time between a finger press event and a finger lift event, but is independent of detecting the intensity of the finger contact between the finger press event and the finger lift event. In some embodiments, a tap gesture is detected based on determining that the length of time between the finger press event and the finger lift event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of the finger contact during the tap reaches a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a light press or deep press intensity threshold. Thus, a finger tap gesture can meet a specific input criterion that does not require the characteristic intensity of the contact to meet a given intensity threshold to meet the specific input criterion. For clarity, the finger contact in a tap gesture is generally required to meet a nominal contact detection intensity threshold to detect a finger press event, below which the contact is not detected. A similar analysis applies to detecting a tap gesture by a stylus or other contact. In cases where the device is capable of detecting contact by a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0088] The same concepts apply in a similar manner to other types of gestures. For example, a swipe gesture, a pinch gesture, an expand gesture, and / or a long press gesture may be optionally detected based on satisfying criteria that are independent of the intensity of the contacts included in the gesture or that do not require one or more contacts performing the gesture to reach an intensity threshold in order to be recognized. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a zoom gesture is detected based on the movement of two or more contacts toward each other; a zoom gesture is detected based on the movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of a contact on the touch-sensitive surface having less than a threshold amount of movement. Thus, the statement that a particular gesture recognition criterion does not require the intensity of the contacts to meet the corresponding intensity threshold in order to satisfy the particular gesture recognition criterion means that the particular gesture recognition criterion can be satisfied when the contacts in the gesture do not meet the corresponding intensity threshold, and can also be satisfied when one or more contacts in the gesture meets or exceeds the corresponding intensity threshold. In some embodiments, a tap gesture is detected based on determining that a finger down event and a finger up event are detected within a predefined time period, regardless of whether the contact is above or below a corresponding intensity threshold during the predefined time period, and a swipe gesture is detected based on determining that the contact moves by more than a predefined amount, even if the contact is above the corresponding intensity threshold at the end of the contact movement. Even in specific implementations where detection of gestures is affected by the intensity of the contact performing the gesture (e.g., the device detects a long press more quickly when the intensity of the contact is above an intensity threshold, or the device delays detection of a tap input when the intensity of the contact is higher), detection of these gestures does not require the contact to reach a particular intensity threshold (e.g., even if the amount of time required to recognize the gesture varies), as long as the criteria for recognizing the gesture can be met without the contact reaching the particular intensity threshold.

[0089] In some cases, contact intensity thresholds, duration thresholds, and movement thresholds are combined in various combinations to create heuristic algorithms that distinguish between two or more different gestures directed to the same input element or area, enabling multiple different interactions with the same input element to provide a richer set of user interactions and responses. The statement that a particular set of gesture recognition criteria does not require the intensity of a contact to meet the corresponding intensity threshold to meet the particular gesture recognition criteria does not preclude the simultaneous evaluation of other intensity-related gesture recognition criteria to identify other gestures whose criteria are met when the gesture includes a contact with an intensity above the corresponding intensity threshold. For example, in some cases, a first gesture recognition criterion for a first gesture (which does not require the intensity of the contact to meet the corresponding intensity threshold to meet the first gesture recognition criterion) competes with a second gesture recognition criterion for a second gesture (which depends on the contact meeting the corresponding intensity threshold). In such a competition, if the second gesture recognition criterion for the second gesture is met first, the gesture is optionally not recognized as meeting the first gesture recognition criterion for the first gesture. For example, if the contact reaches the corresponding intensity threshold before the contact moves a predefined amount, a deep press gesture is detected instead of a swipe gesture. Conversely, if the contact moves a predefined amount before the contact reaches the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in such cases, the first gesture recognition criterion for the first gesture still does not require the intensity of the contact to meet the corresponding intensity threshold in order to satisfy the first gesture recognition criterion because, if the contact remains below the corresponding intensity threshold until the gesture ends (e.g., a swipe gesture with an intensity that does not increase above the corresponding intensity threshold), the gesture will be recognized as a swipe gesture by the first gesture recognition criterion. Thus, a particular gesture recognition criterion that does not require the intensity of the contact to meet the corresponding intensity threshold in order to satisfy the particular gesture recognition criterion may (A) in some cases ignore the intensity of the contact relative to the intensity threshold (e.g., for a tap gesture) and / or (B) in some cases fail to satisfy the particular gesture recognition criterion (e.g., for a long press gesture) if a set of competing intensity-related gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-related gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input, and in this sense still depends on the intensity of the contact relative to the intensity threshold (e.g., for a long press gesture competing with a deep press gesture for recognition).

[0090] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0091] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying the graphics to be displayed from an application program or the like, along with coordinate data and other graphic attribute data, if necessary, and then generates screen image data for output to the display controller 156.

[0092] Haptic feedback module 133 includes various software components for generating instructions (e.g., instructions used by haptic feedback controller 161) to generate tactile output at one or more locations on device 100 using tactile output generator 167 in response to user interaction with device 100.

[0093] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0094] The GPS module 135 determines the location of the computer system and provides this information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services such as the weather widget, the local yellow pages widget, and the map / navigation widget).

[0095] Application 136 optionally includes the following modules (or instruction sets), or a subset or superset thereof:

[0096] Contacts module 137 (sometimes called address book or contact list);

[0097] Telephone module 138;

[0098] Video conferencing module 139;

[0099] Email client module 140;

[0100] Instant messaging (IM) module 141;

[0101] Fitness support module 142;

[0102] A camera module 143 for still and / or video images;

[0103] Image management module 144;

[0104] Browser module 147;

[0105] Calendar module 148;

[0106] Widget module 149, which optionally includes one or more of the following: weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets acquired by the user, and user-created widgets 149-6;

[0107] A widget creator module 150 for forming user-created widgets 149-6;

[0108] Search module 151;

[0109] A video and music player module 152, optionally consisting of a video player module and a music player module;

[0110] Memo module 153;

[0111] Map module 154; and / or

[0112] Online video module 155.

[0113] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

[0114] In combination with the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the contacts module 137 includes executable instructions for managing an address book or contact list (e.g., stored in the application internal state 192 of the contacts module 137 in memory 102 or memory 370), including: adding names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers and / or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing 139, email 140, or instant messaging 141; and so on.

[0115] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 includes executable instructions for entering a character sequence corresponding to a phone number, accessing one or more phone numbers in address book 137, modifying an entered phone number, dialing the corresponding phone number, conducting a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0116] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and phone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0117] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0118] In conjunction with the RF circuitry 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for entering a character sequence corresponding to an instant message, modifying previously entered characters, transmitting the corresponding instant message (e.g., using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for phone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Service (APNs), or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0119] In combination with the RF circuitry 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the video and music player module 152, the fitness support module 142 includes executable instructions for creating a workout (e.g., with time, distance, and / or calorie burn goals); communicating with fitness sensors (in sports equipment and smart watches); receiving fitness sensor data; calibrating sensors for monitoring a workout; selecting and playing music for a workout; and displaying, storing, and transmitting fitness data.

[0120] In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0121] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0122] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching for, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) in accordance with user instructions.

[0123] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0124] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5), or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop widget includes XML (Extensible Markup Language) files and JavaScript files (e.g., the Yahoo! desktop widget).

[0125] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget creator module 150 includes executable instructions for creating a desktop widget (e.g., transferring a user-specified portion of a web page into a desktop widget).

[0126] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0127] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

[0128] In conjunction with the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the memo module 153 includes executable instructions for creating and managing memos, to-do lists, etc. according to user instructions.

[0129] In combination with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and data associated with the maps (e.g., driving directions; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0130] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on touch screen 112 or on an external display connected wirelessly or via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a particular online video.

[0131] Each module and application identified above corresponds to a set of executable instructions for performing one or more functions and methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). These modules (e.g., instruction sets) do not have to be implemented as independent software programs, processes, or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

[0132] In some embodiments, device 100 is a device in which operation of a predefined set of functions on the device is performed exclusively through a touch screen and / or a touchpad. By using a touch screen and / or a touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on device 100 is optionally reduced.

[0133] A predefined set of functions that are exclusively performed through the touch screen and / or trackpad optionally includes navigation between user interfaces. In some embodiments, the trackpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, home menu, or root menu. In such embodiments, the trackpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a trackpad.

[0134] Figure 1B is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 (Figure 3) includes an event classifier 170 (for example, in the operating system 126) and a corresponding application 136-1 (for example, any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0135] The event classifier 170 receives event information and determines the application 136-1 and the application view 191 of the application 136-1 to which the event information is to be delivered. The event classifier 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, the application 136-1 includes an application internal state 192 that indicates one or more current application views displayed on the touch-sensitive display system 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application(s) is currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0136] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or is ready to be displayed by the application 136-1, a state queue for enabling the user to return to a previous state or view of the application 136-1, and a redo / undo queue of previous actions taken by the user.

[0137] Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (through audio circuit 110). The information received by peripherals interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

[0138] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0139] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0140] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0141] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to programmatic levels within the application's programmatic or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as correct input is optionally determined based at least in part on the hit view of the initial touch that started the touch-based gesture.

[0142] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (e.g., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0143] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, views higher in the hierarchy will still remain actively participating views.

[0144] Event dispatcher module 174 dispatches event information to event recognizers (e.g., event recognizer 180). In embodiments that include active event recognizer determination module 173, event dispatcher module 174 delivers the event information to the event recognizer determined by active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.

[0145] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module stored in memory 102, such as contact / motion module 130.

[0146] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events that occur within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more of event recognizers 180 is part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update the application's internal state 192. Alternatively, one or more of the application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0147] A corresponding event identifier 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event identifier 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0148] The event receiver 182 receives event information from the event classifier 170. The event information includes information about sub-events such as touches or touch movements. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0149] Event comparator 184 compares event information with predefined event or sub-event definitions and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, the sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift (touch end) of a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display system 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0150] In some embodiments, event definition 187 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler that is associated with the sub-event and the object that triggered the hit test.

[0151] In some embodiments, the definition of the corresponding event 187 also includes delay actions that delay the delivery of the event information until it has been determined that the sub-event sequence does or does not correspond to the event type of the event identifier.

[0152] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definitions 186, the corresponding event recognizer 180 enters the event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0153] In some embodiments, corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery for actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0154] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating an event handler 190 is different from sending (and deferred sending) sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag obtains the flag and performs a predefined process.

[0155] In some embodiments, event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or with the actively participating view receives the event information and performs a predetermined process.

[0156] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video or music player module 152. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0157] In some embodiments, event handler 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0158] It should be understood that the above discussion of event handling for user touches on a touch-sensitive display also applies to other forms of user input utilizing input devices to operate the multifunction device 100, and not all user input is initiated on a touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a trackpad, such as taps, drags, scrolls, etc.; stylus input; movement of the device; spoken commands; detected eye movement; biometric input; and / or any combination thereof, are optionally used as input corresponding to sub-events defining the event to be recognized.

[0159] Figure 2 A device with a touch screen (e.g., Figure 1AIn some embodiments, the portable multifunction device 100 includes a touch-sensitive display system 112 (e.g., a touch screen). The touch screen optionally displays one or more graphics within a user interface (UI) 200. In these embodiments, and in other embodiments described below, a user can select one or more of the graphics by, for example, making a gesture on the graphics using one or more fingers 202 (not drawn to scale in the figures) or one or more styluses 203 (not drawn to scale in the figures). In some embodiments, selection of the one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or rolling of a finger that has made contact with the device 100 (from right to left, from left to right, up and / or down). In some implementations or in some cases, inadvertent contact with a graphic does not select the graphic. For example, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0160] The device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen display.

[0161] In some embodiments, the device 100 includes a touch screen display, a menu button 204 (sometimes referred to as a home button 204), a push button 206 for powering the device on / off and for locking the device, a volume adjustment button 208, a subscriber identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and holding the button in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing the button before the predefined time interval has passed; and / or to unlock the device or initiate an unlocking process. In some embodiments, the device 100 also accepts voice input for activating or deactivating certain functions through the microphone 113. The device 100 also optionally includes one or more contact force sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more tactile output generators 167 for generating tactile output for the user of the device 100.

[0162] Figure 3A3 is a block diagram of an exemplary multifunctional device with a display and a touch-sensitive surface according to some embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a game system or a control device (for example, a home controller or an industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370 and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuits (sometimes referred to as chipsets) that interconnect system components and control the communication between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is typically a touch screen display. In some embodiments, display 340 is referred to as a display generation component. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on device 300 (e.g., similar to the above referenced embodiments). Figure 1A The one or more tactile output generators 167 described above), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch sensors, and / or sensors similar to those described above) Figure 1A In some embodiments, the device 300 includes a wireless interface 311 for communicating with one or more wearable audio output devices 301.

[0163] Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores data related to portable multifunction device 100 ( Figure 1A ), or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100( Figure 1A )'s memory 102 optionally does not store these modules.

[0164] Figure 3A Each element in the above-mentioned identified element is optionally stored in one or more memory devices in the previously mentioned memory device.Each module in the above-mentioned identified module corresponds to the instruction set for performing the above-mentioned functions.The above-mentioned identified module or program (that is, instruction set) need not be implemented as independent software program, process or module, so the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments.In some embodiments, memory 370 optionally stores the subset of above-mentioned modules and data structure.In addition, memory 370 optionally stores additional modules and data structure not described above.

[0165] Figure 3Bis a block diagram of an exemplary wearable audio output device 301 according to some embodiments. In some embodiments, the wearable audio output device 301 is one or more in-ear headphones, earbuds, over-ear headphones, etc. In some examples, the wearable audio output device 301 is a single earphone or earbud. In some examples, the wearable audio output device 301 includes a pair of earphones or earbuds (e.g., one earphone or earbud for each of the user's ears). In some examples, the wearable audio output device 301 includes over-ear headphones (e.g., a headset with two over-ear earcups to be placed over the user's ears and optionally connected by a headband). In some embodiments, the wearable audio output device 301 includes one or more speakers 306 for providing audio output (e.g., to the user's ears). In some embodiments, the wearable audio output device 301 includes one or more placement sensors 304 to detect the positioning or placement of the wearable audio output device 301 relative to the user's ears, such as detecting the placement of the wearable audio output device 301 in the user's ears. In some embodiments, the wearable audio output device 301 conditionally outputs audio based on whether the wearable audio output device 301 is in or near the user's ear (e.g., foregoes outputting audio when the wearable audio output device 301 is not in the user's ear in order to reduce power usage). In some embodiments where the wearable audio output device 301 includes multiple (e.g., a pair) wearable audio output components (e.g., headphones, earbuds, or earmuffs), each component includes one or more corresponding placement sensors, and the wearable audio output device 301 conditionally outputs audio based on whether one or both components are in or near the user's ear, as described herein. In some embodiments, the wearable audio output device 301 includes an audio I / O logic component 312 that determines the positioning or placement of the wearable audio output device 301 relative to the user's ear based on information received from the placement sensor 304, and in some embodiments, the audio I / O logic component 312 controls the resulting conditional audio output. In some embodiments, the wearable audio output device 301 includes a controller for communicating with a multifunction device such as device 100 ( Figure 1A ) or device 300( Figure 3A ) wireless interface 315 for communicating with a multifunction device such as device 100 ( Figure 1A ) or device 300( Figure 3A ) connection (e.g., via a headphone jack or other audio port). In some embodiments, a user can interact with and provide input to the wearable audio output device 301 via interface 315 (e.g., remotely).

[0166] In some embodiments, the wearable audio output device 301 includes one or more microphones 302 for receiving audio input. In some embodiments, the microphone 302 detects the voice of the user wearing the wearable audio output device 301 and / or the ambient noise around the wearable audio output device 301. In some embodiments, as described herein with reference to Figure 3C In more detail, the plurality of microphones of the microphone 302 are positioned at different locations on the wearable audio output device 301 to measure speech and / or ambient noise at different locations around the wearable audio output device 301. In some embodiments where the wearable audio output device 301 includes multiple (e.g., a pair) wearable audio output components (e.g., headphones or earbuds), each component includes one or more corresponding microphones. In some embodiments, the audio I / O logic component 312 detects or recognizes speech or ambient noise based on the information received from the microphone 302.

[0167] In some embodiments, the wearable audio output device 301 includes one or more input devices 308. In some embodiments where the wearable audio output device 301 includes multiple (e.g., a pair) wearable audio output components (e.g., headphones, earbuds, or earmuffs), each component includes one or more corresponding input devices. In some embodiments, the input device 308 includes a pressure-sensitive (e.g., intensity-sensitive) input device that, in some embodiments, is located within a portion (sometimes referred to as a "handle") of the wearable audio output device 301 that physically extends from a portion of the wearable audio output device 301 that is configured to be inserted into the user's ear (e.g., as shown in FIG. 1 ). Figure 3C The handle 305 shown). In some embodiments, the pressure-sensitive input device detects input from the user in response to the user squeezing the input device (e.g., by pinching the handle of the wearable audio output device 301 between two fingers). In some embodiments, the input device 308 includes a touch-sensitive surface (for detecting touch input), an accelerometer and / or a posture sensor (for determining the posture of the wearable audio output device 301 relative to the physical environment and / or changes in the device's posture), and / or other input devices through which a user can interact with and provide input to the wearable audio output device 301. In some embodiments, the input device 308 includes one or more volume control hardware elements for (e.g., local) volume control of the wearable audio output device 301 (e.g., up / down buttons for volume control, or as described herein with reference to Figure 1A In some embodiments, the input provided via one or more input devices 308 is processed by the audio I / O logic component 312. In some embodiments, the audio I / O logic component 312 is connected to a separate device (e.g., Figure 1A device 100 or Figure 3A 300) that provides instructions or content for audio output and optionally receives and processes input (or information about input) provided via microphone 302, placement sensor 304, and / or input device 308, or via one or more input devices of a separate device. In some embodiments, audio I / O logic 312 is located on device 100 (e.g., as a Figure 1A ) or device 300 (e.g., as part of the peripheral device interface 118 of Figure 3A 301), or alternatively is located partially in device 100 and partially in device 301, or is located partially in device 300 and partially in device 301.

[0168] Figure 3C 1 shows exemplary audio control by a wearable audio output device according to some embodiments. In some embodiments, when a wearable audio output device having earbuds to which replaceable eartips can be attached is worn in a user's ears, the earbuds and eartips together act as a physical barrier, blocking at least some ambient sounds from the surrounding physical environment from reaching the user's ears. For example, in Figure 3C In FIG, a user wears the wearable audio output device 301 such that the earplug 303 and the eartip 314 are in the user's left ear. The eartip 314 extends at least partially into the user's ear canal. Preferably, when the earplug 303 and the eartip 314 are inserted into the user's ear, a seal is formed between the eartip 314 and the user's ear to isolate the user's ear canal from the surrounding physical environment. However, in some embodiments, the earplug 303 and the eartip 314 together block some, but not necessarily all, ambient sounds in the surrounding physical environment from reaching the user's ear. Thus, in some embodiments (e.g., Figure 3B A first microphone (or, in some embodiments, a first set of one or more microphones) 302-1 of the microphone 302 is located on the wearable audio output device 301 to detect ambient sound represented by the waveform 322 in an area 316 of the physical environment surrounding the earbud 303 (e.g., outside the earbud). In some embodiments, (e.g., Figure 3BA second microphone (or, in some embodiments, a second set of one or more microphones) 302-2 of the microphone 302 is located on the wearable audio output device 301 to detect any ambient sound that is not completely blocked by the earbud 303 and the eartip 314 and can be heard in the area 318 within the user's ear canal, as represented by the waveform 324. Therefore, in some cases where the wearable audio output device 301 does not generate a noise cancellation (also known as "inverted phase") audio signal to cancel (e.g., attenuate) ambient sound from the surrounding physical environment (as indicated by waveform 326-1), the ambient sound waveform 324 can be perceived by the user (as indicated by waveform 328-1). In some cases where the wearable audio output device 301 generates an inverted phase audio signal to cancel ambient sound (as shown by waveform 326-2), the ambient sound waveform 324 cannot be perceived by the user (as shown by waveform 328-2).

[0169] In some embodiments, the ambient sound waveform 322 is compared to the attenuated ambient sound waveform 324 (e.g., by the wearable audio output device 301 or a component of the wearable audio output device 301, such as the audio I / O logic 312, or by an electronic device in communication with the wearable audio output device 301) to determine the passive attenuation provided by the wearable audio output device 301. In some embodiments, the amount of passive attenuation provided by the wearable audio output device 301 is taken into account when providing an inverted audio signal to cancel ambient sound from the surrounding physical environment. For example, the inverted audio signal waveform 326-2 is configured to cancel the attenuated ambient sound waveform 324, but not the unattenuated ambient sound waveform 322.

[0170] In some embodiments, the wearable audio output device 301 is configured to operate in one of a plurality of available audio output modes, such as an active noise control audio output mode, an active pass-through audio output mode, and a bypass audio output mode (sometimes also referred to as a noise control off audio output mode). In the active noise control mode (also referred to as "ANC"), the wearable audio output device 301 outputs one or more audio cancellation audio components (e.g., one or more inverted phase audio signals, also referred to as "audio cancellation audio components") to at least partially cancel ambient sound from the surrounding physical environment that would otherwise be perceived by the user. In the active pass-through audio output mode, the wearable audio output device 301 outputs one or more pass-through audio components (e.g., plays at least a portion of the ambient sound from outside the user's ear received by, for example, microphone 302-1), so that the user can hear a greater amount of ambient sound from the surrounding physical environment than would otherwise be perceived by the user (e.g., a greater amount of ambient sound than would be heard using passive attenuation of the wearable audio output device 301 placed in the user's ear). In bypass mode, active noise management is turned off such that the wearable audio output device 301 outputs neither any audio cancellation audio component nor any pass-through audio component (e.g., such that any amount of ambient sound perceived by the user is due to the physical attenuation of the wearable audio output device 301). In some embodiments, in response to a particular type of input, such as a tap and hold gesture, at the handle 305 of the wearable audio output device 301, the wearable audio output device 301 cycles through one or more of the above-described audio output modes, as described in further detail herein, for example with reference to Figures 8A to 8J and Figures 9A to 9C .

[0171] Figure 3D Similar to Figure 3C , the difference is Figure 3D The wearable audio output device 301b is shown as a set of headphones having ear cups that are worn over the user's ears rather than one or more earbuds that are worn in the user's ears. Figure 3D In the example of the wearable audio output device 301b, the earmuffs, such as the earmuff 332 worn over the user's left ear (and the second earmuff typically worn over the user's right ear), act as a physical barrier, blocking at least some ambient sounds from the surrounding physical environment from reaching the user's ears. Microphones 302-1 and 302-2 located on the earmuff 332 detect ambient sounds in an area 316 of the physical environment (represented by waveform 322) and ambient sounds that are not completely blocked by the earmuff 332 and can be heard in an area 318 within the earmuff 332 (represented by waveform 324), as described herein with reference to Figure 3C Describe in more detail. Figure 3DAlso shown are the generated inverted waveforms 326-1 and 326-2 and the resulting perceived ambient sound waveforms 328-1 and 328-2, respectively, also as described herein with reference to Figure 3C Describe in more detail.

[0172] In addition, Figure 3D In the example of FIG. 3 , the wearable audio output device 301 b includes a knob 334 and a button 336 (e.g., as Figure 3B part of the input device 308). In some embodiments, rotation of the knob 334 controls one or more aspects of the audio output (e.g., cycles through a set of possible values ​​for aspects of the audio output), such as the audio output volume level or the degree of noise cancellation. For example, rotation of the knob 334 controls the audio output volume (e.g., rotation in one direction (such as clockwise) increases the volume, and rotation in another direction (such as counterclockwise) decreases the volume). In some embodiments, the knob 334 is configured to receive input other than rotational input. For example, the knob 334 is also a push-button switch or includes a touch-sensitive surface and / or an input intensity-sensitive surface and is used by the user to provide touch, press, or click input to the wearable audio output device 301b, as described herein with reference to 25A to 25D Describe in more detail.

[0173] In some embodiments, button 336 is a push-button switch or includes a touch-sensitive surface and / or an input intensity-sensitive surface and controls a corresponding aspect of the audio output (optionally, an aspect different from one or more aspects controlled using knob 334). In some embodiments, pressing (or tapping, actuating, etc.) button 336 causes wearable audio output device 301b to transition between different audio output modes, as described herein with reference to Figure 9C Described in more detail (e.g., a press input on a button 336 on a set of over-ear headphones performs one or more of the operations described herein in response to detecting a click-and-hold gesture on the stem of an earbud).

[0174] Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .

[0175] Figure 4A An exemplary user interface for an application menu on portable multifunction device 100 is shown according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0176] One or more signal strength indicators of one or more wireless communications, such as cellular signals and Wi-Fi signals;

[0177] ·time;

[0178] Bluetooth indicator;

[0179] Battery status indicator;

[0180] A tray 408 with icons for commonly used applications, such as:

[0181] o An icon 416 labeled "Phone" for the phone module 138, the icon 416 optionally including an indicator 414 of the number of missed calls or voicemails;

[0182] o An icon 418 of the email client module 140 labeled “Mail,” which optionally includes an indicator 410 of the number of unread emails;

[0183] o An icon 420 labeled "Browser" for the browser module 147; and

[0184] o An icon 422 labeled “Music” of the video and music player module 152;

[0185] as well as

[0186] Icons for other apps, such as:

[0187] o Icon 424 labeled "Messages" of the IM module 141;

[0188] o An icon 426 labeled “Calendar” of the calendar module 148;

[0189] o Icon 428 labeled "Photos" of the image management module 144;

[0190] o An icon 430 labeled “Camera” of the camera module 143;

[0191] ○ An icon 432 labeled “Online Video” of the online video module 155;

[0192] ○ Icon 434 labeled “Stock Market” of the Stock Market widget 149-2;

[0193] o An icon 436 labeled “Map” of the map module 154;

[0194] ○ Icon 438 labeled “Weather” of the weather widget 149-1;

[0195] ○ Icon 440 labeled “Clock” of the alarm clock widget 149-4;

[0196] o An icon 442 labeled “Fitness Support” of the fitness support module 142;

[0197] o An icon 444 labeled "Notepad" of the Notepad module 153; and

[0198] o An icon 446 for a settings application or module that provides access to settings for the device 100 and its various applications 136 .

[0199] It should be noted that Figure 4A The icon labels shown in the are merely exemplary. For example, other labels are optionally used for various application icons. In some embodiments, the label of a respective application icon includes the name of the application corresponding to the respective application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0200] Figure 4B An exemplary user interface is shown on a device (e.g., device 300 in FIG. 3 ) having a touch-sensitive surface 451 (e.g., tablet or trackpad 355 in FIG. 3 ) that is separate from a display 450. Although in some examples, input can be received on a touch screen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the main axis (for example, Figure 4B According to these embodiments, the device detects contact with the touch-sensitive surface 451 at a location that corresponds to a corresponding location on the display (e.g., Figure 4B 460 and 462 in (e.g., in Figure 4B , 460 corresponds to 468 and 462 corresponds to 470). Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g., Figure 4B When the user interface 450 in FIG. 4 is separated, the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0201] User interface and associated processes

[0202] Attention is now turned to the possibility of using a display device (e.g. Figure 1A touch-sensitive display system 112 or Figure 3A display 340) and a touch-sensitive surface (e.g., Figure 1Atouch-sensitive display system 112 or Figure 3A 355) and includes one or more wearable audio output devices (e.g., one or more wearable audio output devices 301) or a computer system in communication therewith (e.g., including an electronic device such as Figure 1A The portable multifunction device 100, or Figure 3A Implementation of a user interface ("UI") and associated processes implemented in device 300).

[0203] Figure 5A to Figure 5V An exemplary user interface for pairing and calibrating a wearable audio output device according to some embodiments is shown. 6A to 6O An exemplary user interface is shown for alerting a user when an earbud no longer meets fit criteria during use, according to some embodiments. Figures 7A to 7Q An exemplary user interface and user interaction for changing the audio output mode of a wearable audio output device according to some embodiments are shown. Figures 8A to 8H Example user interactions with a wearable audio output device for controlling audio output are shown according to some embodiments. Figures 9A to 9C An exemplary settings user interface for controlling various features associated with a wearable audio output device and examples of controlling audio output modes are shown according to some embodiments. 13A to 13D An exemplary user interface for pairing a wearable audio output device according to some embodiments is shown. Figures 14A to 14Z An exemplary user interface and user interaction for changing the audio output mode of a wearable audio output device according to some embodiments are shown. Figures 15A to 15E An exemplary settings user interface for controlling various features associated with a wearable audio output device and examples of controlling audio output modes are shown according to some embodiments. Figures 16A to 16G An exemplary user interface for calibrating the acoustic seal quality of a wearable audio output device and displaying information about the same is shown according to some embodiments. Figures 17A to 17J Exemplary user interfaces and user interactions are shown for selecting an audio output device, such as a wearable audio output device, and changing the audio output mode of the selected audio output device via (e.g., using) the wearable electronic device, according to some embodiments. Figures 21A to 21J Exemplary user interfaces and user interactions for changing the audio output mode of a wearable audio output device are shown. 22A to 22D Exemplary user interfaces and user interactions are shown for accessing a settings user interface for controlling various features associated with a wearable audio output device. Figures 23A to 23AAExemplary user interfaces and user interactions are shown for controlling audio output settings of multiple wearable audio output devices using a single settings user interface. Figures 25A to 25G Example user interactions for controlling audio output from a wearable audio output device and example audio alerts and visual alerts in response to such user interactions are shown. The user interfaces, inputs, and audio outputs in these figures are used to illustrate the processes described below, including Figures 10A to 10H 、 11A to 11D 、 12A to 12E 、 18A to 18H 、 19A to 19C 、 FIG. 20A to FIG. 20B 、 24A to 24D and 26A to 26D For ease of explanation, some embodiments of the embodiments will be discussed with reference to operations performed using a wearable audio output device that is worn by a user and that communicates with and is separate from a touch-sensitive input device, such as a touchpad 355, and a computer system having a touch-sensitive display system 112 or display 340. In some embodiments, operations are performed in response to instructions received by the wearable audio output device from the electronic device based on processing performed at the electronic device. In some embodiments, operations are performed by the wearable audio output device based on processing performed at the wearable audio output device. However, in some cases, similar operations are optionally performed using an audio output device that is part of a device having a display generating component and / or a touch-sensitive input device (e.g., a wearable device that integrates one or more audio output devices with a display and / or a touch-sensitive input device, such as a headset or headphones).

[0204] Figure 5A to Figure 5V An exemplary user interface for pairing and calibrating a wearable audio output device according to some embodiments is shown.

[0205] Figure 5A 1 shows an exemplary user interface on the display 112 of the device 100. While displaying the user interface, the device 100 periodically and repeatedly listens for wireless broadcast signals (e.g., pairing requests) from one or more peripheral devices (e.g., earbuds 502-1 and 502-2, and earbud case 502-3) for pairing the one or more peripheral devices with the device 100. In one embodiment, as 5A to 5D As shown in , device 100 is able to detect a pairing request from a peripheral device when the peripheral device is within a threshold distance 508 of device 100, and is unable to detect a pairing request from the peripheral device when the peripheral device is outside the threshold distance 508. For example, Figure 5A At the bottom of Figure 5A1 and 502-2, and an earbud case 502-3, where the earbuds 502-1 and 502-2 and the earbud case 502-3 are outside a threshold distance 508 of the device 100. In contrast, Figure 5B It shows when earbud 502-1, earbud 502-2, and earbud box 502-3 (shown and hereinafter collectively referred to as earbud set 503) are within a threshold distance 508 of device 100, where the device is able to detect pairing requests from these peripheral devices.

[0206] Figure 5B Shown in Figure 5A An exemplary user interface for initiating pairing of the device 100 with the peripheral device (e.g., earbud 502-1) or the group of peripheral devices (e.g., earbud group 503) is displayed after the user interface in and in some embodiments in response to detecting movement of the peripheral device (e.g., earbud 502-1) or the group of peripheral devices (e.g., earbud group 503) from outside the threshold distance 508 to within the threshold distance 508. Figure 5B , the device 100 detects a signal from a peripheral device (e.g., Figure 5A The earplug 502-1 and Figure 5B In some embodiments, in response to detecting the pairing request from earbud 502-1, device 100 determines whether earbud 502-1 meets the coupling criteria. Figure 5B In the example shown, the coupling criteria are met when the earbud (e.g., earbud 502-1) is placed in an earbud case (e.g., earbud case 502-3) and / or is electrically coupled to an earbud case (e.g., forming earbud set 503). After the device 100 determines that the earbud 502-1 meets the coupling criteria (e.g., in response to the device 100 determining that the earbud 502-1 meets the coupling criteria), the device 100 displays the window 520-1 overlying the Figure 5A Window 520-1 includes a representation of the earbud set 503 and a button 522-1 (labeled "Connect") that, when activated by user input, initiates a connection between the device 100 and a peripheral device (e.g., Figure 5A1 and 502-2 and the earbud case 502-3 shown in FIG. 4 . In some embodiments, the user interface shown in window 520-1 is instead displayed on the entirety (or substantially all, e.g., greater than 95%, 96%, 97%, 98%, or 99%) of touch screen 112. Device 100 also displays an exit button 524-1 that, when activated by user input such as a tap gesture, causes window 520-1 to cease display so that the user can perform other operations on device 100. In the event that exit button 524-1 is accompanied by another user interface described herein (e.g., a user interface that is part of a pairing process or fit test), exit button 524-1 causes that user interface to cease display when activated by user input.

[0207] Figures 5C to 5D Shown from Figure 5B Specifically, Figure 5C An input 540 (e.g., a tap gesture) on button 522-1 in window 520-1 is shown. In response to detecting input 540, device 100 initiates pairing of device 100 with the peripheral device of earbud set 503. Figure 5D , after device 100 has paired with the peripheral device of earbud set 503, device 100 displays an indication 523-1 (labeled "Connected") in window 520-2 to indicate that device 100 is paired with (e.g., connected to) earbud set 503. In some embodiments, device 100 displays a button in window 520-2 that, when activated, causes device 100 to proceed from the pairing process to a fit test for determining and optimizing the fit of earbuds 502-1 and 502-2 in the user's ears.

[0208] Figure 5E An exemplary user interface 521-1 is shown for initiating a fit test to determine and optimize the fit of earbud 502-1 and earbud 502-2 in the user's ear. In some embodiments, user interface 521 is displayed on the entirety (or substantially all, e.g., greater than 95%, 96%, 97%, 98%, or 99%) of touch screen 112, rather than on the entirety (or substantially all, e.g., greater than 95%, 96%, 97%, 98%, or 99%) of touch screen 112. Figures 5E to 5V In some embodiments, the user interface 521-1 responds to a button displayed in the immediately preceding user interface (e.g., Figure 5D The button displayed in the window 520-2 is used to advance from the pairing process to the fit test) and is displayed for user input. Figure 5E Shown is a button 525 - 1 (labeled “Start Test”) which, when activated, initiates a fit test. Figure 5EAlso shown is an input 526-1 (e.g., a tap gesture) on button 525-1 to activate button 525-1 and begin a fit test. As described in further detail below, during the fit test, earbud 502-1 and earbud 502-2 obtain audio data (e.g., based on audio detected using a microphone configured to detect in-ear audio and audio detected using a microphone configured to detect ambient audio), and, based at least in part on the obtained audio data, device 100 displays suggestions to the user to improve the fit of earbud 502-1 and / or earbud 502-2 in the user's ear.

[0209] Figure 5F Shown from Figure 5E Specifically, Figure 5F Shown is a response to a user input (e.g., Figure 5E Example user interface 521-2 displayed in response to user input on button 525-1). Figure 5F Also shown are earbuds 502-1 and 502-2 associated with the user's ears 528-1 and 528-2. Figure 5F The earbuds 502-1 and 502-2 are shown not being placed in the user's ears 528-1 and 528-2. Figure 5F As shown, earbuds 502-1 and 502-2 are coupled to EarTips 527-1 and 527-2 (e.g., attached to the earbuds, which may be made of rubber or a rubber-like material such as silicone to help form a seal between the respective earbuds and the respective ears in which they are placed). EarTips 527-1 and 527-2 can be removed by the user and replaced with different EarTips. Figure 5F The user interface 521-2 in includes instructions to the user to place the earbud 502 in his or her ear in the form of text “Please place earbud in ear” and two ear representations, each associated with a corresponding representation of an earbud. The user interface 521-2 also includes indicators 529-1 and 529-2 displayed below the ear representations, with gray circles indicating that an earbud is not placed in the user's ear (or that the device 100 cannot detect that an earbud has been placed in the user's ear). The letter “R” in indicator 529-1 indicates that indicator 529-1 represents an earbud for the right ear 528-1 (e.g., the status of the earbud), which in this example is earbud 502-1. The letter “L” in indicator 529-2 indicates that indicator 529-2 represents an earbud for the left ear 528-2 (e.g., the status of the earbud), which in this example is earbud 502-2. Although some embodiments (e.g., as Figure 5F) uses gray circles to indicate that earbuds 502-1 and 502-2 have not been detected in the user's ears, but one of ordinary skill in the art will recognize that other forms of status indicators may be used instead.

[0210] Figure 5G Shown from Figure 5F Specifically, Figure 5G 502-1 is shown placed in the user's ear 528-1 and the earbud 502-2 is shown placed in the user's ear 528-2. Thus, the indicators 529-1 and 529-2 have changed from gray circles indicating that the earbuds 502-1 and 202-2 are not in the user's ears to black circles indicating that the earbuds 502-1 and 202-2 have been detected as being placed in the user's ears and the next step of the fit test has not yet been performed. Although some embodiments (e.g., Figure 5G ) uses black circles to indicate that earbuds 502-1 and 502-2 have been placed in the user's ears (e.g., before performing the next step of the fit test, such as a calibration step), but one of ordinary skill in the art will recognize that other forms of status indicators may be used instead.

[0211] Figure 5H Shown from Figure 5G Specifically, Figure 5H An exemplary user interface 521-4 is shown that includes one or more indications that a fit test is in progress, such as animation 530 and the text "FIT TEST IN PROGRESS." Figure 5H In some embodiments, in response to device 100 detecting that both earbuds 502-1 and 502-2 have been placed in the user's ears, user interface 521-4 is displayed. In some embodiments, in response to activating the user interface 521-2 ( Figure 5F ) and 521-3( Figure 5G ) to display user interface 521-4 for continuing the fit test after the earbuds 502-1 and 502-2 have been placed in the user's ears (and optionally, in some embodiments, the button is displayed for the first time, or becomes activatable after being displayed as inactivatable (e.g., grayed out), in response to detecting that the earbuds 502-1 and 502-2 are placed in the user's ears). In some embodiments, performing the fit test includes outputting a calibration tone via the earbuds 502 (e.g., using Figure 3B 306), detecting audio via earbuds 502-1 and 502-2 (e.g., using one or more microphones on each earbud, such as Figure 3Bmicrophone 302-2), and determining whether the detected audio indicates that the earbud 502 meets the device fit criteria (e.g., by comparing the detected audio to a calibration tone to determine whether and how much ambient audio is included in the detected audio; if the detected audio includes greater than a threshold amount of ambient audio, the device fit criteria is not met).

[0212] In addition, Figure 5H , indicator 529-1 has changed to a green circle with a check mark, which indicates that earbud 502-1 has met the fit criteria (also referred to as the "device fit criteria") assessed by the fit test. In contrast, the appearance of indicator 529-2 has not changed. Although Figure 5J While the example in FIG. 5 shows that indicator 529-1 has been updated during the fit test to reflect whether earbud 502-1 meets the fit criteria separately from (e.g., prior to) updating indicator 529-2 to reflect whether earbud 502-2 meets the fit criteria, one of ordinary skill will recognize that in some embodiments, these indicators are updated simultaneously, and optionally after the fit test is concluded (e.g., such that indicator 529-1 is updated after Figure 5H Continue to have Figure 5G same appearance as in indicator 529-2).

[0213] Figure 5I Shown from Figure 5H Specifically, Figure 5I An exemplary user interface 521-5 is shown that is displayed after the fit test has been performed and presents the results of the fit test. Indicators 529-1 and 529-2 in the user interface 521-5 show the status of the earbuds 502-1 and 502-2, respectively. Specifically, Figure 5I Indicator 529-2 in is displayed in a different color (e.g., yellow) represented by a changing fill pattern, indicating that the fit test has been performed and the earbud 502-2 does not meet the fit criteria (e.g., because the earbud 502-2 is not properly placed in the user's ear). In addition, the user interface 521-5 includes an instruction 531-1 prompting the user to adjust the placement of the left earbud 502-2 in the user's left ear. Below the instruction 531-1, a button 532-1 (labeled "Continue") is displayed to continue the fit test. In some embodiments, button 532-1 is inactivatable (e.g., grayed out) until the device 100 determines that the user has complied with the instruction 531-1 (e.g., by detecting or receiving information indicating that the earbud 502-2 has been removed from the ear (e.g., ear 528-2) and replaced), after which button 532-1 becomes activatable.

[0214] Figure 5J Shown with Figure 5I Same user interface 521 - 5 . Figure 5J Also shown is earbud 502-2 now properly aligned in ear 528-2 (e.g., in response to instruction 531-1), and input 533 received at user interface 521-5, specifically at button 532-1, to continue with the fit test (e.g., repeating the fit test because left ear 502-2 failed the fit test). Figure 5J The left earbud 502-2 is shown with Figure 5I , but in some embodiments, the indicator 529-2 changes to a gray circle in response to the earbud 502-2 being removed (e.g., when the earbud 502-2 is not detected as being placed in the user's ear) and changes to a black circle in response to the earbud 502-2 being replaced in the user's ear (e.g., to indicate that the earbud 502-2 has been detected as being placed in the user's ear and the next step of the fit test has not yet been performed).

[0215] Figure 5K Shown in response to input 533 ( Figure 5J ). User interface 521-6 includes one or more indications that a fit test is in progress (e.g., in this case, a second time), such as animation 530 and the text "Fit test in progress." Figure 5K , the user interface 521-6 continues to indicate that the left earbud 502-2 does not meet the fit criteria, as indicated by the patterned fill of the indicator 529-2. Figure 5H As explained, in some embodiments, the appearance of indicator 529-2 is not updated during the fit test, but is updated after the fit test has been performed based on the results of the fit test (e.g., the appearance of indicator 529-2 is updated after the fit test has been performed). Figure 5J In and in Figure 5K (same as in [ ].

[0216] Figure 5LAn exemplary user interface 521-7 is shown that displays and presents the results of the fit test after the fit test has been performed (e.g., in response to completion of the fit test). Indicators 529-1 and 529-2 in the user interface 521-7 respectively show the status of earplugs 502-1 and 502-2. Specifically, indicator 529-2 indicates that earplug 502-2 still does not meet the fit standard, while indicator 529-1 indicates that earplug 502-1 continues to meet the fit standard. In addition, user interface 521-7 includes instructions 531-2, which prompts the user to replace the earplug 527-2 on earplug 502-2 to achieve a better fit and then reinsert the earplug with the new earplug into ear 528-2. Below instruction 531-2, button 532-2 is displayed to continue the fit test. In some embodiments, button 532-2 is inactivatable (e.g., grayed out) until device 100 determines that the user has complied with instruction 531-2 (e.g., by detecting or receiving information indicating that earbud 502-2 has been removed from the ear and replaced, or in some embodiments where device 100 is able to identify which eartip is attached to a particular eartip, by detecting that the eartip attached to eartip 502-2 has been replaced), after which button 532-2 becomes activatable. For example, in some embodiments, button 532-2 is inactivatable ( Figure 5O ).

[0217] Figures 5M to 5N Replacement of the EarTip attached to earbud 502-2 is shown. Figures 5M to 5N Shown with Figure 5L Same user interface 521-7, except Figures 5M to 5N The earbud 502-2 is shown as having been removed from the ear 528-2. Thus, the indicator 529-2 has changed to a gray circle to indicate that the earbud 502-2 is no longer in the user's ear. Figures 5M to 5N The EarTip 527-2 is shown replaced with a different EarTip 527-3 (eg, a larger EarTip to accommodate a larger ear).

[0218] Figure 5O The earbud 502-2, now with the eartip 527-3 attached, is shown to have been replaced (e.g., put back) in the ear 528-2. In response to the user replacing the left earbud 502-2 in the ear 528-2, the indicator 529-2 changes from a gray circle to a black circle, indicating that the earbud 502-2 has been detected as being placed in the user's ear and the next step of the fit test has not yet been performed. Figure 5O Also shown is input 534 received at button 532 - 2 (labeled “Continue”) to continue the fit test.

[0219] Figure 5P Shown from Figure 5O Specifically, Figure 5P Exemplary user interface 521-8 is shown, displayed in response to input 534 at button 532-2, and including one or more indications that a fit test is in progress (e.g., in this case, for the third time), such as animation 530 and the text "FIT TEST IN PROGRESS." Figure 5P In the example shown, indicators 529-1 and 529-2 are displayed during the fit test (e.g., Figure 5P ) has the same characteristics as before performing the fit test (e.g., as Figure 5O Same appearance as shown).

[0220] Figure 5Q An exemplary user interface 521-9 is shown that displays and presents the results of the fit test after the fit test has been performed (e.g., in response to completion of the fit test). Indicators 529-1 and 529-2 in the user interface 521-9 show the status of earplugs 502-1 and 502-2, respectively. Specifically, indicator 529-1 (e.g., a green circle with a check mark) indicates that earplug 502-1 (e.g., still) meets the fit standard assessed by the fit test. Indicator 529-2 has been updated to a different color (e.g., yellow) represented by a changed fill pattern, indicating that earplug 502-2 (e.g., still) does not meet the fit standard (e.g., because earplug 502-2 is still not properly aligned in ear 528-2). In addition, user interface 521-9 includes instructions 531-3 prompting the user to adjust the placement of the left earplug 502-2 in the user's left ear. Below instruction 531-3, a button 532-3 (labeled "Continue") is displayed to continue the fit test. In some embodiments, button 532-3 is inactivatable (e.g., grayed out) until device 100 determines that the user has complied with instruction 531-3 (e.g., by detecting or receiving information indicating that earbud 502-2 has been removed from the ear (e.g., ear 528-2) and replaced), after which button 532-3 becomes activatable.

[0221] Figure 5R The earbud 502-2 with the eartip 527-3 is shown now properly aligned in the ear 528-2 (eg, in response to instruction 531-3). Figure 5R Shown with Figure 5Q Same user interface 521-9, except Figure 5RIndicator 529-2 in has changed to a black circle in response to earbud 502-2 being replaced in the user's ear, optionally after having changed to a gray circle in response to earbud 502-2 being removed. Figure 5R Also shown is input 535 to continue fit testing at button 532-3.

[0222] Figure 5S In response to input 535 ( Figure 5R ). User interface 521-10 includes one or more indications that a fit test is in progress (e.g., in this case, the fourth time), such as animation 530 and the text "Fit test in progress." Figure 5S In the example shown, indicators 529-1 and 529-2 are displayed during the fit test (e.g., Figure 5S ) has the same characteristics as before performing the fit test (e.g., as Figure 5R Same appearance as shown).

[0223] Figure 5T An exemplary user interface 521-11 is shown that displays and presents the results of the fit test after the fit test has been performed (e.g., in response to the completion of the fit test). Indicators 529-1 and 529-2 in the user interface 521-11 show the status of earplugs 502-1 and 502-2, respectively. Specifically, indicator 529-1 (e.g., a green circle with a check mark) indicates that earplug 502-1 (e.g., still) meets the fit standard evaluated by the fit test. Indicator 529-2 has been updated to a different color (e.g., yellow) represented by a changed fill pattern, indicating that earplug 502-2 (e.g., still) does not meet the fit standard. In addition, user interface 521-11 includes instructions 531-4, which inform the user that the fit test did not reach the desired threshold (e.g., for the amount of ambient audio detected during the fit test and / or the calibration tone). The user interface 521-11 also includes an identification of the Ear Tip that produced the best fit test result (e.g., "First Ear Tip"). In some embodiments, the user interface 521-11 is displayed based at least in part on a determination that fit test attempts have been performed using multiple Ear Tips, none of which were successful (e.g., the Ear Tip did not meet the fit criteria during any of the fit test attempts). Below the instruction 531-4, two buttons are shown: a button 536 (labeled "Done") and a button 537 (labeled "Restart Test"). The "Done" button 536, when selected, exits the fit test user interface regardless of the results of the test. The "Restart Test" button 537, when selected, will restart the fit test sequence (e.g., by redisplaying a button similar to Figure 5F User interface 521-2 in (or alternatively, Figure 5H The user interface 521-4) of the user interface and repeat the fit test process, as described in this article Figures 5F to 5S described above).

[0224] Figure 5U to Figure 5V The results of the fit test are shown. Specifically, Figure 5U Shown with Figure 5T The same user interface 521 - 11 and input 538 at the “Done” button 536 . Figure 5V An exemplary user interface 521-12 is shown displayed in response to input 538. The user interface 521-12 indicates that the fit test has been completed and can then be accessed via the settings menu. Figures 9A to 9B In some embodiments, as shown in Figure 5V In the example shown, the user interface 521 - 12 includes a visual indicator (eg, a single green check mark) indicating that the test is complete.

[0225] 6A to 6O An exemplary user interface is shown for alerting a user when the user's earbuds no longer meet fit criteria during use, according to some embodiments. Figure 6A Portable multifunction device 100 is shown paired with earbuds 502-1 and 502-2. Figure 5A to Figure 5V different, 6A to 6O The notification in (also referred to as an "alert") is triggered during use of the earbud 502 outside of pairing and calibration using the settings menu, and optionally when listening to media content (e.g., music, soundtracks of movies and television shows, etc.) using the earbud 502. In some embodiments, where one or more portions of the audio played via the earbud 502 are similar to the calibration tone used for fit testing (e.g., as described herein with reference to Figure 5H As described above), the device 100 uses corresponding portions of the audio to perform a fit test (e.g., so that a fit test can be performed without playing a calibration tone that might interfere with the media content being played). In this way, a fit test can be performed so that notifications about the fit of the earbuds can be provided while the user is using the earbuds outside of the earbud setup process (e.g., as described herein with reference to 6A to 6O Notifications 610, 615, 617, 618, and 619).

[0226] exist Figure 6A, earbuds 502 have been placed in the user's ears 528-1 and 528-2, and as indicated by status indicators 602 and 603 (e.g., two green circles with check marks), the fit of earbud 502-1 in ear 528-1 and the fit of earbud 502-2 in ear 528-2 meet the fit criteria. Figure 6A Also shown is an exemplary user interface 601-1 of a fitness application that is associated with the user's current activity (e.g., running) and is being used to play media content (e.g., music) via earbuds 502, which are in communication (e.g., wired or wireless) with device 100. 6A to 6O The examples in FIG. 5 are based on a fitness application, but one of ordinary skill will recognize that the user may be presented when a different application is in use (e.g., as long as the application is different from the earbud setup application used to manage earbud settings and perform earbud setup, and optionally, when the application in use is being used to play media content via the earbud 502). 6A to 6O In some embodiments, 6A to 6O The notification described in is displayed above at least a portion of the user interface of the application that the user is using. In some embodiments, the notification is triggered when the touch screen 112 of the device 100 is turned off. 6A to 6O , thereby causing touch screen 112 to turn on so that these notifications are displayed on the lock screen of device 100. Figure 6A In the example shown, media content (e.g., a song or track) from album 604 (entitled "Album") is playing. User interface 601-1 also includes audio playback controls, such as a previous button 606 for moving to the previous track or rewinding the audio being played, a play / pause button 607 for toggling audio playback on and off, and a next button 608 for moving to the next track or fast-forwarding through the audio being played. Volume control 605 is also displayed below the audio playback controls.

[0227] Figure 6B It is shown that earbud 502-2 in ear 528-1 has become misaligned. As a result, status indicator 603 has changed (e.g., to a yellow circle with the letter "L") to indicate that earbud 502-2 no longer meets the fit criteria. Additionally, notification 610 is displayed to alert the user to the fit issue with earbud 502 and prompt the user to check the fit of earbud 502 in the user's ear. 6A to 6O610 and other notifications are shown as banner notifications (e.g., overlaid on a portion of the user interface of an application, such as a fitness application), but corresponding notifications may also or alternatively be shown on the lock screen of the device 100 and / or in the notification user interface. In some embodiments, an audio output corresponding to the notification 610 is played (e.g., via the earbuds 502) and / or a tactile output corresponding to the notification 610 is provided. One of ordinary skill in the art will recognize that any combination of alerts (e.g., visual, audio, and / or tactile, etc.) can be used to alert the user that one or more earbuds no longer meet the fit criteria.

[0228] Figures 6C to 6E 6 shows an exemplary user response to a notification 610 alerting the user to one or more issues with the fit of the earbud 502. Specifically, Figure 6C Input 611 is shown received at notification 610 to initiate the fit test user interface. Figure 6D Shown is a diagram of a fit test user interface 612-1 (e.g., similar to a user interface 612) displayed in response to input 611 at notification 610. Figure 5G 521-3 is shown. User interface 612-1 includes indicators 629-1 and 629-2 that indicate the status of right earbud 502-1 and left earbud 502-2, respectively, and correspond to status indicators 602 and 603, respectively. Furthermore, user interface 612-1 includes a button 613 that can be activated to start a fit test and an input 614 that is received at button 613 to start a fit test. Figure 6E User interface 612-2 is shown replacing the user interface 612-2 in response to receiving input 614 at button 613 to start the fit test. Figure 6D User interface 612-1. User interface 612-2 includes one or more indications that the fit test is being performed (e.g., in progress), such as animation 530 and the text "Fit test in progress." Note that in Figures 6D to 6E In the example shown, the user interface 612 is displayed in a window. In some embodiments, the user interface 612 is displayed on the entirety (or substantially all, e.g., greater than 95%, 96%, 97%, 98%, or 99%) of the touch screen 112, rather than in a window.

[0229] Figure 6F6 and 603 indicate that the earplug 502 does not meet the fit standard (e.g., as indicated by the patterned fill of the status indicators 602 and 603, which represents yellow). Therefore, a notification 615 is displayed to prompt the user to change the earplug attached to the earplug 502 to a larger earplug based on the user's current activity (e.g., based on the fact that the user is running, to achieve a better earplug fit during running). In some embodiments, the device 100 (e.g., automatically) detects the user's current activity. In some embodiments, the user indicates his or her current activity to the device 100 (e.g., by launching a fitness application). In some embodiments, monitoring of the fit of the earplug 502 is performed during a specific activity such as exercise (e.g., running).

[0230] Figures 6G to 6H The earbuds 502-1 and 502-2 are shown removed from the user's ears 528-1 and 528-2, as indicated by status indicators 602 and 603 which have become solid grey circles. Figure 6G Also shown are currently attached EarTips 616-1 and 616-2 being replaced with larger EarTips 616-3 and 616-4, respectively. Notably, EarTips 616-4 is in poor condition (e.g., has some debris (e.g., earwax) or is damaged (e.g., torn EarTips)). Figure 6H It is shown that earbuds 502-1 and 502-2 are no longer coupled to earbuds 616-1 and 616-2, respectively, but are now coupled to earbuds 616-3 and 616-4, respectively. Figures 6G to 6H In the example shown, in response to the user at least partially complying with the prompt in the notification 615 (e.g., by removing the earbud 502), the notification 615 stops being displayed. Optionally, in some embodiments, the notification 615 continues to be displayed after the earbud 502 is removed. Additionally, in some embodiments, such as in Figures 6G to 6H In the example shown, when the earbuds 502 are removed from the user's ears, the device 100 continues to play audio. Optionally, in some embodiments, the device 100 pauses audio playback in response to one or both earbuds being removed, and optionally resumes playback when the earbuds are replaced in the ears.

[0231] Figure 6I6-4. FIG. 6 illustrates earbuds 502-1 and 502-2 replaced in the user's ears 528-1 and 528-2, respectively. Indicator 602 indicates that the right earbud 502-1 and EarTip 616-3, placed in the right ear 528-1, meet the fit criteria, while indicator 603 indicates that the left earbud 502-2 and EarTip 616-4, placed in the left ear 528-2, do not meet the fit criteria (e.g., as indicated by the patterned fill of status indicator 603, which is yellow). In this exemplary case, the earbud 502-2 and EarTip 616-4 do not meet the fit criteria because the EarTip 616-4 is in poor condition (e.g., damaged or dirty) and does not form a seal (or a sufficient seal) within the ear 528-2. Based on the EarTip 616 - 4 attached to the earbud 502 - 2 being in poor condition (eg, damaged or dirty), a notification 617 is displayed to inform the user that the earbud 502 - 2 and the EarTip 616 - 4 should be cleaned and / or the EarTip 616 - 4 should be replaced.

[0232] Figure 6J The earbud 502-2 is shown removed from the ear 528-2, as indicated by the status indicator 603 having changed to a solid grey circle. Figure 6J Also shown is the EarTip 616-4 being cleaned. Figure 6J In the example shown, notification 617 ceases to be displayed in response to the user at least partially complying with the prompt in notification 617 (e.g., by removing earbud 502-2). Optionally, in some embodiments, notification 617 continues to be displayed after earbud 502-2 is removed.

[0233] Figure 6K The earbud 502-2 is shown replaced in the ear 528-2 after being cleaned. However, the earbud 502-2 is not yet properly positioned in the ear 528-2 and therefore does not meet the fit criteria, as indicated by the status indicator 603 (e.g., a patterned fill indicating yellow). Therefore, the device 100 displays a notification 618 to prompt the user to correct the placement of the left earbud 502-2 in the left ear 528-2.

[0234] Figure 6L The earbud 502-2 is shown properly aligned in the ear 528-2 (e.g., after being repositioned in response to the notification 618). Figure 6K ). However, ear tip 502-2 does not (e.g., still) meet the fit criteria, as indicated by status indicator 603 (e.g., a patterned fill indicating yellow). Therefore, device 100 displays notification 619 to prompt the user to replace the current ear tip 616-4 with an ear tip of a different size (e.g., smaller) in an attempt to improve the ear tip fit.

[0235] Figures 6M to 6NThe earbud 502-2 is shown removed from the ear 528-2, as indicated by the status indicator 603 having changed to a solid grey circle. Figure 6M Also shown is the EarTip 616-4 (which is currently attached to the earbud 502-2) being replaced with a smaller EarTip 616-5. Figure 6N The earbud 502-2 is no longer coupled to the EarTip 616-4, but is now coupled to the smaller EarTip 616-5. The now attached EarTip 616-4 is then placed next to the smaller EarTip 616-5.

[0236] Figure 6O The earbud 502-2 (now with the eartip 616-5 attached) is shown being replaced in the user's ear 528-2. The status indicator 603 indicates that the earbud 502-2 is now properly seated in the ear 528-2. In some embodiments, the device 100 displays an indication that the earbud 502-2 is now properly seated in the user's ear (e.g., an indication that both earbuds 502 are now properly seated in the user's ear) (e.g., after the earbud 502-2 (with the eartip 616-5 attached) is replaced in the ear 528-2 and, based on determining that the earbud 502-2 meets the fit criteria, the fit test is repeated).

[0237] Figures 7A to 7Q An exemplary user interface and user interaction for changing the audio output mode of a wearable audio output device according to some embodiments are shown.

[0238] Figure 7A An exemplary user interface 700 is shown, which is similar to the reference Figure 6A User interface 601-1 is shown and described. User interface 700 is a user interface of a fitness application that is associated with the user's current activity (e.g., running) and is being used to play media content (e.g., music) via earbud 502. Figure 7A A swipe gesture 701 is shown starting from an initial position 701 - 1 in the upper right corner of the touch screen 112 and moving downward across the touch screen 112 .

[0239] Figure 7B Device 100 is shown displaying a first portion of a settings user interface 702 (also sometimes referred to as a control panel user interface) over at least a portion of user interface 700 in response to a first portion of a swipe gesture 701 from an initial position 701 - 1 to a second position 701 - 2 .

[0240] Figure 7CThe settings user interface 702 is shown displayed over the entirety (or substantially all, e.g., greater than 95%, 96%, 97%, 98%, or 99%) of the user interface 700 (e.g., in response to ceasing to detect the swipe gesture 701, optionally after the swipe gesture 701 moves further downward across the touch screen 112). In some embodiments, the user interface 700 is at least partially obscured or obscured behind the settings user interface 702, such that different features of the user interface 700 are not recognizable. The settings user interface 702 includes a number of controls for various features of the device 100. Notably, the settings user interface 702 includes a volume control 703. Up and down swipe inputs on the volume control 703 can be provided to increase or decrease the volume of the audio output from the device 100 (e.g., via the earbuds 502), respectively.

[0241] Figure 7D Input 704 at the volume control slider is shown (eg, a press gesture that satisfies an intensity threshold above a nominal contact detection intensity threshold, or a long press gesture that is maintained on touch screen 112 for at least a threshold amount of time).

[0242] Figure 7E Shown from Figure 7D Specifically, Figure 7E An enhanced volume control user interface 705 is shown, which responds to the volume control 703 ( Figure 7D ) is displayed. The enhanced volume control user interface 705 includes a volume control 706, which is an enlarged version of the volume control 703 ( Figure 7D ) is amplified and allows finer (e.g., more granular) volume control. Additionally, the user interface 705 includes a noise management control 707-1 that indicates the audio output mode in which the earbud 502 is currently operating. Figure 7E As shown, the earbud 502 is operating in an active pass-through audio output mode, in which one or more pass-through audio components are output so that the user can hear a greater amount of ambient sound from the surrounding physical environment than would otherwise be perceived by the user (e.g., as described herein with reference to FIG. Figure 3C as described), which is indicated by the pass-through icon 711 within the noise management control 707-1. Figure 7E Also shown is input 708 (eg, a tap gesture) at noise management control 707 - 1 .

[0243] Figure 7F7. An expanded noise management control 707-2 is shown that includes representations of three available audio output modes for the earbud 502, each of which is associated with a different audio output mode available for the earbud 502. Specifically, the expanded noise management control 707-2 includes a pass-through icon 711, a bypass icon 710, and an active noise control icon 709. The active noise control icon 709 represents an active noise control ("ANC") audio output mode, in which one or more audio cancellation audio components are output to at least partially cancel ambient sounds from the surrounding physical environment that would otherwise be perceived by the user. The bypass icon 710 represents a bypass audio output mode, in which neither an audio cancellation audio component nor a pass-through audio component is provided (e.g., any amount of ambient sound perceived by the user is due to the physical attenuation of the earbud 502 (and any attached earplug) in the user's ear). A selection indicator 713 displayed above the pass-through icon 711 (eg, and not displayed above the bypass icon 710 or the active noise control icon 709 ) indicates that the audio pass-through mode represented by the pass-through icon 711 is the mode in which the earbud 502 is currently operating.

[0244] Figure 7G An input 712 (eg, a tap gesture) at the active noise control icon 709 is shown. Figure 7H It is shown that in response to detecting input 712 at the active noise control icon 709, the selection indicator 713 ceases to be displayed over the pass-through icon 711 and is instead displayed over the active noise control icon 709. Additionally, the audio output mode of the earbud 502 changes from the audio pass-through mode represented by the pass-through icon 711 to the active noise control mode represented by the active noise control icon 709. In some embodiments, an audible tone is output to indicate that the audio output mode has changed.

[0245] Figure 7I The expanded noise management control 707-2 is shown to have been collapsed into the noise management control 707-1, which indicates (e.g., only) the audio output mode in which the earbud 502 is currently operating (e.g., the active noise control mode represented by the active noise control icon 709), without displaying any representation of any other audio output mode (e.g., the bypass icon 710 or the pass-through icon 711 is not displayed).

[0246] Figures 7J to 7Q The automatic switching of audio output modes in response to different types of input is shown. Specifically, Figures 7J to 7K The behavior in response to a user removing an earbud from his or her ear is shown.

[0247] Figure 7JThe earbud 502-2 is shown removed from the user's ear 528-2. In response to the removal of the earbud 502-2, the earbuds 502-1 and 502-2 switch from active noise control mode to pass-through mode, as indicated by the pass-through icon 711 now displayed in the noise management controls 707-1. In some embodiments, as Figure 7J As shown, in response to earbud 502-1 or 502-2 being removed from the user's ear (e.g., based on the assumption that the user removed the earbud to better hear the ambient audio), both earbuds 502 switch to the pass-through mode. Additionally, playback of media content to the user via earbud 502 is paused, as indicated by displaying a play button 714 instead of a pause button, as shown in FIG. Figure 7I shown.

[0248] Figure 7K The earbud 502-2 is shown replaced in the ear 528-2. In response to the earbud 502-2 being replaced in the ear 528-2, the earbuds 502-1 and 502-2 switch from the pass-through mode back to the previous mode, which in this case is the active noise control mode as indicated by the active noise control icon 709. In addition, the device 100 resumes media playback, such as by displaying the pause button 715 instead of the pause button 715. Figure 7J As indicated by the play button 714 shown.

[0249] Figures 7L to 7Q Shown is a change in audio output mode in response to detecting a particular type of speech. Figure 7L It is from Figure 7K or Figure 7I An exemplary transformation of Figure 7L , user 716 wearing earbuds 502 - 1 and 502 - 2 is participating in a conversation with a second individual 717 . Figure 7L The user 716 is shown speaking (e.g., in the form of voice 718). In response to detecting the voice 718 of the user 716, the earbud 502 switches from the active noise control mode (e.g., Figure 7K and Figure 7I 9 shown) to switch to pass-through mode, as indicated by the pass-through icon 711 in the noise management controls 707-1. Figure 7L In the example shown, media content playback is paused in response to detecting speech 718, such as by displaying a play button 714 instead of a pause button 715 ( Figure 7K In some embodiments, in response to detecting voice 718, playback of the media content is not paused (e.g., continues to play), although optionally in some embodiments, the volume of the playing media content is reduced.

[0250] Figure 7MThe earbud 502 is shown remaining in the pass-through mode as the conversation between the user 716 and the individual 717 continues with a response (e.g., speech) 719 from 717. In some embodiments, the earbud 502 detects speech (e.g., speech 718 by the user 716, Figure 7L In some embodiments, the earbud 502 remains in the pass-through mode (or transitions back to the pass-through mode) for a predetermined amount of time in response to detecting that someone is speaking to the user 716 (e.g., in the form of a response 719).

[0251] Figure 7N The user 716 and the individual 717 who is no longer participating in the conversation are shown. Thus, the earbud 502 has been restored to the last mode (e.g., when a predetermined amount of time has passed since the last time speech was detected), which in this example is the active noise control mode, as indicated by the active noise control icon 709 in the noise management control 707-1. Optionally, as in Figure 7N As in the example shown, in response to detecting the voice of user 716 (e.g., Figure 7L In the event that playback of the media content is paused due to a voice 718 detected by the user, playback of the media content is resumed after the session ends (e.g., when a predetermined amount of time has passed since the last time voice was detected), such as by Figure 7N A pause button 715 is displayed instead of a play button 714 ( Figure 7M ) is indicated. In the absence of a response to detecting speech (e.g., Figure 7L In some embodiments where the media content is paused due to voice detection 718), playback of the media content continues during the session and after the session ends, although optionally in some embodiments where the volume of the playing media content is reduced (e.g., reduced to a level above zero), the volume is restored (e.g., raised to the same level as before the voice was detected).

[0252] Figure 7O Individual 717 is shown re-initiating a conversation with user 716 in the form of a voice 720 that begins with the name of user 716 (e.g., "Delilah"). In response to detecting that someone has spoken the name of user 716, earbud 502 switches to a pass-through mode, as indicated by the pass-through icon 711 in noise management controls 707-1. Optionally, as in Figure 7O In the example shown, media content playback is paused in response to detecting speech 720, such as by displaying a play button 714 instead of a pause button 715 ( Figure 7N ) indicated.

[0253] Figure 7PUser 716's response 721 to individual 717 is shown while earbud 502 remains in a pass-through mode (eg, to make it easier for user 716 to hear and respond to individual 717), as indicated by pass-through icon 711 in noise management controls 707-1.

[0254] Figure 7Q The user 716 and the individual 717 who is no longer participating in the conversation are shown. Thus, the earbud 502 has been restored to the last mode (e.g., when a predetermined amount of time has passed since the last time speech was detected), which in this example is the active noise control mode, as indicated by the active noise control icon 709 in the noise management control 707-1. Optionally, as described herein with reference to Figure 7N As described, resume media playback, such as by Figure 7Q A pause button 715 is displayed instead of a play button 714 ( Figure 7P ), or alternatively, media playback volume is restored after having been lowered.

[0255] Figures 8A to 8J 1 shows exemplary user interactions with earbuds for controlling audio output according to some embodiments. In some embodiments, the earbuds each include a stem (e.g., extending from the portion of the earbud that is inserted into the user's ear) that the user can use to provide input to the earbud. Figure 8A As shown, earplug 502-2 includes a handle 801. In some embodiments, handle 801 is or includes a pressure-sensitive input device that responds to a pressure input applied to the handle when the handle 801 is held between two fingers of hand 802 and squeezed, such as Figure 8A Although only one earbud with a stem is shown (e.g., earbud 502-2 with stem 801), persons of ordinary skill will recognize that earbud 502-1 can have a similar structure as a corresponding stem, and that the same functionality described herein with reference to earbud 502-2 and stem 801 can also be achieved using earbud 502-1 and its corresponding stem. Figure 8AAlso shown is input 803 received at the handle 801 (e.g., a long squeeze gesture comprising a squeeze of the handle 801 maintained for at least a threshold amount of time) when the current audio output mode of the earbud 502 is the active noise control mode, as indicated by the active noise control icon 709 in the noise management controls 707-1. In some embodiments, an audible tone is output to indicate a "press click," which occurs when the intensity of the input received at the handle 801 has met or exceeded an input intensity threshold (e.g., associated with a squeeze gesture). In some embodiments, an audible tone (e.g., the same as or different from the "press click" audible tone) is output to indicate a "release click," which occurs when an input whose intensity has met or exceeded an input intensity threshold has been released such that the input intensity decreases to or below the input intensity threshold. In some embodiments, setting the "press click" intensity threshold to be higher than the "release click" intensity threshold provides hysteresis so that unintentional fluctuations in input intensity (e.g., due to instability of the user's finger when applying pressure to the input device) do not cause the wearable audio output device to inadvertently detect the release of the input. In some embodiments, the wearable audio output device 301 includes one or more tactile output generators, optionally located in the handle 801. In some such embodiments, an activation tactile output is output to indicate a "press click" and / or a release tactile output is output to indicate a "release click". In some embodiments, the activation tactile output is the same as the release tactile output (e.g., having the same tactile output amplitude, frequency, and pattern). In some embodiments, the activation tactile output is different from the release tactile output (e.g., in terms of amplitude, frequency, and / or pattern) (e.g., so that the user can distinguish between detected activation and detected release).

[0256] Figure 8B In response to input 803, Figure 8A In response to input 803 (e.g., and based on determining that input 803 is a long squeeze gesture), earbud 502 switches from active noise control mode to the next audio output mode, which in this example is bypass mode, in a predefined order of audio output modes, as described by Figure 8B In some embodiments, the predefined order of audio output modes is configurable using a settings menu (e.g., as described herein with reference to Figures 9A to 9C In conjunction with changing the audio output mode, the earbud 502 outputs an audible tone 806-1 indicating that the audio output mode has changed. In some embodiments, the audible tone used to indicate that the audio output mode has changed is different from the audible tone output to indicate a "press click" and the audible tone output to indicate a "release click," so that the user can distinguish between different types of operations performed in response to the input. Figure 8BSubsequent input 804 at handle 801 (eg, another long squeeze gesture) is also shown.

[0257] Figure 8C In response to input 804, Figure 8B In response to input 804 (e.g., and based on determining that input 804 is a long squeeze gesture), earbud 502 switches from bypass mode to the next audio output mode in a predefined order, which in this example is pass-through mode, as indicated by pass-through icon 711 in noise management controls 707-1 in FIG8 . In conjunction with changing the output mode, earbud 502 outputs an audible tone 806-2 indicating that the audio output mode has changed. Figure 8C Subsequent input 805 at handle 801 (eg, another long squeeze gesture) is also shown.

[0258] Figure 8D In response to input 805, Figure 8C In response to input 805 (e.g., and based on determining that input 805 is a long squeeze gesture), earbud 502 switches from the pass-through mode to the next audio output mode in a predefined order. Figures 8A to 8D In the example shown, the pass-through mode is the last audio output mode in the predefined order (e.g., the predefined order includes the active noise control mode, followed by the bypass mode, and then the pass-through mode). Thus, the earbud 502 cycles back to the active noise control mode (e.g., in which the earbud 502 Figure 8A ), as indicated by the active noise control icon 709 in the noise management control 707-1. In addition, Figure 8D An upward swipe gesture 808 from the bottom of the touch screen 112 is shown for dismissing the enhanced volume control user interface 705 .

[0259] Figure 8E The user interface 700 is shown redisplayed after the enhanced volume control user interface 705 is dismissed. The user interface 700 is being used to play media content (eg, music) from an album 812 via the earbuds 502, as indicated by the pause button 715. Figure 8E Also shown is an input 810 at the stem 801 of the earbud 502-2 (e.g., including a threshold amount of time (e.g., reference Figure 8A A short, single squeeze gesture of squeezing the handle 801 that is released within the same threshold amount of time as the long squeeze gesture described in .

[0260] Figure 8F In response to input 810, Figure 8EIn response to input 810 (e.g., and based on determining that input 810 is a single squeeze gesture), playback of media content from album 812 is paused, such as by Figure 8F A play button 714 is displayed instead of a pause button 715 ( Figure 8E ) indicated. Figure 8F Subsequent input 811 at handle 801 (eg, another single squeeze gesture) is also shown.

[0261] Figure 8G Shown in response to input 811 from Figure 8F In response to input 811 (e.g., and based on determining that input 811 is a single squeeze gesture), playback of media content from album 812 is resumed, such as by Figure 8G The pause button 715 is redisplayed instead of the play button 714 ( Figure 8F ) indicated. Figure 8G Also shown is a subsequent input 813 at handle 801 (eg, a double squeeze gesture comprising two single squeeze gestures made within a threshold amount of time of each other).

[0262] Figure 8H In response to input 813, Figure 8G In response to input 813 (e.g., and based on determining that input 813 is a double squeeze gesture), device 100 stops playing the media content from album 812 and skips forward to the next track from album 814 (named “Album 2”). Functionally, this double press gesture is similar to a tap gesture for next button 815.

[0263] Figure 8I Shown with Figure 8H The same user interface also shows input 816 (eg, a triple squeeze gesture) made by two fingers of hand 802 at the stem 801 of earbud 502-2.

[0264] Figure 8J In response to input 816, Figure 8I In response to input 816 (e.g., and based on determining that input 816 is a triple squeeze gesture), device 100 stops playing media content from album 814 and returns to the previous track from album 812 (named “Album”). Functionally, the triple squeeze gesture is similar to a tap gesture for previous button 817.

[0265] Figures 9A to 9C An example of a settings user interface 900 for controlling various features associated with earbuds (e.g., earbuds 502) and controlling audio output modes is shown in accordance with some embodiments. Specifically, Figure 9AA settings user interface 900-1 is shown that includes a plurality of activatable control options, such as the following options, or a subset or superset thereof:

[0266] A control option 901 , labeled “Disconnect,” for disconnecting the earbud from the device 100 ;

[0267] A control option 902 , labeled “Forget this device,” for disconnecting (e.g., unpairing) the earbuds from the device 100 ;

[0268] • Control option 903, labeled "Name," for assigning a name to the earbud (e.g., Figure 9A "Delilah's Earbuds" in the example shown);

[0269] Control option 904, labeled "Single Click," for assigning a type of action to be performed (e.g., currently "Play / Pause") in response to receiving a single click (e.g., also referred to herein as a "single squeeze") gesture at the earbud (e.g., using the stem of the earbud)

[0270] Operation type);

[0271] Control option 905, labeled “Double Click,” for assigning a type of action to be performed (e.g., currently “Next Track”) in response to receiving a double click (e.g., also referred to herein as a “double squeeze”) gesture at the earbud (e.g., using the stem of the earbud)

[0272] Operation type);

[0273] • Control option 906, labeled “Click and Hold,” for responding to receiving a click and hold (e.g., also referred to herein as a click and hold) at the earbud (e.g., using the stem of the earbud)

[0274] "Long squeeze") gesture to assign the type of action to be performed (e.g., currently the "noise management" action type); and

[0275] • Control option 907, labeled "Auto Switch Mode," for controlling automatic switching between different audio output modes (eg, via earbuds).

[0276] also, Figure 9A Input 908 (eg, a tap gesture) received at control option 906 is shown.

[0277] Figure 9BA user interface 900-2 (e.g., a submenu of the settings user interface 900-1) is shown displayed in response to input 908 (e.g., replacing the display of the settings user interface 900-1). The user interface 900-2 allows the user to respond to a tap and hold gesture (e.g., on the stem 801 of the earbud 502 or other input device 308 ( Figure 3B ) to select the type of action to perform. Options for the type of action to perform include the following options, or a subset or superset of them:

[0278] Option 909, labeled “Siri,” selecting which causes a tap-and-hold gesture to invoke the virtual assistant;

[0279] Option 910, labeled "Play / Pause," selecting which causes a tap-and-hold gesture to toggle playback of media content (e.g., music, a soundtrack for a television show or movie, etc.);

[0280] Option 911, labeled "Next Track," selecting which causes a tap-and-hold gesture to switch to playing the next track (e.g., in a list of tracks, such as a playlist or album list);

[0281] Option 912, labeled "Previous Track," selecting which causes a tap-and-hold gesture to switch to playing the previous track (e.g., in a track list); and

[0282] Option 913, labeled "Noise Management," selection of which causes a tap and hold gesture to toggle between selected options for audio output modes listed under option 913, such as:

[0283] An active noise control mode option 914 , labeled “Active Noise Control,” in which the earbuds output one or more audio-canceling audio components to at least partially cancel ambient sounds;

[0284] o A bypass mode option 915, labeled "Bypass (Off)", in which the earbuds output neither the audio cancellation audio component nor the pass-through audio component; and

[0285] o A pass-through mode option 916, labeled “Active Pass-through,” in which the earbuds output one or more pass-through audio components so that the user can hear a greater amount of ambient sound (e.g., a greater amount of ambient sound than would be heard with passive attenuation of the earbuds placed in the ears).

[0286] In some embodiments, selecting the above option assigns the selected type of operation to the operation of both earbuds in a pair of earbuds (e.g., both earbuds 502-1 and 502-2 of earbuds 502). In some embodiments, the two earbuds in a pair of earbuds can be configured (e.g., via the settings user interface 900) to perform different operations in response to a particular input gesture.

[0287] Figure 9B Option 913 (e.g., "Noise Management") is shown as the type of operation to be performed in response to the tap-and-hold gesture. Under option 913, all three audio output mode options are selected (e.g., "Active Noise Control," "Bypass (Off)," and "Active Pass-Through"). Thus, in response to the tap-and-hold gesture, the earbud's audio output mode will cycle through active noise control mode, bypass mode, and active pass-through mode, then cycle back to active noise control mode, and so on. Additionally, mode options 914, 915, and 916 can be reordered (e.g., via an input on touchscreen 112 that includes a long press on the corresponding option and then moving the input to a different position in the list of mode options 914, 915, and 916). For example, the user can reorder the list of mode options so that the bypass mode option precedes the active noise control mode option, which is followed by the active pass-through mode option. In such an example, in response to the tap-and-hold gesture, the earbud's audio output mode will cycle through bypass mode, active noise control mode, and active pass-through mode, then cycle back to bypass mode, and so on.

[0288] Additionally, the user may deselect one or more of the mode options 914, 915, and 916 to remove the deselected mode options from the loop, as described below with reference to Figure 9C Describe in more detail.

[0289] Figure 9C 1 shows one or more of the mode options 914 (e.g., “Active Noise Control”), 915 (e.g., “Bypass (Off)”), and 916 (e.g., “Active Pass-Through”) under the selection of different combinations of audio output mode options, e.g., option 913 (e.g., “Noise Management”), as described herein with reference to FIG. Figure 9B ) in response to a plurality of inputs of a particular type (e.g., according to a method configured to control an audio output mode (e.g., by a method as described herein with reference to Figure 9B801 ). Figure 3D A designated noise control button (e.g., Figure 3D Similar functionality is available by pressing a button on the earbud 502-1 and / or earbud 502-2 (or by one or both ear cups 332 of the headset, Figure 3D ) plays audible tone 917.

[0290] Figure 9C Four rows and four columns are shown. Each row represents one of four different groups of audio output mode selections, labeled "Selection #1" 942, "Selection #2" 943, "Selection #3" 944, and "Selection #4" 945 (e.g., the earbud 502 will cycle through these selections in response to a tap-and-hold gesture (also known as a long squeeze gesture)). The selected mode option in the corresponding group is indicated by a check mark next to the mode name. The first column, labeled "Mode #1" 938 and located to the right of the initial state indicator 918, indicates the initial audio output mode of the earbud 502. The remaining columns, labeled "Mode #2" 939, "Mode #3" 940, and "Mode #4" 941, indicate a sequence of audio output modes to which the earbud 502 transitions after (e.g., in response to) three different inputs: a first squeeze gesture 919, a second squeeze gesture 920, and a third squeeze gesture 921 (e.g., a tap-and-hold gesture), each associated with a corresponding column to the right of the corresponding input. While several combinations are shown, one of ordinary skill will recognize that many other combinations of audio output mode selections are possible (e.g., by changing which mode options are selected, how many mode options are selected, and / or the order of the mode options).

[0291] Moving to the first row labeled "Select #1" 942, the user has selected all three mode options: an active noise control mode option, a bypass mode option, and a pass-through mode option. Prior to receiving the first squeeze gesture 919, the earbud 502 was in "Mode #1" 938, which in this example is active noise control mode 922. In response to receiving the first squeeze gesture 919, the earbud 502 transitions from "Mode #1" 938 to "Mode #2" 939—i.e., from active noise control mode 922 to bypass mode 923. In response to receiving the second squeeze gesture 920, the earbud 502 transitions from "Mode #2" 939 to "Mode #3" 940—i.e., from bypass mode 923 to pass-through mode 924. In response to receiving the third squeeze gesture 921, the earbud 502 transitions from "Mode #3" 940 to "Mode #4" 941—i.e., from pass-through mode 924 back to active noise control mode 922. Thus, the user has cycled through all three selected mode options.

[0292] Similarly, in one set of over-ear headphones (e.g. Figure 3D The wearable audio output device 301b) includes buttons for noise control (e.g., Figure 3D In an embodiment of the button 336), pressing (or tapping, actuating, etc.) the button causes the earphone to turn on. Figure 9C 942 ). Thus, the user has cycled through all three selected mode options.

[0293] Moving to the second row labeled "Select #2" 943, the user has selected only two of the three mode options: the active noise control mode option and the bypass mode option (e.g., not the pass-through mode option). Prior to receiving the first squeeze gesture 919, the earbud 502 was in "Mode #1" 938, which in this example is active noise control mode 926. In response to receiving the first squeeze gesture 919, the earbud 502 transitions from "Mode #1" 938 to "Mode #2" 939—i.e., from active noise control mode 926 to bypass mode 927. In response to receiving the second squeeze gesture 920, the earbud 502 transitions from "Mode #2" 939 to "Mode #3" 940—i.e., from bypass mode 927 back to active noise control mode 926. In response to receiving the third squeeze gesture 921, the earbud 502 transitions from "Mode #3" 940 to "Mode #4" 941—i.e., from active noise control mode 926 back to bypass mode 927. Thus, the user has cycled through the two selected mode options. Similarly, in one of the sets of over-ear headphones (e.g. Figure 3D The wearable audio output device 301b) includes buttons for noise control (e.g., Figure 3D In an embodiment of the button 336), pressing the button causes the earphone to Figure 9C 943 . For example, when the headset is in “Mode #1” 938 (e.g., active noise control mode 926 in this example), a first button press causes the headset to transition from “Mode #1” 938 to “Mode #2” 939 (here, from active noise control mode 926 to bypass mode 927). In response to receiving a second button press, the headset transitions from “Mode #2” 939 to “Mode #3” 940 (here, from bypass mode 927 back to active noise control mode 926). In response to receiving a third button press, the headset transitions from “Mode #3” 940 to “Mode #4” 941 (here, from active noise control mode 926 back to bypass mode 927).

[0294] 944, the user has selected two different modes of the three modes: an active noise control mode option and a pass-through mode option (e.g., instead of a bypass mode option). Prior to receiving the first squeeze gesture 919, the earbud 502 was in "Mode #1" 938, which in this example is active noise control mode 930. In response to receiving the first squeeze gesture 919, the earbud 502 transitions from "Mode #1" 938 to "Mode #2" 939—i.e., from active noise control mode 930 to pass-through mode 931. In response to receiving the second squeeze gesture 920, the earbud 502 transitions from "Mode #2" 939 to "Mode #3" 940—i.e., from pass-through mode 931 back to active noise control mode 930. In response to receiving the third squeeze gesture 921, the earbud 502 transitions from "Mode #3" 940 to "Mode #4" 941—i.e., from active noise control mode 930 back to pass-through mode 931. Thus, the user has cycled through the two selected mode options. Similarly, in one set of over-ear headphones (e.g., Figure 3D The wearable audio output device 301b) includes buttons for noise control (e.g., Figure 3D In an embodiment of the button 336), pressing the button causes the earphone to Figure 9C 930 ).

[0295] Finally, moving to the fourth row labeled "Select #4" 945, the user has selected only one of the three mode options: the bypass mode option. Prior to receiving the first squeeze gesture 919, the earbud 502 was in "Mode #1" 938, which in this example is bypass mode 934. In response to receiving the first squeeze gesture 919, because only one mode option was selected, the earbud 502 remains in bypass mode 934 (e.g., "Mode #1" 938 and "Mode #2" 939 are the same). Similarly, in response to receiving the second squeeze gesture 920, the earbud 502 remains in bypass mode 934 (e.g., "Mode #2" 939 and "Mode #3" 940 are also the same). Similarly, in response to receiving the third squeeze gesture 921, the earbud 502 remains in bypass mode 934 (e.g., "Mode #3" 940 and "Mode #4" 941 are the same). Therefore, as long as only one audio output mode option is selected, a squeeze gesture does not change the audio output mode in which the earbud 502 operates. Similarly, in one set of over-ear headphones (e.g. Figure 3D The wearable audio output device 301b) includes buttons for noise control (e.g., Figure 3D In an embodiment of the button 336), if only one audio output mode is selected (e.g., bypass mode 934), the headset operates in the selected audio output mode and pressing the button does not change the audio output mode in which the headset operates.

[0296] Figures 10A to 10H is a flow chart illustrating a method 1000 for pairing and calibrating a wearable audio output device according to some embodiments. The method 1000 may be performed on a display device (e.g., Figure 1A touch-sensitive display system 112 or Figure 3A 340) and a touch-sensitive surface (e.g., Figure 1A touch-sensitive display system 112 or Figure 3A 355) of a computer system (e.g., Figure 1A The portable multifunction device 100 or Figure 3A Some operations in method 1000 may be optionally combined, and / or the order of some operations may be optionally changed.

[0297] As described below, method 1000 provides an improved interface for pairing and calibrating a wearable audio output device (e.g., a headset) with a display device in a computer system (e.g., a smartphone, tablet, or personal computer) to optimize fit and, thereby, the audio experience of the wearable audio output device. During the pairing and calibration process, a user is prompted to place the wearable audio output device in their ear, a calibration tone is output via the wearable audio output device, and the fit of the wearable audio output device is determined based on whether the audio detected while outputting the calibration tone meets specific criteria. If the detected audio does not meet the specific criteria, the user is prompted to adjust the wearable audio output device (e.g., adjust one or both of the wearable audio output devices). Prompts related to placement and adjustment of the wearable audio output device are displayed on the display device to guide the user through the pairing and calibration process, and visual feedback is provided to the user at various points during the process regarding the actions to be performed, whether the process was performed properly, and / or whether the wearable audio output device is properly fitted. Providing improved visual feedback to the user enhances the operability of the computer system and associated devices (e.g., the wearable audio output device and / or the display device) and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. In addition, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., relative to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), thereby further reducing power usage and extending the battery life of the device.

[0298] The method includes establishing 1004 a wireless connection with a pair of wearable audio output devices, the pair of wearable audio output devices including: a first wearable audio output device (e.g., earbuds or in-ear headphones, in some embodiments one of a pair (e.g., Figure 5A In some embodiments, the first wearable audio output device includes one or more first sensors for detecting placement of the first wearable audio output device; and in some embodiments, the second wearable audio output device includes one or more first sensors for detecting placement of the second wearable audio output device (e.g., Figure 3COne or more second sensors placed in the microphones 302-1 and 302-2).

[0299] In some embodiments, after establishing (1006) a wireless connection with the pair of wearable audio output devices, a user interface object is displayed via the display device prompting the user to place the first wearable audio output device in the user's ear and the second wearable audio output device in the user's ear. In some embodiments, the display device may display a visual indicator that the audio output device is not placed in the user's ear, such as by Figures 5F to 5U As indicated by the status indicators 529-1 and 529-2 in .

[0300] Displaying a prompt on a display device to the user to place the wearable audio output device in his or her ear provides visual feedback to the user regarding the actions to be performed to advance the pairing and calibration process. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0301] Method 1000 includes detecting (1008) (e.g., via the one or more first sensors) that a first wearable audio output device has been placed in an ear of a user. In some embodiments, the user (e.g., the wearer of the wearable audio output device) is a user of a computer system, such as Figures 7L to 7Q The method 1000 also includes detecting 1010 (eg, via the one or more second sensors) that a second wearable audio output device has been placed in an ear of the user.

[0302] Method 1000 includes, after (e.g., in response to) detecting 1012 that the first wearable audio output device has been placed in the user's ear and the second wearable audio output device has been placed in the user's ear: in some embodiments, performing subsequent operations in response to detecting that the first wearable audio output device has been placed in the user's ear and the second wearable audio output device has been placed in the user's ear. In some embodiments, after detecting that the first wearable audio output device has been placed in the user's ear and the second wearable audio output device has been placed in the user's ear, and in response to an intervening trigger or input (e.g., user input indicating that the user wants to continue calibrating the first wearable audio output device and the second wearable audio output device, such as Figure 5E , where a user provides input 526-1 over a button to start testing 525-1).

[0303] Figure 10B Shown from Figure 10A Method 1000 is continued in Section B of . Method 1000 includes outputting 1014 a first calibration tone via a first wearable audio output device and a second wearable audio output device. In some embodiments, in response to detecting that the first wearable audio output device has been placed in an ear of a user and the second wearable audio output device has been placed in an ear of the user, outputting 1016 the calibration tone via the first wearable audio output device and the second wearable audio output device occurs (e.g., automatically). In some embodiments, automatic output of the calibration tone occurs each time the wearable audio output device is placed in an ear of the user during setup (e.g., as by Figure 5P ).

[0304] Outputting a calibration tone in response to detecting placement of the wearable audio output device in the user's ear performs a calibration process without requiring further input from the user. Performing an operation (e.g., automatically) when a set of conditions have been met without requiring further user input reduces the number of inputs required to perform the operation, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0305] In some embodiments, the length of the calibration tone is determined (e.g., adjusted) based on whether the detected first audio and second audio meet the device fit criteria associated with the calibration tone 1018. In some embodiments, audio is detected (e.g., sampled) one or more times via the microphones of the first wearable audio output device and the second wearable audio output device during the calibration tone. In some embodiments, the calibration tone continues to be output until the amount of audio other than the calibration tone in the corresponding detected audio samples is less than a threshold amount (e.g., as described herein with reference to operation 1024 of method 1000). In some embodiments, the calibration tone continues to be output until the amount of audio other than the calibration tone in the detected audio converges or reaches stability (e.g., is less than a threshold amount for at least a threshold number of times). In some embodiments, the length of the calibration tone may be determined based on the number of attempts made to meet the device fit criteria.

[0306] Dynamically varying the length of the calibration tone based on the time it takes to determine whether the detected audio meets a particular criterion indicating that the wearable audio output device is properly fitted reduces the number of times calibration needs to be repeated due to inconclusive test results during a fixed-length calibration. Reducing the number of inputs required to perform an operation enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0307] The method includes detecting 1020 first audio via one or more first microphones of a first wearable audio output device (e.g., while outputting a first calibration tone, such as by Figure 5H The method further includes detecting 1022 second audio via one or more second microphones of a second wearable audio output device (e.g., while outputting the first calibration tone, as by Figure 5H ).

[0308] The method of claim 1000, comprising: determining that the detected first audio does not meet a device fit criterion associated with the first calibration tone (e.g., a criterion associated with the quality of a seal formed by the device with the user's ear, which is Figures 5I to 5V , wherein the user performs multiple adjustments in response to not satisfying the fit test), an alert is displayed via the display device prompting the user to perform adjustments to the first wearable audio output device; and based on determining that the detected second audio does not satisfy the device fit standard associated with the first calibration tone, an alert 1024 is displayed via the display device prompting the user to perform adjustments to the second wearable audio output device.

[0309] In some embodiments, determining whether the corresponding detected audio meets the device fit criteria includes comparing the detected audio to a calibration tone and determining that the detected audio includes a threshold amount of audio (e.g., 50 dB, 40 dB, 30 dB, 25 dB, or 20 dB) that is less than the calibration tone. In some embodiments, based on determining that the first detected audio does not meet the device fit criteria and / or the second detected audio does not meet the device fit criteria (e.g., based on determining that at least one of a pair of wearable audio output devices does not meet the device fit criteria), an alert is displayed prompting the user to perform an adjustment to the pair of wearable audio output devices (e.g., to both the first wearable audio output device and the second wearable audio output device, without identifying the specific device on which the adjustment is to be performed, such as Figure 6BIn some embodiments, the user is prompted to perform different actions for different wearable audio output devices of the pair of wearable audio output devices (e.g., Figures 5I to 5V , where the user is prompted to secure only the earbud 502-2 in the user's ear 528-2). In some embodiments, the user is prompted to reposition the corresponding wearable audio output device in the user's ear and / or change the corresponding accessory coupled to the corresponding wearable audio output device to a different accessory (e.g., Figures 5M to 5N , where the user replaces the ear tip 527-2 with a larger ear tip 527-3). In some embodiments, based on determining that the corresponding wearable audio output device meets the device fit standard, the computer system forgoes displaying an alert prompting the user to perform adjustments (e.g., replace accessories or reposition) on the wearable audio output device. (For example, Figure 6A shown)

[0310] In some embodiments, after displaying an alert prompting the user to perform an adjustment of the wearable audio output device, the computer system detects that the user has performed the adjustment, for example, by detecting that the wearable audio output device has been placed (e.g., removed and then replaced) in the user's ear. In some embodiments, for the corresponding wearable audio output device that has been replaced in the user's ear, the computer system detects that the user has performed the adjustment by detecting that an accessory of the corresponding wearable audio output device has been changed (e.g., from a previously detected accessory to a currently detected accessory).

[0311] Figure 10C Continuing from method flow block 1012. In some embodiments, a status indicator is displayed 1026 via a display device, wherein the status indicator indicates whether detection of placement of the first wearable audio output device in the user's ear and placement of the second wearable audio output device in the user's ear (e.g., in-ear detection of the wearable audio output devices) is enabled (e.g., by the computer system). In some embodiments, the user has the option to turn off (e.g., disable) in-ear detection, in which case the computer system will not detect whether the first wearable audio device and the second wearable audio device are placed in the user's ears, and in some such cases, the first wearable audio output device and the second wearable audio output device continue to operate in the same manner (e.g., perform the same function) regardless of whether the first wearable audio output device and the second wearable audio output device are placed in the user's ears (e.g., Figures 9A to 9C , where the user can choose which modes to enable or disable).

[0312] Displaying a status indicator on a display device indicating whether detection of placement of a wearable audio output device in a user's respective ear (sometimes referred to as "in-ear detection") is enabled provides visual feedback to the user indicating the relevant status of the computer system during the pairing and calibration process. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0313] In some embodiments, after (e.g., in response to) detecting 1028 that the first wearable audio output device has been placed in the user's ear and the second wearable audio output device has been placed in the user's ear, a user interface object for initiating a fit test is displayed via a display device. The fit test optimizes (e.g., or helps the user optimize) the fit of the first wearable audio output device in the user's ear and the fit of the second wearable audio output device in the user's ear.

[0314] Displaying a user interface object on a display device, the user interface object is selectable for initiating a fit test of the wearable audio output device (e.g., to continue calibration after pairing) after or in response to detecting that the wearable audio output device has been placed in the user's ear, thereby providing the user with visual feedback confirming the position of the wearable audio output device in the user's ear and facilitating execution of the next step in the calibration process. Providing improved feedback to the user enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending device battery life by enabling the user to use the device more quickly and efficiently. Furthermore, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending device battery life.

[0315] In some embodiments, method 1000 includes displaying (1030) via a display device: a first indication that the first wearable audio output device has been placed in the user's ear (e.g., in response to detecting that the first wearable audio output device has been placed in the user's ear (e.g., Figures 5I to 5Vand a second indication that the second wearable audio output device has been placed in the user's ear (e.g., in response to detecting that the first wearable audio output device has been placed in the user's ear (e.g., Figures 5I to 5V Status indicator 529-2)).

[0316] In some embodiments, displaying a corresponding indication that the corresponding wearable audio output device has been placed in the user's ear includes displaying a user interface object indicating that the corresponding wearable audio output device has been placed in the user's ear. In some embodiments, displaying the corresponding indication includes changing the appearance of a corresponding user interface object (e.g., already displayed) representing the corresponding wearable audio output device, wherein the change in appearance indicates the placement of the corresponding wearable audio output device in the user's ear. In some embodiments, the user interface object is a colored indicator (e.g., a black or yellow circle, such as Figure 5I In some embodiments, the change in the appearance of the user interface object includes a change in the color of the user interface object (e.g., from gray to black, as shown in FIG. Figures 5F to 5G In some embodiments, the user interface object includes an indication of which wearable audio output device the user interface object represents (e.g., the user interface object for the left earbud includes the letter "L," while the user interface object for the right earbud includes the letter "R," as shown). Figure 5F In some embodiments, the user interface object also indicates whether the fit criteria are met. In such embodiments, a different visual indicator is displayed (e.g., the appearance of the visual indicator is changed) when the fit criteria are met (e.g., a green circle or a green circle with a check mark inside it, where the check mark indicates that the wearable audio output device is in the user's ear, as shown). Figure 5J If the user removes the audio device from their ear, a third visual indicator is shown (e.g., a gray circle, as shown). Figure 5M In some embodiments, when the user initially places the wearable audio output device in the user's ear, but before the electronic device determines whether the fit criteria have been met, the visual indicator will be black, and the circle will only turn yellow and / or green after the analysis of the device fit criteria has begun and / or the analysis has been completed.

[0317] Displaying an indication that the wearable audio output device has been placed in the user's ear provides visual feedback to the user that the placement of the wearable audio output device in the user's ear has been detected and confirmed. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0318] In some embodiments, after detecting 1032 that the first wearable audio output device has been placed in the user's ear and the second wearable audio output device has been placed in the user's ear, a user interface showing that a calibration tone is playing is displayed via the display device.

[0319] When the wearable audio output device has been placed in the user's ear, the calibration process is performed by transitioning to the next screen in the pairing and calibration process without requiring further user input. Performing an operation without further user input when a set of conditions has been met (e.g., automatically) reduces the number of inputs required to perform the operation, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0320] Figure 10D and Figure 10E All depends on Figure 10B In some embodiments, the method 1000 includes adjusting the corresponding wearable audio output device, including repositioning 1034 the corresponding wearable audio output device in the user's ear. In some embodiments, the repositioning of the corresponding wearable audio output device is detected via one or more placement sensors on the corresponding wearable audio output device (e.g., Figures 5G to 5I In some embodiments, the accessory is a grommet that attaches to the wearable audio output device (e.g., a silicone eartip accessory on a pair of earbuds that forms a seal around the user's ears, which is removed when the user replaces the eartip). Figures 5M to 5N shown in ).

[0321] Prompting the user to reposition the wearable audio output device in his or her ear provides the user with visual feedback regarding the action to be performed to advance the pairing and calibration process and improves the fit of the wearable audio output device. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. Additionally, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., relative to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), thereby further reducing power usage and extending the battery life of the device.

[0322] In some embodiments, a first wearable audio output device is coupled to a first accessory (e.g., a first earplug), and a second wearable audio output device is coupled to a second accessory (e.g., a second earplug), and the method further comprises: upon (e.g., in response to) detecting repositioning of the first wearable audio output device and the second wearable audio output device: outputting a second calibration tone via the first wearable audio output device and the second wearable audio output device; detecting a third audio via one or more first microphones of the first wearable audio output device; detecting a fourth audio via one or more second microphones of the second wearable audio output device; based on determining that the detected third audio does not meet the device fit criteria associated with the second calibration tone, displaying an alert via a display device prompting a user to replace the first accessory coupled to the first wearable audio output device with a third accessory; and based on determining that the detected fourth audio does not meet the device fit criteria associated with the second calibration tone, displaying an alert via a display device prompting a user to replace the second accessory coupled to the second wearable audio output device with a fourth accessory. In some embodiments, the repositioning prompt is displayed before any accessory is added or removed from the audio output device, such as Figure 5I shown.

[0323] Prompting the user to first reposition the wearable audio output device and then to replace an accessory (such as an eartip) on the wearable audio output device provides the user with visual feedback regarding the preferred order in which actions should be performed as part of the pairing and calibration process, thereby improving the fit of the wearable audio output device. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. Additionally, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), thereby further reducing power usage and extending the battery life of the device.

[0324] In some embodiments of method 1000, adjusting the corresponding wearable audio output device includes replacing 1038 a corresponding accessory coupled to the corresponding wearable audio output device with a different accessory. (For example, an eartip for sealing an in-ear headphone or earbud to the user's ear, which may help maintain the in-ear headphone or earbud in place within the user's ear during movement, such as Figures 5M to 5N , wherein the portable electronic device prompts the user to change to a larger ear tip size. )

[0325] Prompting the user to replace accessories (such as ear tips) on the wearable audio output device provides the user with visual feedback about the actions to be performed to advance the pairing and calibration process and improves the fit of the wearable audio output device. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. In addition, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending the battery life of the device.

[0326] In some embodiments, the method determines whether adjusting the corresponding wearable audio output device includes replacing 1040 the corresponding accessory coupled to the corresponding wearable audio output device with a different accessory or repositioning the corresponding wearable audio output device in the user's ear based on the manner in which the corresponding detected audio does not meet the device fit criteria. In some embodiments, the accessory is a grommet that attaches to the wearable audio output device (e.g., a silicone eartip accessory on a pair of earbuds for forming a seal around the user's ear, such as Figures 5M to 5N 527-2 and 527-3 in FIG.

[0327] Determining whether to reposition the wearable audio output device or replace an accessory based on the results of the calibration test reduces the number of adjustments the user needs to perform, thereby helping the user make progress in the pairing and calibration process and more quickly achieve a good fit for the wearable audio output device. Providing improved feedback to the user and reducing the number of inputs required and the time spent performing the pairing and calibration process enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending device battery life by enabling the user to use the device more quickly and efficiently. Furthermore, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending device battery life.

[0328] In some embodiments, method 1000 includes determining whether adjusting the corresponding wearable audio output device includes replacing 1042 the corresponding accessory coupled to the corresponding wearable audio output device with a larger accessory or a smaller accessory (e.g., compared to the corresponding accessory) based on corresponding detected audio therein (e.g., detected via one or more microphones of the corresponding wearable audio output device, such as Figure 3C ) does not meet the device fit standards.

[0329] Determining whether to recommend replacing the accessory of the wearable audio output device with a larger or smaller accessory based on the results of the calibration test reduces the number of adjustments the user needs to perform, thereby helping the user make progress in the pairing and calibration process and more quickly achieve a good fit for the wearable audio output device. Providing improved feedback to the user and reducing the number of inputs required and the time spent performing the pairing and calibration process enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending device battery life by enabling the user to use the device more quickly and efficiently. Furthermore, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending device battery life.

[0330] In some embodiments, the transition to Figure 10F , which is connected to box 1024, including displaying 1044 an indication that the first wearable audio output device is fitted to the user's ear via the display device based on determining that the detected first audio meets the device fit standard associated with the calibration tone; and displaying an indication that the second wearable audio output device is fitted to the user's ear via the display device based on determining that the detected second audio meets the device fit standard associated with the calibration tone.

[0331] In some embodiments, displaying a corresponding indication that the corresponding wearable audio output device has been placed in the user's ear includes displaying a user interface object (e.g., Figure 5F Gray status indicators 529-1 and 529-2 are shown indicating that the earbuds are not placed in the ears, and Figure 5G In some embodiments, displaying the corresponding indication includes changing the appearance of a corresponding user interface object (e.g., already displayed) representing the corresponding wearable audio output device, wherein the change in appearance indicates the placement of the corresponding wearable audio output device in the user's ear. In some embodiments, the user interface object is a colored indicator (e.g., Figure 5F Gray status indicators 529-1 and 529-2 are shown indicating that the earbuds are not placed in the ears, and Figure 5GBlack status indicators 529-1 and 529-2 are shown indicating that the earbuds are placed in the user's ears. In some embodiments, the change in the appearance of the user interface object includes a change in the color of the user interface object (e.g., from gray to black, as in Figures 5F to 5G In some embodiments, the user interface object includes an indication of which wearable audio output device the user interface object represents (e.g., the user interface object for the left earbud includes the letter "L" and the user interface object for the right earbud includes the letter "R," which Figures 5F to 5G In some embodiments, the user interface object also indicates whether the fit criteria are met. In such embodiments, a different visual indicator is displayed (e.g., the appearance of the visual indicator is changed) when the fit criteria are met (e.g., a green circle or a green circle with a check mark inside it, where the check mark indicates that the wearable audio output device is in the user's ear, such as Figure 5H If the user removes the audio device from their ear, a third visual indicator (e.g., a gray circle, such as Figure 5M 529-2 in the figure, and gray when the user removes the earbud 502-2 to replace the earbud) to indicate that the audio device has been removed from the user's ear. In addition, in some embodiments, when the user initially places the wearable audio output device in the user's ear, but before the electronic device determines whether the fit criteria have been met, the visual indicator will be black, and only after the analysis of the device fit criteria has begun and / or the analysis has been completed, will the circle turn yellow and / or green (for example, Figures 5H to 5I , where the status indicator 529-2 changes from a solid background to a hatched background (or a yellow background).

[0332] Displaying an indication on the display device that the wearable audio output device is fitted to the user's ear provides visual feedback to the user regarding the fit of the wearable audio output device and indicates that the pairing and calibration process has been successfully completed. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. In addition, when the wearable audio output device is fitted to the user's ear, the seal between the wearable audio output device and the user's ear enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as opposed to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), thereby further reducing power usage and extending the battery life of the device.

[0333] In some embodiments, method 1000 includes displaying 1046 an alert prompting the user to perform an adjustment to the first wearable audio output device (e.g., based on determining that the detected first audio does not meet the device fit criteria) independently of whether the detected second audio meets the device fit criteria; and displaying an alert prompting the user to perform an adjustment to the second wearable audio output device (e.g., based on determining that the detected second audio does not meet the device fit criteria) independently of whether the detected first audio meets the device fit criteria.

[0334] In some embodiments, the prompted adjustment of the first wearable audio output device is a first type of adjustment (e.g., repositioning in the user's ear), and no adjustment is prompted for the second wearable audio output device, or the type of adjustment prompted for the second wearable audio output device is a different type of adjustment than the first adjustment (e.g., the type of adjustment prompted for the second wearable audio output device is a second type of adjustment, such as replacing an accessory (e.g., an eartip for the second wearable audio output device)). In some embodiments, a status indicator is presented for one of the wearable audio output devices (e.g., a green circle with a check mark indicating that fit test criteria are met), and a different status indicator is presented for the other wearable audio output device (e.g., a yellow circle with a check mark indicating that fit test criteria are met), such as Figures 5I to 5J shown.

[0335] Determining whether adjustment is required and prompting the user to perform adjustment on one of the wearable audio output devices independently of the other wearable audio output device (e.g., for a pair of wearable audio output devices), providing visual feedback to the user indicating which wearable audio output device specifically requires adjustment, which can reduce the number of adjustments the user needs to perform, thereby helping the user make progress in the pairing and calibration process and achieve a good fit of the wearable audio output device more quickly. Providing improved feedback to the user and reducing the number of inputs required and the time spent performing the pairing and calibration process enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. Additionally, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., relative to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), thereby further reducing power usage and extending the battery life of the device.

[0336] Figure 10G Including from Figure 10DIn some embodiments, the first wearable audio output device is coupled to a first accessory (e.g., a first ear tip) upon detecting 1048 the first audio, and the second wearable audio output device is coupled to a second accessory (e.g., a second ear tip) upon detecting the second audio, and the method further comprises: based on determining that the detected first audio does not meet the device fit criteria and based on determining that the detected second audio does not meet the device fit criteria: after (e.g., in response to) detecting that the first accessory coupled to the first wearable audio output device is replaced with a third accessory and the second accessory coupled to the second wearable audio output device is replaced with a fourth accessory: outputting a second calibration tone via the first wearable audio output device and the second wearable audio output device; and outputting a second calibration tone via one or more first wearable audio output devices. The microphone detects a third audio; detects a fourth audio via one or more second microphones of a second wearable audio output device; based on determining that the detected third audio does not meet the device fit standard associated with the second calibration tone: determines whether the detected first audio or the detected third audio is closer to meeting the device fit standard; and displays, via a display device, an alert prompting the user to use the corresponding accessory of the first accessory and the third accessory that is closer to meeting the device fit standard; based on determining that the detected fourth audio does not meet the device fit standard associated with the second calibration tone: determines whether the detected second audio or the detected fourth audio is closer to meeting the device fit standard; and displays, via a display device, an alert prompting the user to use the corresponding accessory of the second accessory and the fourth accessory that is closer to meeting the device fit standard. In some embodiments, the computer system uses data retrieved from a first attempt to meet the device fit standard (e.g., a calibration attempt when using the first accessory and the second accessory).

[0337] When calibration has been attempted with multiple accessories (e.g., ear tips), prompting the user to use the accessory that comes closest to meeting the device fit criteria provides the user with visual feedback indicating which accessory will provide the best possible audio experience, even if no accessory fully meets the device fit criteria. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. Additionally, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., relative to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending the battery life of the device.

[0338] Figure 10H Including from Figure 10D In some embodiments, after detecting that the first wearable audio output device has been replaced in the user's ear (e.g., after being removed from the user's ear) and the second wearable audio output device has been replaced in the user's ear (e.g., after being removed from the user's ear), displaying, via a display device, a user interface including a user interface object that, when selected, causes the computer system to: output, via the first wearable audio output device and the second wearable audio output device, a second calibration tone; detect, via the one or more first microphones of the first wearable audio output device, a third audio; detect, via the one or more second microphones of the second wearable audio output device, a fourth audio; based on a determination that the detected second audio does not meet a device fit criterion associated with the second calibration tone (e.g., a criterion associated with a quality of a seal formed by the device with the user's ear), displaying, via the display device, an alert prompting the user to perform an adjustment to the first wearable audio output device; and based on a determination that the detected third audio does not meet a device fit criterion associated with the second calibration tone, displaying, via the display device, an alert prompting the user to perform an adjustment to the second wearable audio output device.

[0339] In some embodiments, determining whether the corresponding detected audio meets the device fit criteria includes comparing the detected audio to a calibration tone and determining that the detected audio includes audio that is less than a threshold amount other than the calibration tone (e.g., 50 dB, 40 dB, 30 dB, 25 dB, or 20 dB). In some embodiments, when the user selects a second user interface object, the device uses the last detected audio to meet the device fit criteria.

[0340] Providing a user interface object to re-output the calibration tone and repeat the calibration process after detecting that the wearable audio output device has been replaced in the user's ear provides visual feedback to the user regarding the next action to be performed as part of the pairing and calibration process and reduces the number of inputs required by the user to perform the next action. Providing improved feedback to the user and reducing the number of inputs required to perform an action enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0341] It should be understood that Figures 10A to 10H The specific order in which the operations are described in the foregoing is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will appreciate various ways to reorder the operations described herein. Additionally, it should be noted that the details of other processes described herein with respect to other methods described herein (e.g., methods 1100, 1200, 1800, 1900, 2000, 2400, and 2600) also apply in a similar manner to the details of the processes described above with respect to Figures 10A to 10H Method 1000 is described. For example, the devices, user interfaces, audio outputs, audio output modes, alerts, adjustments, and accessories described above with reference to method 1000 optionally have one or more of the features of the devices, user interfaces, audio outputs, audio output modes, alerts, adjustments, and accessories described herein with reference to other methods described herein (e.g., methods 1100, 1200, 1800, 1900, 2000, 2400, and 2600). For the sake of brevity, these details are not repeated here.

[0342] 11A to 11D 1 is a flow chart illustrating a method 1100 for monitoring the fit of a wearable audio output device during use (e.g., outside of a setup user interface) according to some embodiments. Figure 1A touch-sensitive display system 112 or Figure 3A340) and with one or more wearable audio output devices (e.g., Figure 3B A computer system (e.g., Figure 1A The portable multifunction device 100 or Figure 3A 300), wherein the computer system is configured to execute multiple applications (e.g., Figure 1A 136 in the application), and the one or more wearable audio output devices include one or more sensors for detecting placement of the one or more wearable audio output devices (e.g., Figure 3B placement sensor 304 in the ) and one or more microphones (e.g., Figure 3B Some operations in method 1100 are optionally combined, and / or the order of some operations is optionally changed.

[0343] As described below, method 1100 provides an improved interface for optimizing the audio experience by intelligently suggesting adjustments to the wearable audio output device in response to detecting that the wearable audio device no longer meets device fit criteria while the user is using the wearable audio output device (e.g., outside of a pairing and calibration process). This provides visual feedback to the user indicating that an issue with the wearable audio output device that may affect the audio experience has been detected, and helps the user improve the fit of the wearable audio output device. Providing improved visual feedback to the user enhances the operability of the computer system and associated devices (e.g., the wearable audio output device and / or the display device) and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. Additionally, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., relative to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), thereby further reducing power usage and extending the battery life of the device.

[0344] The method 1100 includes 1104 when the one or more wearable audio output devices are in one or more respective positions relative to the user's ears and a media presentation application on a computer system is being used to play media via the one or more wearable audio output devices without displaying a setup user interface for configuring a fit of the one or more wearable audio output devices.

[0345] The method 1100 also includes providing 1106 audio output based on the media from the media presentation application via the one or more wearable audio output devices, wherein the media presentation application is separate from the settings user interface; in some embodiments, the computer system also monitors (e.g., continuously or at repeated intervals) audio output via one or more microphones (e.g., such as Figure 3C 302 in the embodiment of the present invention) to determine whether the one or more wearable audio output devices meet the device fit standard.

[0346] In addition, method 1100 includes determining 1108 that the one or more wearable audio output devices no longer meet the device fit criteria based on audio output based on media from the media presentation application; and in response to determining that the one or more wearable audio output devices no longer meet the device fit criteria, displaying 1110 on a display device an alert corresponding to (e.g., including) information about the fit of the one or more wearable audio output devices. In some embodiments, based on determining that the user is not in a corresponding predefined context (e.g., the user is in a context outside of one or more predefined contexts for which device fit is monitored, such as Figures 6A to 6L The computer system does not display the information on the display device (e.g., the fitness application shown in FIG. Figure 6A ) prompting a user to perform an adjustment to one or more wearable audio output devices, even though the one or more wearable audio output devices no longer meet device fit criteria, as determined based on audio output based on media from a media presentation application.

[0347] See also Figure 11B , which originates from Figure 11A In method block 1106, optionally, based on one or more audio properties of the audio output of the media from the media presentation application and the audio output of the media for (e.g., via Figures 9A to 9B In some embodiments, the audio output based on the media from the media presentation application has a sound distribution (e.g., frequency distribution) similar to the calibration tone for configuring the fit of the one or more wearable audio output devices using the settings user interface shown in the figure. In other words, in some embodiments, the audio output based on the media from the media presentation application has a sound distribution (e.g., frequency distribution) similar to the calibration tone for at least a short period of time. Alternatively, the audio output based on the media has sufficient energy in an audio frequency band corresponding to the audio frequency band of the calibration tone for at least a short period of time to serve as an equivalent to the calibration tone.

[0348] Performing calibration while the user is using the wearable audio output device to listen to audio (e.g., outside of a pairing and calibration process) effectively monitors the fit of the wearable audio output device and enables feedback to be provided to the user regarding the fit without interrupting the audio experience and without requiring the user to separately navigate to a settings user interface to initiate the calibration process. This provides improved feedback and reduces the number of inputs required to monitor the fit of the device, enhancing the operability of the device and making the user-device interface more efficient (e.g., by helping the user achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0349] Now go to Figure 11C , which originates from method block 1108 in FIG. 11 , optionally includes, in some embodiments, performing a determination 1114 that the one or more wearable audio output devices no longer meet device fit criteria based on determining that the user is within a corresponding context of one or more predefined contexts.

[0350] In some embodiments, when a user is performing one or more predefined activities (e.g., exercising, such as running, walking, swimming, etc., Figures 6A to 6L When a user is in a specific activity in a fitness application (such as a fitness application in a computer), the user is in a corresponding context of one or more predefined contexts. The one or more predefined activities can be specified by the user or defined by the computer system (e.g., by default). Whether the user is in a corresponding context can be specified by the user (e.g., by launching a fitness application, such as a fitness application in a computer). Figures 6A to 6L In some embodiments, based on determining that the user is not in a corresponding context of the one or more predefined contexts, when a media presentation application on the computer system is used to play media without displaying a settings user interface, the computer system abandons determining whether the one or more wearable audio output devices no longer meet the device fit criteria.

[0351] Monitoring device fit during a specific context (e.g., during a specific activity) enables providing feedback to the user regarding the fit of the wearable audio output device when relevant or desired, without requiring the user to separately navigate to a settings user interface to initiate a calibration process. Providing improved feedback to the user and performing an action without further user input when a set of conditions have been met (e.g., automatically) reduces the number of inputs required to perform an action (e.g., monitoring device fit), which enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0352] Go to Figure 11D , which originates from Figure 11A 110, optionally including additional features. The method 1100 optionally includes, after displaying 1116 on a display device an alert corresponding to information about the fit of the one or more wearable audio output devices, detecting an input corresponding to the alert (e.g., a tap input on a touch-sensitive surface of a computer system at a location corresponding to the displayed alert). In some embodiments, the alert is a notification displayed above at least a portion of a currently displayed user interface, such as Figure 6B -1 to Figure 6B -3. In some such embodiments, a corresponding alert is shown in the notification user interface. In some embodiments, the corresponding alert is an audio alert played at one or more of the audio output devices in response to detecting an input corresponding to the alert: a settings user interface is displayed. In some embodiments, the settings user interface is a user interface specific to audio output (e.g., Figures 9A to 9B In some embodiments, where the computer system is displaying a particular user interface, such as a media application user interface (e.g., Figures 6A to 6L When the input is detected, the computer system stops displaying the specific user interface in conjunction with displaying the settings user interface (eg, the settings user interface replaces the displayed specific user interface).

[0353] Displaying a settings user interface in response to an input corresponding to an alert having information about the fit of the wearable audio output device provides the user with access to additional control options without requiring the user to separately navigate to the settings user interface. Reducing the number of inputs required to access the additional control options enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0354] In some embodiments, the alert corresponding to the information about the fit of the one or more wearable audio output devices includes a prompt to perform an adjustment to the one or more wearable audio output devices 1118. For example, in some embodiments, the alert may suggest that the user recalibrate the audio output device (e.g., via a settings user interface for configuring the fit of the one or more wearable audio output devices, such as Figure 5U 、 Figure 6B -1 to Figure 6B -3). In some embodiments, the alert may suggest that the user reposition one or more audio output devices in the user's ears (e.g., Figure 6H The banner 618 in the figure prompts the user to correct the placement of the earbud 502-2).

[0355] Prompting a user to perform adjustments to a wearable audio output device as part of an alert with fit information about the wearable audio output device provides the user with visual feedback indicating how to address fit issues with the wearable audio output device. Providing improved feedback to the user enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending device battery life by enabling the user to use the device more quickly and efficiently. Furthermore, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending device battery life.

[0356] In some embodiments, method 1000 optionally includes determining that the one or more wearable audio output devices no longer meet device fit criteria 1120 based on determining that the user is in a corresponding context of one or more predefined contexts; and an alert corresponding to the information about the fit of the one or more wearable audio output devices includes a prompt to the user to replace a first set of accessories coupled to the one or more wearable audio output devices with a second set of accessories based on the corresponding context. In some embodiments, the alert includes prompting the user to replace an accessory (e.g., a silicone eartip accessory on a pair of earbuds for forming a seal around the user's ears, such as Figures 6D to 6E shown) from one size (e.g., small) to another size (e.g., larger size to ensure a tight seal during activities, such as Figures 6D to 6E shown).

[0357] Based on the user's current context (e.g., current activity), the user is prompted to change to a specific set of accessories. As part of an alert with fit information about the wearable audio output device, the user is provided with visual feedback indicating how to resolve the fit issue with the wearable audio output device. Providing improved feedback to the user enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently. Furthermore, improving the fit of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending the battery life of the device.

[0358] In some embodiments, the alert corresponding to the information about the fit of the one or more wearable audio output devices includes information about a condition of the wearable audio output device 1122. In some embodiments, the condition is an accessory (e.g., an ear tip) or a clogged accessory (e.g., an ear tip clogged with earwax, which is Figures 6D to 6G , where the physical condition of the EarTip 616-4 is dirty and needs cleaning.

[0359] Providing information about the condition of a wearable audio output device or an accessory to the device (e.g., whether the device or accessory is blocked, worn, or torn) as part of an alert with fit information about the wearable audio output device provides visual feedback to the user indicating how to address the fit or condition of the wearable audio output device. Providing improved feedback to the user enhances device operability and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending device battery life by enabling the user to use the device more quickly and efficiently. Furthermore, improving the fit or condition of the wearable audio output device improves the seal between the wearable audio output device and the user's ear, which enables audio to be played at a lower volume to produce the same effective audio volume as perceived by the user (e.g., as compared to when the wearable audio output device is not properly fitted, in which case hearing some ambient noise may cause the user to increase the audio output volume), further reducing power usage and extending device battery life.

[0360] It should be understood that 11A to 11D The particular order in which the operations are described in the foregoing is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will appreciate various ways to reorder the operations described herein. Additionally, it should be noted that the details of other processes described herein with respect to other methods described herein (e.g., methods 1000, 1200, 1800, 1900, 2000, 2400, and 2600) also apply in a similar manner to the details of the processes described above with respect to 11A to 11D Method 1100 is described. For example, the devices, user interfaces, audio outputs, audio output modes, alerts, adjustments, and accessories described above with reference to method 1100 optionally have one or more of the features of the devices, user interfaces, audio outputs, audio output modes, alerts, adjustments, and accessories described herein with reference to other methods described herein (e.g., methods 1000, 1200, 1800, 1900, 2000, 2400, and 2600). For the sake of brevity, these details are not repeated here.

[0361] 12A to 12E 1 is a flow chart illustrating a method 1200 for controlling audio output using input at a wearable audio output device according to some embodiments. The method 1200 includes an input device (e.g., Figure 3B input device 308) and one or more microphones (e.g., Figure 3B 302) and a wearable audio output device in the physical environment (e.g., Figure 3B100 is performed at the wearable audio output device 301). In some embodiments, the input device is pressure-sensitive (also known as "intensity-sensitive"). For example, the input device responds to a squeeze input (e.g., an input that applies intensity (also known as pressure) to the input device when held between two fingers and squeezed) that meets an intensity threshold, which in some embodiments is greater than a nominal contact detection intensity threshold that would be used for touch input. In some embodiments, the input device is touch-sensitive (e.g., the input device responds to touch input, such as input by a finger or stylus that meets a nominal contact detection intensity threshold). Some operations in method 1200 are optionally combined, and / or the order of some operations is optionally changed.

[0362] As described below, method 1200 provides an improved interface for controlling audio output by changing the audio output mode of an audio output device (such as a wearable device) between audio passthrough and audio cancellation in response to a specific type of gesture. Providing additional control options for controlling audio output (such as changing the audio output mode) without cluttering the user interface with additional displayed controls and reducing the number of inputs required to perform the additional control options enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0363] When ambient sound from the physical environment is being detected by the one or more microphones (1202): when the wearable audio output device is in a first audio output mode (e.g., a pass-through audio output mode, such as Figure 8C As shown, for example, corresponding to Figure 9C The direct mode 924 and referenced herein Figure 3C ), the wearable audio output device provides a first audio output based at least in part on ambient sounds from the physical environment (1204). The first audio output includes one or more pass-through audio components selected to increase the audio pass-through of the ambient sounds from the physical environment (e.g., as described herein with reference to Figure 3C described above).

[0364] The wearable audio output device detects (1206) a first input via an input device (e.g., Figure 8C Input 805).

[0365] In response to detecting the first input, and based on determining that the first input is a first type of gesture (e.g., the input satisfies first gesture criteria, such as a long squeeze gesture criterion requiring a squeeze input to be applied to the input device for at least a threshold amount of time (e.g., above an intensity threshold), the wearable audio output device transitions from a first audio output mode (e.g., a pass-through mode) to a second audio output mode (e.g., an active noise control audio output mode, such as Figure 8D As shown, corresponding to, for example Figure 9C Active Noise Control Mode 922 and herein referenced Figure 3C Description) (1208). This first type of gesture is sometimes referred to herein as a long squeeze gesture, a tap and hold input, or a tap and hold gesture.

[0366] When the wearable audio output device is in a second audio output mode, the wearable audio output device provides a second audio output based at least in part on ambient sounds from a physical environment (1210). The second audio output includes one or more cancellation audio components (e.g., one or more anti-phase signals, such as those provided by the audio output device) selected to increase audio cancellation of the ambient sounds from the physical environment. Figure 3C In some embodiments, the first audio output mode corresponds to a pass-through audio output mode (e.g., a mode in which audio pass-through is active), and the second audio output mode corresponds to a noise cancellation audio output mode (e.g., a mode in which active noise control is active), and the first input, which is a gesture of the first type, transitions the wearable audio output device directly from the pass-through mode to the noise cancellation mode (or vice versa) without first transitioning the wearable audio output device to an intermediate mode (e.g., a bypass mode, also referred to as an "off" mode) in which active noise management is disabled such that neither pass-through nor active noise control is active.

[0367] In some embodiments, the first audio output includes (1212): one or more pass-through audio components at a first ambient sound audio level (e.g., corresponding to ambient sounds from the physical environment that are actively passing through); and one or more cancellation audio components at a first audio cancellation audio level (e.g., noise cancellation audio, also known as "inverted" audio). In some embodiments, when ambient sounds from the physical environment are actively passing through (e.g., the first ambient sound audio level is non-zero), noise cancellation is disabled (e.g., the first audio cancellation audio level is zero). In some embodiments, when noise cancellation is enabled (e.g., the first audio cancellation audio level is non-zero), no ambient sounds from the physical environment are actively passing through (e.g., the first ambient sound audio level is zero). In some embodiments, the second audio output includes: one or more pass-through audio components at a second ambient sound audio level that is different from the first ambient sound audio level; and one or more cancellation audio components at a second audio cancellation audio level that is different from the first audio cancellation audio level. In some embodiments, based on the first audio output being selected or configured to increase audio pass-through of ambient sounds from the physical environment, the first ambient sound audio level of the first audio output is greater than the second ambient sound audio level of the second audio output. In some embodiments, the second audio cancellation audio level is greater than the first audio cancellation audio level in accordance with the second audio output being selected or configured to increase audio cancellation of ambient sounds from the physical environment.

[0368] Using different corresponding levels of passthrough audio components and cancellation audio components in different audio output modes provides the user with flexibility between different levels of audio immersion (via ambient audio cancellation) or audio transparency (via ambient audio passthrough) that can be achieved with a single gesture. Providing this flexibility without cluttering the user interface with additional displayed controls and reducing the number of inputs required to obtain this flexibility enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0369] In some embodiments, the wearable audio output device (1214) is connected to an electronic device (e.g., Figure 1A3 ). In some embodiments, based on determining that the first input is a second type of gesture (e.g., the input satisfies a second gesture criterion, such as a single squeeze gesture criterion requiring that the intensity applied to the input device by the input satisfies an intensity threshold, the intensity applied to the input device by the input decreases below the intensity threshold within a threshold amount of time since the input intensity satisfies the intensity threshold, or alternatively, since the input was initially detected via the input device (e.g., due to the input ceasing contact with the input device)), the wearable audio output device switches to playing a first media audio component based on a first media (e.g., from an electronic device) via the wearable audio output device, the first media being independent of ambient sounds from the physical environment (e.g., music or video). This second type of gesture is sometimes referred to herein as a single squeeze gesture, a click input, or a click gesture. In some embodiments, the second type of gesture switches playback of the corresponding media audio component (e.g., independent of ambient sounds from the physical environment and based on media content from or obtained by the electronic device). For example, as referenced Figures 8E to 8G As shown and described, if an input of a second type of gesture is detected (e.g., Figure 8E 810) is playing a corresponding media audio component, then pause the playback of the corresponding media audio component (e.g., as by Figure 8F A play button 714 is displayed instead of a pause button 715 ( Figure 8E ) indicated); if an input of a second type of gesture is detected (for example, Figure 8F If the corresponding media audio component is paused when a single squeeze gesture input 811 is performed, the playback of the corresponding media audio component is resumed (for example, by Figure 8G The pause button 715 is redisplayed instead of the play button 714 ( Figure 8F ) indicated by ). In some embodiments, the media audio component is combined with one or more pass-through audio components (e.g., where the device is operating in the first mode) or one or more audio cancellation audio components (e.g., where the device is operating in the second mode).

[0370] Switching playback of media (e.g., independent of ambient sound) in response to a particular type of gesture (where the particular type of gesture is distinct from a gesture associated with changing an audio output mode) provides the user with additional control over the audio output that can be implemented with a single, unique gesture. Providing additional control options for controlling the audio output without cluttering the user interface with additional displayed controls and reducing the number of inputs required to execute the additional control options enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0371] In some embodiments, based on determining that the first input is a gesture of the third type (e.g., the input satisfies a third gesture criterion, such as a double squeeze gesture criterion requiring two single squeeze gestures to be performed within a threshold amount of time of each other) (1216): based on determining that the first input is detected while playing a first media audio component based on the first media via the wearable audio output device (e.g., in combination with providing the one or more pass-through audio components (e.g., where the device is operating in the first mode) or the one or more cancellation audio components (e.g., where the device is operating in the second mode), the wearable audio output device stops playing the first media audio component based on the first media; and the wearable audio output device plays a second media audio component based on the second media (e.g., from the electronic device) via the wearable audio output device, the second media being independent of ambient sounds from the physical environment and distinct from the first media (e.g., music or video). In some embodiments, the media audio component is combined with the one or more pass-through audio components (e.g., where the device is operating in the first mode) or the one or more cancellation audio components (e.g., where the device is operating in the second mode). In some embodiments, detecting a third type of gesture while providing a corresponding media audio component (e.g., a first audio track from an electronic device) skips any remaining portion of the corresponding media audio component and instead plays a different corresponding audio component (e.g., a second, next audio track from an electronic device). Figures 8G to 8H As shown and described, in response to the double squeeze input 813, the device 100 skips forward to the next track.

[0372] Changing the media being played (e.g., from a first audio track to a second audio track) in response to a particular type of gesture (where the particular type of gesture is distinct from a gesture associated with changing an audio output mode and a gesture associated with switching playback of media) provides the user with additional control over the audio output that can be implemented with a single, unique gesture. Providing additional control options for controlling the audio output without cluttering the user interface with additional displayed controls and reducing the number of inputs required to execute the additional control options enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0373] In some embodiments, the operation associated with the first type of gesture is (1218) configurable using a settings user interface (e.g., displayed on a display of an electronic device that communicates with the wearable audio output device) and is selected from a first group of operations including switching a corresponding audio output mode of the wearable audio output device; the operation associated with the second type of gesture is configurable using the settings user interface and is selected from a second group of operations including switching playback of a corresponding media audio component; and the operation associated with the third type of gesture is configurable using the settings user interface and is selected from a third group of operations including stopping playback of the first corresponding media audio component in conjunction with playback of the second corresponding media audio component. For example, as referenced Figures 9A to 9B As shown and described, the corresponding operations to be performed in response to the click, double-click, and click and hold gestures can be configured via the settings user interface 900; wherein the set of available operations for the click gesture includes the currently assigned "play / pause" operation type, the set of available operations for the double-click gesture includes the currently assigned "next track" operation type, and the set of available operations for the click and hold gesture includes the currently assigned "noise management" operation type.

[0374] Allowing the user to configure the actions that are performed in response to particular types of gestures (or conversely, allowing the user to configure the types of gestures that result in the performance of particular actions) provides the user with flexibility in customizing his or her interaction with the audio output device (e.g., enabling particular operations (such as those that the user performs more frequently) to be performed via the audio output device using gestures without requiring the user to navigate a complex hierarchy of settings menus on a separate device, and making the interaction more intuitive for the user). Providing flexible and more intuitive user interactions for performing audio output control operations and reducing the number of inputs required to perform those operations enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0375] In some embodiments, the electronic device includes a display (e.g., Figure 1A touch-sensitive display system 112 or Figure 3A 340 in the display) and a second input device (e.g., a touch-sensitive surface such as Figure 1A touch-sensitive display system 112 or Figure 3A ), and detecting (1220) a second input via a second input device (e.g., 7A to 7B In some embodiments, in response to detecting the second input, the electronic device displays a settings user interface on the display, and displays an output mode enable representation for controlling the audio output mode of the wearable audio output device in the settings user interface. In some embodiments, the settings user interface includes a volume enable representation, and in response to activation of the volume enable representation (e.g., by a hard press or a long press on a second input device, where the second input device includes a touch-sensitive surface, such as Figure 7D The electronic device updates the settings user interface (or in other words, replaces the display of the (e.g., first) settings user interface with the display of the second settings user interface) to include an output mode affordance. For example, as described herein with reference to Figures 7D to 7E As shown, the setting user interface 702 includes a volume control 703 and responds to the volume control 703 by input 704 ( Figure 7D ) activates the volume control 703, the device 100 displays an enhanced volume control user interface 705, which includes a noise management control 707-1 ( Figure 7E ).

[0376] Providing control options for the audio output mode in the settings user interface provides users with quick access to additional controls for the audio output without requiring the user to navigate a complex settings menu hierarchy. Reducing the number of inputs required to access control options enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0377] In some embodiments, based on determining that the wearable audio output device is in communication with the electronic device, an output mode enable indication is displayed (1222) in the settings user interface. In some embodiments, based on determining that the wearable audio output device is in communication with the electronic device and that the wearable audio output device is in use (e.g., by playing audio or preparing to play audio), an output mode enable indication is displayed in the settings user interface. For example, Figure 7E The noise management control 707-1 in the device 100 is displayed based on determining that the earbud 502 is paired with the device 100 (and optionally based on determining that the earbud 502 is being used to play audio, such as by Figure 7E In some embodiments, the output mode enable indication is not displayed in the settings user interface when the wearable audio output device is not communicating with the electronic device (e.g., if the earbuds 502 are not connected to the device 100, the noise management control 707-1 is not displayed in the enhanced volume control user interface 705).

[0378] Based on whether the audio output device and the device on which the setup user interface is displayed are communicating with each other, control options for the audio output mode (e.g., audio output mode controls) are conditionally displayed in the setup user interface, such that the control options are displayed only when the two devices are communicating, providing the user with quick access to the control options when relevant (e.g., when the two devices are in communication), but not providing them when those control options are invalid (e.g., when the devices are not in communication). Providing additional control options without unnecessarily cluttering the user interface with additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0379] In some embodiments, the output mode enable representation includes (1224) a representation of a first corresponding audio output mode that is the current audio output mode of the wearable audio output device, and does not include a representation of any other audio output mode of the wearable audio output device. For example, if the audio output device is operating in a first mode, the output mode enable representation includes a representation of the first audio output mode (e.g., does not include a representation of the second audio output mode or any other output mode), or if the audio output device is operating in a second mode, includes a representation of the second audio output mode (e.g., does not include a representation of the first audio output mode or any other output mode). For example, as described herein with reference to Figure 7E As described, the noise management control 707-1 includes a pass-through icon 711 representing the pass-through audio output mode to indicate that the earbud 502 is in the pass-through audio output mode. Alternatively, if the earbud 502 is in the active noise control audio output mode, the noise management control 707-1 includes an active noise control icon 709, and if the earbud 502 is in the bypass audio output mode, it includes a bypass icon 710.

[0380] Displaying an indication of the current audio output mode of the audio output device in the audio output mode control provides visual feedback to the user, clearly indicating which audio output mode the audio output device is currently operating in, without displaying other audio output mode options that may distract the user or mislead the user. Providing improved feedback to the user without unnecessarily cluttering the user interface with additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0381] In some embodiments, the electronic device detects (1226) a third input corresponding to an output mode enable representation via a second input device; and, in response to detecting the third input, the electronic device displays corresponding representations of a plurality of audio output modes of the wearable audio output device (e.g., including a representation of a first corresponding audio output mode that is the current audio output mode). For example, as described herein with reference to Figures 7E to 7FAs described, in response to detecting input 708 at noise management control 707-1, device 100 displays expanded noise management control 707-2, which includes a pass-through icon 711 indicating pass-through mode, a bypass icon 710 indicating bypass mode, and an active noise control icon 709 indicating active noise control mode. In some embodiments, (e.g., when displaying representations of the multiple audio output modes) a selection indicator (e.g., a selection indicator) is displayed on or above the representation of the first corresponding audio output mode (or the representation of the first corresponding audio output mode is otherwise visually distinguished from other representations of available audio output modes). Figure 7F ) to indicate that the first corresponding audio output mode is the current audio output mode of the wearable audio output device. In some embodiments, the electronic device detects a fourth input corresponding to a representation of a second corresponding audio output mode different from the first corresponding audio output mode via a second input device; and in response to detecting the fourth input: the wearable audio output device changes from the first corresponding audio output mode to the second corresponding audio output mode (e.g., making the second corresponding audio output mode the current audio output mode). For example, as described herein with reference to Figures 7G to 7H As described, in response to input 712 at the active noise control icon 709, the audio output mode of the earbud 502 changes from the pass-through mode to the active noise control mode. In some embodiments, in response to detecting the fourth input, the electronic device stops displaying representations of audio output modes other than the second corresponding audio output mode (e.g., as described herein with reference to FIG. Figure 7I As described with respect to folding the expanded noise management control 707-2). In some embodiments (e.g., where the electronic device continues to display representations of other audio output modes), in response to detecting the fourth input, a selection indicator is displayed on or above the representation of the second corresponding audio output mode instead of the first corresponding audio output mode (or the representation of the second corresponding audio output mode is otherwise visually distinguished and the representation of the first corresponding audio output mode ceases to be visually distinguished from other representations of available audio output modes) to indicate that the second corresponding audio output mode is now the current audio output mode of the wearable audio output device. For example, as described herein with reference to Figures 7G to 7H In response to selection of active noise control icon 709 via input 712 ( Figure 7G ), the selection indicator 713 stops being displayed above the pass-through icon 711 and is instead displayed above the active noise control icon 709 ( Figure 7H ).

[0382] Displaying additional options for audio output modes in response to selection of an audio output mode control, and then transitioning the audio output mode of the audio output device in response to selection of a different audio output mode option, provides the user with access to the additional audio output modes upon request without prematurely displaying other audio output mode options that may distract or mislead the user. Providing additional control options without unnecessarily cluttering the user interface with additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0383] In some embodiments, the first corresponding audio output mode (1228) is different from a third audio output mode in which the wearable audio output device provides audio output independent of ambient sounds from the physical environment (e.g., an audio output mode in which the wearable audio output device provides neither a pass-through audio component nor a canceled audio component, such as provided by Figure 7G ), and the second corresponding audio output mode is different from the third audio output mode.

[0384] Allowing a user to change the audio output mode of an audio output device between two different audio output modes (e.g., between an audio passthrough mode and an audio cancellation mode or vice versa), neither of which is independent of ambient sound from the physical environment, provides the user with flexibility between different levels of audio immersion or audio transparency that can be achieved with a single input, rather than requiring the user to separately turn audio immersion on or off and audio transparency on or off. Providing this flexibility while reducing the number of inputs required to achieve this flexibility enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0385] In some embodiments, detecting the first input includes (1230) detecting an intensity increase that satisfies an activation intensity threshold, and the wearable audio output device: provides an activation audio output in response to detecting an intensity increase that satisfies the activation intensity threshold (e.g., an increase in intensity to or above the activation intensity threshold); detects an intensity decrease that satisfies a release intensity threshold; and provides a release audio output in response to detecting a decrease in intensity that satisfies the release intensity threshold (e.g., a decrease in intensity to or below the release intensity threshold). In some embodiments, the activation intensity threshold is the same as the release intensity threshold. In some embodiments, the activation intensity threshold is different from the release intensity threshold. For example, setting the activation intensity threshold above the release intensity threshold provides hysteresis so that unintentional fluctuations in input intensity (e.g., due to instability of a user's finger when applying pressure to the input device) does not cause the wearable audio output device to detect a release of the input. In some embodiments, the activation audio output is the same as the release audio output. In some embodiments, the activation audio output is different from the release audio output (e.g., so that a user can distinguish between detected activation and detected release). References herein Figure 8A Inputs including an increase to or above an intensity threshold (e.g., a "press click") and a decrease to the same (or in some embodiments, a different) intensity threshold or below the same intensity threshold (e.g., a "release click") and corresponding audio outputs are described. In some embodiments, in response to detecting an input that does not include an increase in intensity above an activation intensity threshold, the wearable audio output device does not provide (e.g., abstains from providing) an activation audio output based on a determination that the input does not include an increase in intensity above the activation intensity threshold. In some embodiments, the device does not provide (e.g., abstains from providing) a release audio output until a decrease in intensity below a release intensity threshold is detected.

[0386] Providing audio output when the intensity of the input increases to meet an activation intensity threshold and when the intensity of the input decreases to meet a release intensity threshold provides audio feedback to the user that these thresholds have been met and any actions associated with these thresholds will be performed in response to the input. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0387] In some embodiments, the wearable audio output device includes (1232) a first wearable audio output component having a first corresponding input device and a second wearable audio output component having a second corresponding input device (e.g., first and second earpieces having corresponding pressure-sensitive input devices, such as earbuds 502-1 and 502-2 with corresponding handles, as described herein with reference to Figure 8A In some embodiments, detecting the corresponding input via the input device includes detecting the input of the first corresponding input device via the first wearable audio output component, or detecting the input of the second corresponding input device via the second wearable audio output component; and providing the corresponding audio output via the wearable audio output device includes...

Claims

1. A method comprising: At a wearable audio output device comprising a wearable component for use with a single ear of a user, the wearable component including an input device and one or more microphones and being in a physical environment: While ambient sound from the physical environment is being detected by the one or more microphones: providing a first audio output based at least in part on the ambient sounds from the physical environment when the wearable audio output device is in a first audio output mode, wherein the first audio output includes one or more pass-through audio components selected to increase audio pass-through of the ambient sounds from the physical environment; detecting a first input via the input device while the wearable audio output device is in the first audio output mode; in response to detecting the first input while the wearable audio output device is in the first audio output mode, transitioning the wearable audio output device from the first audio output mode to a second audio output mode based on determining that the first input is a first type of gesture; providing a second audio output based at least in part on the ambient sounds from the physical environment when the wearable audio output device is in the second audio output mode, wherein the second audio output includes one or more cancellation audio components selected to increase audio cancellation of the ambient sounds from the physical environment; detecting, while the wearable audio output device is in the second audio output mode, a second instance of the first input via the input device; in response to detecting the second instance of the first input while the wearable audio output device is in the second audio output mode, transitioning the wearable audio output device from the second audio output mode to a third audio output mode different from the second audio output mode based on determining that the second instance of the first input is the first type of gesture; as well as Based on determining that the first input is a second type of gesture different from the first type of gesture, switching playback of a first media audio output component based on a first media via the wearable audio output device without changing a current audio output mode of the wearable audio output device, the first media being independent of the ambient sound from the physical environment.

2. The method according to claim 1, wherein: The first audio output comprises: the one or more pass-through audio components at a first ambient sound audio level; and the one or more cancelled audio components at a first audio cancelled audio level; and The second audio output includes: the one or more pass-through audio components at a second ambient sound audio level different from the first ambient sound audio level; and The one or more audio components are eliminated at a second audio level different from the first audio level. 3 . The method of claim 1 , wherein the third audio output mode is different from the second audio output mode and different from the first audio output mode.

4. The method according to claim 3, comprising: According to determining that the first input is a gesture of the third type: stopping playing the first media audio component based on the first media based on the first media based on determining that the first input is detected while the first media audio component based on the first media is played via the wearable audio output device; as well as A second media audio component based on a second media is played via the wearable audio output device, the second media being independent of the ambient sounds from the physical environment and different from the first media.

5. The method according to claim 4, wherein: the operation associated with the first type of gesture being configurable using a settings user interface and selected from a first group of operations including transitioning a corresponding audio output mode of the wearable audio output device; The operation associated with the second type of gesture is configurable using the settings user interface and is selected from a second group of operations including switching playback of a corresponding media audio component; and The operation associated with the third type of gesture is configurable using the settings user interface and is selected from a third group of operations including stopping playback of a first respective media audio component in conjunction with playback of a second respective media audio component.

6. The method according to any one of claims 1 to 5, comprising: detecting, at an electronic device having a display and a second input device, a second input via the second input device; In response to detecting the second input, displaying a settings user interface on the display of the electronic device; as well as An output mode affordance for controlling an audio output mode of the wearable audio output device is displayed in the settings user interface.

7. The method of claim 6, wherein the output mode affordance is displayed in the settings user interface based on determining that the wearable audio output device is in communication with the electronic device.

8. The method of claim 6, wherein the output mode affordance representation includes a representation of a first corresponding audio output mode that is a current audio output mode of the wearable audio output device, and does not include a representation of any other audio output mode of the wearable audio output device.

9. The method according to claim 8, comprising: detecting a third input corresponding to the output mode affordance via the second input device; in response to detecting the third input, displaying respective representations of a plurality of audio output modes of the wearable audio output device; detecting a fourth input via the second input device corresponding to a representation of a second corresponding audio output mode different from the first corresponding audio output mode; as well as In response to detecting the fourth input: The wearable audio output device is transitioned from the first corresponding audio output mode to the second corresponding audio output mode.

10. The method of any one of claims 1 to 5, wherein the third audio output mode is different from the second audio output mode and the first audio output mode.

11. The method of any one of claims 1 to 5, wherein detecting the first input comprises detecting an increase in intensity of the first input that satisfies an activation intensity threshold, and the method comprises: providing an activation audio output in response to detecting the increase in intensity of the first input satisfying the activation intensity threshold; detecting a decrease in intensity of the first input that satisfies a release intensity threshold; as well as In response to detecting the decrease in intensity of the first input that satisfies the release intensity threshold, a release audio output is provided.

12. The method according to any one of claims 1 to 5, wherein: The wearable audio output device includes a first wearable audio output component having a first corresponding input device and a second wearable audio output component having a second corresponding input device; Detecting the corresponding input via the input device comprises detecting the corresponding input via the first corresponding input device of the first wearable audio output component, or detecting the corresponding input via the second corresponding input device of the second wearable audio output component; and Providing respective audio outputs via the wearable audio output device includes providing the respective audio outputs via the first wearable audio output component and the second wearable audio output component.

13. The method according to any one of claims 1 to 5, comprising: detecting speech by a user of the wearable audio output device while the wearable audio output device is in the second audio output mode; as well as In response to detecting the voice of the user, transitioning the wearable audio output device from the second audio output mode to the first audio output mode.

14. The method according to any one of claims 1 to 5, comprising: When the wearable audio output device is in the second audio output mode: Based on determining that the ambient sound from the physical environment includes a name of a user of the wearable audio output device, the wearable audio output device is transitioned from the second audio output mode to the first audio output mode.

15. The method according to any one of claims 1 to 5, comprising: Based on determining that the first input is the first type of gesture, providing an audio output associated with transitioning an audio output mode of the wearable audio output device.

16. The method of any one of claims 1 to 5, wherein the wearable audio output device is configured to transition to a set of audio output modes in response to input of the first type of gesture is configurable using a settings user interface.

17. The method of claim 16, wherein the set of audio output modes comprises a plurality of audio output modes, and the method comprises: while displaying the settings user interface on a display of the electronic device, detecting an input via a second input device of the electronic device for reordering two or more of the set of audio output modes to form a modified set of audio output modes; after detecting the input for reordering the two or more of the set of audio output modes to form the modified set of audio output modes, detecting a subsequent input via the input device of the wearable audio output device that is a gesture of the first type; as well as In response to detecting the subsequent input being the first type of gesture, transitioning the wearable audio output device from a current audio output mode to a next audio output mode in the modified set of audio output modes.

18. The method according to any one of claims 1 to 5, wherein: The wearable audio output device includes a first wearable audio output component at a first position relative to a first ear of a user, and a second wearable audio output component at a second position relative to a second ear of the user; the second audio output comprises a respective media audio component based on respective media, the respective media being independent of the ambient sounds from the physical environment; and The method comprises: When the wearable audio output device is in the second audio output mode: Based on determining that the first wearable audio output component is removed from the first position relative to the first ear of the user: The wearable audio output device is transitioned from the second audio output mode to the first audio output mode, and while the wearable audio output device is in the first audio output mode, the first audio output is provided based at least in part on the ambient sound from the physical environment.

19. The method of any one of claims 1 to 5, wherein the wearable audio output device comprises one or more in-ear headphones, earbuds, or over-ear headphones.

20. The method of any one of claims 1 to 5, wherein the first type of gesture comprises: a squeeze input that occurs when a handle of the wearable component is squeezed between two fingers of a user; Or a single click and hold input on the input device of the wearable component.

21. A wearable audio output device in a physical environment, the wearable audio output device comprising: Input devices; one or more microphones; one or more processors; and A memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs comprising instructions for performing the following operations: While ambient sound from the physical environment is being detected by the one or more microphones: providing a first audio output based at least in part on the ambient sounds from the physical environment when the wearable audio output device is in a first audio output mode, wherein the first audio output includes one or more pass-through audio components selected to increase audio pass-through of the ambient sounds from the physical environment; detecting a first input via the input device while the wearable audio output device is in the first audio output mode; in response to detecting the first input while the wearable audio output device is in the first audio output mode, transitioning the wearable audio output device from the first audio output mode to a second audio output mode based on determining that the first input is a first type of gesture; providing a second audio output based at least in part on the ambient sounds from the physical environment when the wearable audio output device is in the second audio output mode, wherein the second audio output includes one or more cancellation audio components selected to increase audio cancellation of the ambient sounds from the physical environment; detecting, while the wearable audio output device is in the second audio output mode, a second instance of the first input via the input device; in response to detecting the second instance of the first input while the wearable audio output device is in the second audio output mode, transitioning the wearable audio output device from the second audio output mode to a third audio output mode different from the second audio output mode based on determining that the second instance of the first input is the first type of gesture; as well as Based on determining that the first input is a second type of gesture different from the first type of gesture, switching playback of a first media audio output component based on a first media via the wearable audio output device without changing a current audio output mode of the wearable audio output device, the first media being independent of the ambient sound from the physical environment.

22. A wearable audio output device according to claim 21, wherein the one or more programs include instructions for performing the method according to any one of claims 2 to 20.

23. A computer program comprising instructions which, when executed by a wearable audio output device comprising an input device and one or more microphones and in a physical environment, cause the wearable audio output device to: While ambient sound from the physical environment is being detected by the one or more microphones: providing a first audio output based at least in part on the ambient sounds from the physical environment when the wearable audio output device is in a first audio output mode, wherein the first audio output includes one or more pass-through audio components selected to increase audio pass-through of the ambient sounds from the physical environment; detecting a first input via the input device while the wearable audio output device is in the first audio output mode; in response to detecting the first input while the wearable audio output device is in the first audio output mode, transitioning the wearable audio output device from the first audio output mode to a second audio output mode based on determining that the first input is a first type of gesture; providing a second audio output based at least in part on the ambient sounds from the physical environment when the wearable audio output device is in the second audio output mode, wherein the second audio output includes one or more cancellation audio components selected to increase audio cancellation of the ambient sounds from the physical environment; detecting, while the wearable audio output device is in the second audio output mode, a second instance of the first input via the input device; in response to detecting the second instance of the first input while the wearable audio output device is in the second audio output mode, transitioning the wearable audio output device from the second audio output mode to a third audio output mode different from the second audio output mode based on determining that the second instance of the first input is the first type of gesture; as well as Based on determining that the first input is a second type of gesture different from the first type of gesture, switching playback of a first media audio output component based on a first media via the wearable audio output device without changing a current audio output mode of the wearable audio output device, the first media being independent of the ambient sound from the physical environment.

24. A computer program according to claim 23, wherein the computer program comprises instructions which, when executed by the wearable audio output device, cause the wearable audio output device to perform the method according to any one of claims 2 to 20.

Images