Apparatus, method, and user interface for providing audio notifications

Abstract

The invention relates to a device, a method and a user interface for providing audio notifications. An electronic device is in communication with one or more wearable audio output devices. The electronic device detects an occurrence of one or more first events when a user wears the one or more wearable audio output devices. In response, the electronic device outputs audio content corresponding to the one or more first events via the one or more wearable audio output devices. After outputting audio content corresponding to the one or more first events, the electronic device detects a movement of the one or more wearable audio output devices, and in response to detecting the movement and in accordance with a determination that a first movement of the one or more wearable audio output devices satisfies a first movement criterion, outputs audio content corresponding to the one or more first events. Additional audio content corresponding to the one or more events is output via the one or more wearable audio output devices.

Description

[0001] Division Statement

[0002] This application is a divisional application of a Chinese invention patent application with a filing date of September 20, 2019, an invention name of “Device, method and user interface for providing audio notifications” and an application number of 201980056882.4. Technical Field

[0003] The general disclosure relates to electronic devices having audio output devices, such as wearable audio output devices, including but not limited to electronic devices that provide audio notifications via the audio output devices. Background Art

[0004] Audio notifications are widely used to convey information to users verbally. But conventional methods of providing audio notifications are cumbersome, inefficient, and limited. In some cases, conventional methods of providing audio notifications require multiple inputs to generate an audio notification or respond to an audio notification. In some cases, conventional methods of providing audio notifications provide too little information or too much information at a time. In some cases, conventional methods of providing audio notifications provide audio notifications in an intrusive and distracting manner. In addition, conventional methods take longer than necessary and require more user interaction, thereby wasting energy. This latter consideration is particularly important in battery-powered devices. Summary of the invention

[0005] Therefore, there is a need for electronic devices with improved methods and interfaces for providing audio notifications. Such methods and interfaces optionally supplement or replace conventional methods of providing audio notifications. Such methods and interfaces reduce the amount, degree and / or nature of input from the user and produce a more efficient human-computer interface. For battery-powered devices, such methods and interfaces can save power and increase the time between battery charges.

[0006] The above-mentioned defects and other problems associated with providing audio notifications are reduced or eliminated by the disclosed device. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a laptop, a tablet computer or a handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the device has a touchpad (and / or communicates with it). In some embodiments, the device has a touch-sensitive display (also referred to as a "touch screen" or "touch screen display") (and / or communicates with it). In some embodiments, the device has a graphical user interface (GUI), one or more processors, a memory and one or more modules, a program or instruction set stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI in part by touching a stylus and / or a finger and by gestures on a touch-sensitive surface. In some embodiments, these functions optionally include playing games, image editing, drawing, demonstrating, word processing, making electronic spreadsheets, making calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, note-taking and / or digital video playback. Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0007] According to some embodiments, a method is performed at an electronic device in communication with one or more wearable audio output devices and one or more sensors for detecting placement of the one or more wearable audio output devices. The method includes, when the one or more wearable audio output devices are not in corresponding positions relative to a user's ears, detecting the occurrence of multiple events without outputting notifications corresponding to the events via the one or more audio output devices. The method includes, after detecting the occurrence of the multiple events, detecting via the one or more sensors whether the one or more wearable audio output devices are placed in the corresponding positions relative to the user's ears. The method includes, in response to detecting via the one or more sensors that the one or more wearable audio output devices have been placed in the corresponding positions relative to the user's ears, outputting information via the one or more wearable audio output devices, the information including one or more audio notifications corresponding to one or more of the multiple events.

[0008] According to some embodiments, a method is performed at an electronic device having one or more audio output devices. The method includes detecting the occurrence of an event; and in response to detecting the occurrence of the event, outputting an audio notification via the one or more audio output devices, the audio notification including audio content representing the event. Based on determining that the event is associated with a first feature in a first set of features, the audio notification includes the audio content representing the event and has a first audio output attribute selected based on the first feature. Based on determining that the event is associated with a second feature in the first set of features that is different from the first feature, the audio notification includes the audio content representing the event and has a second audio output attribute selected based on the second feature that is different from the first output attribute.

[0009] According to some embodiments, a method is performed at an electronic device in communication with one or more wearable audio output devices and with one or more sensors for detecting placement of the one or more wearable audio output devices. The method includes detecting an occurrence of a first event; and after detecting the occurrence of the first event, outputting one or more audio notifications corresponding to the first event via the one or more wearable audio output devices. The method includes, after starting to output the one or more audio notifications, detecting an input directed to the one or more wearable audio output devices. The method includes, in response to detecting the input directed to the one or more wearable audio output devices: performing a first operation based on determining that the input was detected within a predefined time period, the predefined time period being related to the one or more audio notifications corresponding to the first event, wherein the first operation is associated with the one or more audio notifications corresponding to the first event; and performing a second operation based on determining that the input was detected after the predefined time period has passed, the predefined time period being related to the one or more audio notifications corresponding to the first event, wherein the second operation is not associated with the one or more audio notifications corresponding to the first event.

[0010] According to some embodiments, a method is performed at an electronic device that communicates with one or more wearable audio output devices. The method includes detecting an occurrence of a first event; and after detecting the occurrence of the first event, outputting one or more audio notifications corresponding to the first event via the one or more wearable audio output devices. The method includes, after outputting the one or more audio notifications corresponding to the first event, outputting a corresponding audio output separate from the one or more audio notifications indicating that the electronic device is ready to receive a response to the first event. The method includes, while outputting the corresponding audio output, detecting an input corresponding to a response to the first event, and generating a response to the first event based on the input.

[0011] According to some embodiments, a method is performed at an electronic device that communicates with one or more wearable audio output devices. The method includes detecting an occurrence of a first event while a user is wearing the one or more wearable audio output devices. The method includes, in response to detecting the occurrence of the first event, outputting one or more audio notifications corresponding to the first event via the one or more wearable audio output devices, including: delaying output of the one or more audio notifications corresponding to the first event until the session has ended based on determining that the user of the electronic device is currently participating in a session; and outputting the one or more audio notifications corresponding to the first event without delaying the output based on determining that the user of the electronic device is not currently participating in a session.

[0012] According to some embodiments, the electronic device includes one or more audio output devices (e.g., one or more wearable audio output devices), one or more sensors for detecting the placement of the audio output device, optionally a display and / or a touch-sensitive surface, optionally one or more accelerometers and / or gesture sensors, optionally one or more additional input devices (e.g., a rotatable input mechanism), one or more processors, and a memory storing one or more programs (and / or communicating with these devices); the one or more programs are configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the execution of the operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium has instructions stored therein, which, when executed by an electronic device including one or more audio output devices, optionally one or more sensors for detecting the placement of the audio output device, optionally a display and / or a touch-sensitive surface, optionally one or more accelerometers and / or gesture sensors, and optionally one or more additional input devices (and / or communicating with these devices), causes the device to perform or cause the execution of the operations of any of the methods described herein. According to some embodiments, a graphical user interface on a device comprising one or more audio output devices, optionally one or more sensors for detecting the placement of the audio output devices, optionally a display and / or a touch-sensitive surface, optionally one or more accelerometers and / or gesture sensors, and optionally one or more additional input devices, a memory, and one or more processors for executing one or more programs stored in the memory (and / or communicating with these devices) includes one or more of the elements displayed in any of the methods described herein, and the elements are updated in response to input, as described in any of the methods described herein. According to some embodiments, an electronic device comprises one or more audio output devices, optionally one or more sensors for detecting the placement of the audio output devices, optionally a display and / or a touch-sensitive surface, optionally one or more accelerometers and / or gesture sensors, and optionally one or more additional input devices, and means for performing or causing the operations of any of the methods described herein to be performed (and / or communicating with these devices). According to some embodiments, an information processing device used in an electronic device that includes one or more audio output devices, optionally one or more sensors for detecting the placement of the audio output device, optionally a display and / or a touch-sensitive surface, optionally one or more accelerometers and / or gesture sensors, and optionally one or more additional input devices (and / or communicates with these devices) includes a device for performing or causing the performance of the operations of any of the methods described herein.

[0013] Thus, an electronic device having an improved method and interface for providing audio notifications is provided, the electronic device having one or more audio output devices, optionally one or more sensors for detecting the placement of the audio output device, optionally a display and / or a touch-sensitive surface, optionally one or more accelerometers and / or gesture sensors, and optionally one or more additional input devices (and / or communicating with these devices), thereby improving the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can supplement or replace conventional methods for providing audio notifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0015] Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display according to some embodiments.

[0016] Figure 1B is a block diagram illustrating example components for event processing according to some embodiments.

[0017] Figure 2 A portable multifunction device with a touch screen according to some embodiments is shown.

[0018] Figure 3A is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.

[0019] Figure 3B is a block diagram of an exemplary wearable audio output device according to some embodiments.

[0020] Figure 4A An exemplary user interface for an application menu on a portable multifunction device according to some embodiments is shown.

[0021] Figure 4B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0022] FIG. 4C to FIG. 4E Examples of dynamic intensity thresholds according to some embodiments are shown.

[0023] FIG. 5A to FIG. 5AD Exemplary audio notifications and exemplary user interfaces and interactions with audio notifications are shown according to some embodiments.

[0024] FIG. 6A to FIG. 6Eis a flowchart of a process for providing audio notifications to an audio output device worn by a user according to some embodiments.

[0025] 7A to 7C is a flowchart of a process for providing audio context to convey additional information about an event according to some embodiments.

[0026] FIG. 8A to FIG. 8D is a flowchart of a process for performing operations based on the timing of user input related to an audio notification according to some embodiments.

[0027] 9A to 9C is a flowchart of a process for prompting a user and receiving a response to an audio notification according to some embodiments.

[0028] FIG. 10A to FIG. 10B is a flowchart of a process for conditionally outputting audio notifications based on user activity and context according to some embodiments. DETAILED DESCRIPTION

[0029] As described above, audio notifications can be used to convey information to a user by providing the information verbally and enabling the user to conveniently listen to the information without requiring the user to view the information. Conventional methods for providing audio notifications are generally limited in function or effect. In some cases, conventional methods for providing audio notifications require multiple inputs to generate or respond to audio notifications. In some cases, conventional methods for providing audio notifications provide audio notifications in an intrusive and distracting manner. The devices, methods, and user interfaces described herein improve user interface interactions with audio notifications in a variety of ways. For example, the embodiments disclosed herein provide more efficient ways for electronic devices to provide audio notifications and receive responses to audio notifications. In addition, the embodiments herein provide improved audio notifications that more effectively convey information to users.

[0030] under, Figure 1A to Figure 1B , Figure 2 as well as FIG. 3A to FIG. 3B A description of an exemplary apparatus is provided. FIG. 4A to FIG. 4B An exemplary user interface of an exemplary device on which embodiments disclosed herein are implemented is shown. FIG. 5A to FIG. 5AD An exemplary audio notification and an exemplary user interface and interaction with the audio notification are shown. FIG. 6A to FIG. 6E A flow chart of a method for providing an audio notification to an audio output device worn by a user is shown. 7A to 7C A flow chart of a method of providing audio context to convey additional information about an event is shown. FIG. 8A to FIG. 8D A flow chart of a method for performing operations based on the timing of user input related to an audio notification is shown. 9A to 9CA flow chart of a method of prompting a user and receiving a response to an audio notification is shown. FIG. 10A to FIG. 10B A flow chart of a method for conditionally outputting an audio notification based on user activity and context is shown. FIG. 5A to FIG. 5AD The user interface and interaction in FIG. 6A to FIG. 6E , 7A to 7C , FIG. 8A to FIG. 8D , 9A to 9C and FIG. 10A to FIG. 10B process.

[0031] Exemplary Devices

[0032] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Many specific details are shown in the following detailed description in order to provide a full understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments can be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail, so as not to unnecessarily obscure various aspects of the embodiments.

[0033] It will also be understood that, although in some cases, the terms "first", "second", etc. are used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first contact can be named as the second contact, and similarly, the second contact can be named as the first contact without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact unless the context clearly indicates otherwise.

[0034] The terms used in the description of various described embodiments herein are only for the purpose of describing specific embodiments, and are not intended to be limited. As used in the description of various described embodiments and in the appended claims, the singular forms "one" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more items in the associated listed items. It will also be understood that the term "includes", "including", "comprises" and / or "comprising" when used in this specification specifies the presence of stated features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their grouping.

[0035] As used herein, the term "if" is optionally interpreted to mean "when" followed by "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [a stated condition or event] is detected" are optionally interpreted to mean "upon determining that" or "in response to determining that" or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0036] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or a music player function. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple iPod from Apple Inc. (Cupertino, California). iPod and Device. Other portable electronic devices are optionally used, such as laptop computers or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or trackpads). It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or trackpads).

[0037] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse and / or joystick.

[0038] The device typically supports a variety of applications, such as one or more of the following: a gaming application, a note-taking application, a drawing application, a presentation application, a word processing application, a spreadsheet application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0039] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed by the device are optionally adjusted and / or varied for different applications, and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the device (such as a touch-sensitive surface) optionally supports a variety of applications with a user interface that is intuitive and clear to the user.

[0040] Attention is now turned to embodiments of portable devices having touch-sensitive displays. Figure 1A 1 is a block diagram showing a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes called a "touch screen" for convenience, and is sometimes simply referred to as a touch-sensitive display. The device 100 includes a memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, an RF circuit 108, an audio circuit 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input, or control devices 116, and an external port 124. The device 100 optionally includes one or more optical sensors 164 (e.g., as part of one or more cameras). The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 163 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0041] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, which will be detected by a user using the user's sense of touch. For example, in the case where a device or a component of the device is in contact with a user's touch-sensitive surface (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" to a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click", even when the physical actuation button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the touch-sensitive surface does not change, the movement of the touch-sensitive surface may be optionally interpreted or sensed by the user as the "roughness" of the touch-sensitive surface. Although such interpretation of touch by the user will be limited by the user's individualized sensory perception, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click", "release click", "roughness"), unless otherwise stated, the generated tactile output corresponds to a physical displacement of the device or its components that will generate the sensory perception of a typical (or ordinary) user. Providing tactile feedback to the user using tactile output enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating the device / interacting with the device), which additionally reduces power usage and improves the battery life of the device by enabling the user to use the device more quickly and more efficiently.

[0042] It should be understood that device 100 is merely one example of a portable multifunction device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof (including one or more signal processing circuits and / or application specific integrated circuits).

[0043] The memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. The access of other components of the device 100 (such as the CPU 120 and the peripheral device interface 118) to the memory 102 is optionally controlled by the memory controller 122.

[0044] Peripheral interface 118 may be used to couple the device's input and output peripherals to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0045] In some embodiments, peripherals interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0046] RF (radio frequency) circuit 108 receives and sends RF signals, also referred to as electromagnetic signals. RF circuit 108 converts electrical signals into / from electromagnetic signals and communicates with a communication network and other communication devices via electromagnetic signals. RF circuit 108 optionally includes well-known circuits for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, and the like. RF circuit 108 optionally communicates with networks and other devices via wireless communications, such as the Internet (also referred to as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). The wireless communication optionally uses any of a variety of communication standards, protocols and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution-Data-only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11d), IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11d, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilizing Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including communication protocols that have not been developed on the date of filing of this document.

[0047] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., Figure 2 The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset having both output (e.g., a single or dual-ear headset) and input (e.g., a microphone).

[0048] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch-sensitive display system 112 and other input or control devices 116, to a peripheral device interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from / to other input or control devices 116. Other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, the one or more input controllers 160 are optionally coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. One or more buttons (e.g., Figure 2 208) optionally includes an up / down button for volume control of the speaker 111 and / or the microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2 206).

[0049] The touch-sensitive display system 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch-sensitive display system 112 and / or sends electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some visual outputs or all of the visual outputs correspond to user interface objects. As used herein, the term "indicator" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object configured to respond to input directed to a graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0050] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from a user based on tactile and / or haptic contact. The touch-sensitive display system 112 and display controller 156 (together with any associated modules and / or instruction sets in memory 102) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In some embodiments, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0051] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112 to detect contact and any movement or interruption thereof. In some embodiments, projected mutual capacitance sensing technology is used, such as the . iPod and Technology found in.

[0052] The touch-sensitive display system 112 optionally has a video resolution of more than 100 dpi. In some embodiments, the touch screen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi or greater). The user optionally uses any suitable object or attachment such as a stylus, finger, etc. to contact the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as accurate as stylus-based input due to the larger contact area of ​​the finger on the touch screen. In some embodiments, the device converts rough finger-based input into precise pointer / cursor positions or commands for performing the actions desired by the user.

[0053] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface that is separate from the touch-sensitive display system 112, or an extension of the touch-sensitive surface formed by the touch screen.

[0054] The device 100 also includes a power system 162 for powering the various components. The power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0055] Device 100 optionally also includes one or more optical sensors 164 (eg, as part of one or more cameras). Figure 1A An optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. One or more optical sensors 164 optionally include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. One or more optical sensors 164 receive light projected through one or more lenses from the environment and convert the light into data representing an image. In conjunction with an imaging module 143 (also called a camera module), one or more optical sensors 164 optionally capture static images and / or videos. In some embodiments, an optical sensor is located on the rear portion of the device 100 opposite to the touch-sensitive display system 112 on the front of the device, so that the touch screen can be used as a viewfinder for static image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to obtain an image of the user (e.g., for self-portraits, for video conferencing when the user is watching other video conference participants on the touch screen, etc.).

[0056] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1A A contact force sensor coupled to a force sensor controller 159 in the I / O subsystem 106 is shown. One or more contact force sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of contact on a touch-sensitive surface). One or more contact force sensors 165 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the rear of the device 100 opposite to the touch-sensitive display system 112 located on the front of the device 100.

[0057] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0058] Device 100 optionally also includes one or more tactile output generators 163. Figure 1AA tactile output generator coupled to a tactile feedback controller 161 in the I / O subsystem 106 is shown. In some embodiments, one or more tactile output generators 163 include one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile outputs on the device). One or more tactile output generators 163 receive tactile feedback generation instructions from the tactile feedback module 133 and generate tactile outputs on the device 100 that can be felt by a user of the device 100. In some embodiments, at least one tactile output generator is arranged in juxtaposition or proximity to a touch-sensitive surface (e.g., a touch-sensitive display system 112), and optionally generates tactile outputs by moving the touch-sensitive surface vertically (e.g., inward / outward of the surface of the device 100) or laterally (e.g., backward and forward in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on a rear portion of the device 100 opposite the touch-sensitive display system 112 located on the front of the device 100.

[0059] The device 100 optionally also includes one or more accelerometers 167, gyroscopes 168, and / or magnetometers 169 (eg, as part of an inertial measurement unit (IMU)) for obtaining information about the position (eg, attitude) of the device. Figure 1A Sensors 167, 168, and 169 are shown coupled to peripheral device interface 118. Alternatively, sensors 167, 168, and 169 are optionally coupled to input controller 160 in I / O subsystem 106. In some embodiments, information is displayed in a portrait view or a landscape view on the touch screen display based on analysis of data received from the one or more accelerometers. Device 100 optionally includes a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information about the location of device 100.

[0060] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a tactile feedback module (or instruction set) 133, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 stores device / global internal state 157, such as in Figure 1A3. The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications (if any) are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state, including information obtained from the device's various sensors and other input or control devices 116; and position and / or orientation information about the device's position and / or posture.

[0061] The operating system 126 (e.g., iOS, Android, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.), and facilitates communication between various hardware and software components.

[0062] The communication module 128 facilitates communication with other devices through one or more external ports 124, and also includes various software components for processing data received by the RF circuit 108 and / or the external port 124. The external port 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) is suitable for coupling directly to other devices, or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a device that is compatible with some of Apple Inc. (Cupertino, California). iPod and In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used in Apple Inc. (Cupertino, California). iPod and In some embodiments, the external port is a USB Type-C connector that is the same as, similar to, and / or compatible with the Lightning connector used in Apple Inc. (Cupertino, California) electronic devices.

[0063] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the strength of the contact (e.g., the force or pressure of the contact, or a substitute for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has stopped (e.g., detecting a finger lift event or contact disconnection). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact or stylus contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, contact / motion module 130 and display controller 156 detect contact on a touch pad.

[0064] The contact / motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Therefore, gestures are optionally detected by detecting specific contact patterns. For example, detecting a single-finger tap gesture includes detecting a finger press event, and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at an icon location). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event. Similarly, taps, swipes, drags, and other gestures of the stylus are optionally detected by detecting a specific contact pattern of the stylus.

[0065] In some embodiments, detecting a finger tap gesture depends on detecting the length of time between a finger press event and a finger lift event, but is independent of the strength of the finger contact between the finger press event and the finger lift event. In some embodiments, a tap gesture is detected based on determining that the length of time between the finger press event and the finger lift event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the strength of the finger contact during the tap reaches a given strength threshold (greater than a nominal contact detection strength threshold), such as a light press or deep press strength threshold. Therefore, a finger tap gesture can meet a specific input criterion that does not require the characteristic strength of the contact to meet a given strength threshold to meet the specific input criterion. For clarity, the finger contact in a tap gesture is generally required to meet a nominal contact detection strength threshold to detect a finger press event, and the contact will not be detected below the nominal contact detection strength threshold. A similar analysis applies to detecting a tap gesture by a stylus or other contact. In the case where the device is capable of detecting contact with a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0066] The same concepts apply in a similar manner to other types of gestures. For example, a swipe gesture, a pinch gesture, an expand gesture, and / or a long press gesture may be optionally detected based on satisfying criteria that are independent of the strength of the contacts included in the gesture or that do not require one or more contacts performing the gesture to reach a strength threshold in order to be recognized. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a zoom gesture is detected based on the movement of two or more contacts toward each other; a zoom gesture is detected based on the movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of a contact on a touch-sensitive surface having less than a threshold amount of movement. Thus, the statement that a particular gesture recognition criterion does not require the strength of the contacts to satisfy a corresponding strength threshold in order to satisfy the particular gesture recognition criterion means that the particular gesture recognition criterion can be satisfied when the contacts in the gesture do not reach the corresponding strength threshold, and can also be satisfied when one or more contacts in the gesture reach or exceed the corresponding strength threshold. In some embodiments, a tap gesture is detected based on determining that a finger down event and a finger up event are detected within a predefined time period, regardless of whether the contact is above or below a corresponding strength threshold during the predefined time period, and a swipe gesture is detected based on determining that the contact moves greater than a predefined amount, even if the contact is above the corresponding strength threshold at the end of the contact movement. Even in specific implementations where detection of gestures is affected by the strength of the contact performing the gesture (e.g., the device detects a long press more quickly when the strength of the contact is above the strength threshold, or the device delays detection of a tap input when the strength of the contact is higher), detection of these gestures does not require the contact to reach a particular strength threshold (e.g., even if the amount of time required to recognize the gesture varies), as long as the criteria for recognizing the gesture can be met without the contact reaching the particular strength threshold.

[0067] In some cases, the contact intensity threshold, duration threshold, and movement threshold are combined in various combinations to create a heuristic algorithm to distinguish between two or more different gestures for the same input element or area, so that multiple different interactions with the same input element can provide a richer set of user interactions and responses. The statement that a particular set of gesture recognition criteria does not require the intensity of one or more contacts to meet the corresponding intensity threshold in order to meet the particular gesture recognition criteria does not exclude the simultaneous evaluation of other intensity-related gesture recognition criteria to identify other gestures with criteria that are met when the gesture includes a contact with an intensity above the corresponding intensity threshold. For example, in some cases, a first gesture recognition criterion for a first gesture (which does not require the intensity of the contact to meet the corresponding intensity threshold to meet the first gesture recognition criterion) competes with a second gesture recognition criterion for a second gesture (which depends on the contact reaching the corresponding intensity threshold). In such a competition, if the second gesture recognition criterion for the second gesture is met first, the gesture is optionally not recognized as satisfying the first gesture recognition criterion for the first gesture. For example, if the contact reaches the corresponding intensity threshold before the contact moves a predefined amount of movement, a deep press gesture is detected instead of a swipe gesture. Conversely, if the contact moves the predefined amount of movement before the contact reaches the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in such cases, the first gesture recognition criteria for the first gesture still does not require the intensity of the contact to satisfy the corresponding intensity threshold to satisfy the first gesture recognition criteria because if the contact remains below the corresponding intensity threshold until the gesture ends (e.g., a swipe gesture with an intensity of the contact that does not increase above the corresponding intensity threshold), the gesture will be recognized as a swipe gesture by the first gesture recognition criteria. Thus, a particular gesture recognition criterion that does not require the intensity of the contact to satisfy the corresponding intensity threshold to satisfy the particular gesture recognition criterion will (A) in some cases ignore the intensity of the contact relative to the intensity threshold (e.g., for a tap gesture) and / or (B) in some cases fail to satisfy the particular gesture recognition criterion (e.g., for a long press gesture) if a set of competing intensity-related gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-related gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input, and in this sense still depends on the intensity of the contact relative to the intensity threshold (e.g., for a long press gesture competing with the deep press gesture for recognition).

[0068] In conjunction with accelerometer 167, gyroscope 168 and / or magnetometer 169, attitude module 131 optionally detects attitude information about the device, such as the attitude of the device in a particular reference frame (e.g., roll, pitch and / or yaw). Attitude module 131 includes software components for performing various operations related to detecting device position and detecting changes in device attitude.

[0069] Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other displays, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, without limitation, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0070] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes for specifying the graphics to be displayed from an application program, etc., and also receives coordinate data and other graphic attribute data together if necessary, and then generates screen image data to be output to the display controller 156.

[0071] The tactile feedback module 133 includes various software components for generating instructions (e.g., instructions used by the tactile feedback controller 161) to produce tactile output at one or more locations on the device 100 using one or more tactile output generators 163 in response to user interaction with the device 100.

[0072] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0073] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services such as the Weather widget, the Local Pages widget, and the Maps / Navigation widget).

[0074] Notification module 145 provides logic for processing detected events such as incoming messages or phone calls, calendar event invitations or reminders, application events, system events, etc. The notification module facilitates and / or controls the presentation of notifications and reminders associated with the detected events (e.g., on a display, via an audio output, or via another type of output). For example, notification module 145 may interact with application 136 to present notifications associated with the application and / or interact with operating system 126 to present system notifications.

[0075] The application 136 optionally includes the following modules (or instruction sets) or a subset or superset thereof:

[0076] Contacts module 137 (sometimes called address book or contact list);

[0077] Telephone module 138;

[0078] Video conferencing module 139;

[0079] Email client module 140;

[0080] Instant messaging (IM) module 141;

[0081] Fitness support module 142;

[0082] A camera module 143 for still images and / or video images;

[0083] Image management module 144;

[0084] Browser module 147;

[0085] Calendar module 148;

[0086] A widget module 149, which optionally includes one or more of the following: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm widget 149-4, a dictionary widget 149-5, and other widgets acquired by the user, and a user-created widget 149-6;

[0087] A widget creator module 150 for forming a user-created widget 149-6;

[0088] Search module 151;

[0089] A video and music player module 152, optionally consisting of a video player module and a music player module;

[0090] Memo module 153;

[0091] Map module 154; and / or

[0092] Online video module 155.

[0093] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

[0094] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contacts module 137 includes executable instructions for managing an address book or contact list (e.g., stored in the application internal state 192 of the contacts module 137 in memory 102 or memory 370), including: adding names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers and / or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing 139, email 140, or instant messaging 141; and the like.

[0095] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 includes executable instructions for entering a character sequence corresponding to a phone number, accessing one or more phone numbers in address book 137, modifying an entered phone number, dialing a corresponding phone number, conducting a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communications optionally use any of a variety of communication standards, protocols, and technologies.

[0096] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0097] In conjunction with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0098] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for performing the following operations: inputting a character sequence corresponding to an instant message, modifying previously input characters, transmitting the corresponding instant message (e.g., using a short message service (SMS) or multimedia message service (MMS) protocol for phone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Service (APNs) or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or enhanced messaging services (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs or IMPS).

[0099] In combination with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and video and music player module 152, fitness support module 142 includes executable instructions for creating workouts (e.g., with time, distance, and / or calorie burn goals); communicating with fitness sensors (in sports equipment and smart watches); receiving fitness sensor data; calibrating sensors for monitoring fitness; selecting and playing music for a workout; and displaying, storing, and transmitting fitness data.

[0100] In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0101] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, displaying (e.g., in a digital slide show or album), and storing still images and / or video images.

[0102] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132 and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) in accordance with user instructions.

[0103] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0104] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5), or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, the desktop widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., the Yahoo! desktop widget).

[0105] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget creator module 150 includes executable instructions for creating a desktop widget (e.g., transferring a user-specified portion of a web page to a desktop widget).

[0106] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0107] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

[0108] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the notepad module 153 includes executable instructions for creating and managing notepads, to-do lists, etc. according to user instructions.

[0109] In combination with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and data associated with the maps (e.g., driving routes; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0110] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on touch screen 112 or on an external display connected wirelessly or via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a particular online video.

[0111] Each module and application identified above corresponds to a set of executable instructions for performing one or more functions and methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented with independent software programs, processes or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

[0112] In some embodiments, the device 100 is a device where the operation of a predefined set of functions on the device is performed exclusively through a touch screen and / or a touch pad. By using a touch screen and / or a touch pad as the primary input control device for operating the device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on the device 100 is optionally reduced.

[0113] A predefined set of functions that are performed exclusively through the touch screen and / or trackpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates the device 100 to a main menu, a home menu, or a root menu from any user interface displayed on the device 100. In such embodiments, a touch-sensitive surface is used to implement a "menu button." In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touch-sensitive surface.

[0114] Figure 1B 1 is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3A ) includes an event classifier 170 (e.g., in the operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0115] The event classifier 170 receives event information and determines the application 136-1 and the application view 191 of the application 136-1 to which the event information is to be delivered. The event classifier 170 includes an event monitor 171 and an event distributor module 174. In some embodiments, the application 136-1 includes an application internal state 192, which indicates one or more current application views displayed on the touch-sensitive display system 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application(s) is currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0116] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or is ready to be displayed by the application 136-1, a state queue for enabling a user to return to a previous state or view of the application 136-1, and a repeat / undo queue of previous actions taken by the user.

[0117] Event monitor 171 receives event information from peripherals interface 118. Event information includes information about sub-events (e.g., a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 167, and / or microphone 113 (through audio circuit 110). The information received by peripherals interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

[0118] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0119] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0120] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0121] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as correct input is optionally determined at least in part based on the hit view of the initial touch that started the touch-based gesture.

[0122] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it is identified as the hit view.

[0123] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, higher views in the hierarchy will still remain as actively participating views.

[0124] Event distributor module 174 distributes event information to event recognizers (e.g., event recognizers 180). In embodiments including active event recognizer determination module 173, event distributor module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event distributor module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.

[0125] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or is part of another module stored in memory 102, such as contact / motion module 130.

[0126] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events occurring in a corresponding view of the user interface of the application. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, the corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more event recognizers in event recognizers 180 are part of an independent module, which is a higher-level object such as a user interface toolkit (not shown) or application 136-1 from which methods and other properties are inherited. In some embodiments, the corresponding event handler 190 includes one or more of the following: data updater 176, object updater 177, GUI updater 178, and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177 or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views in application view 191 include one or more corresponding event handlers 190. In addition, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0127] A corresponding event identifier 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event identifier 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0128] Event receiver 182 receives event information from event classifier 170. Event information includes information about sub-events such as touch or touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of the touch, the event information optionally also includes the speed and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another orientation (for example, from a longitudinal orientation to a transverse orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0129] Event comparator 184 compares event information with predefined event or sub-event definitions, and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift (touch end) of a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display system 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0130] In some embodiments, event definition 187 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler associated with the sub-event and the object that triggered the hit test.

[0131] In some embodiments, the definition of the corresponding event 187 also includes delay actions that delay the delivery of the event information until it has been determined that the sub-event sequence does or does not correspond to the event type of the event identifier.

[0132] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definition 186, the corresponding event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0133] In some embodiments, the corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0134] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is different from sending (and deferred sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a tag associated with the identified event, and the event handler 190 associated with the tag obtains the tag and performs a predefined process.

[0135] In some embodiments, the event delivery instructions 188 include a sub-event delivery instruction that delivers event information about a sub-event without activating an event handler. Instead, the sub-event delivery instruction delivers the event information to an event handler associated with a sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or with an actively participating view receives the event information and performs a predetermined process.

[0136] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video or music player module 152. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the position of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0137] In some embodiments, event handler 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0138] It should be understood that the above discussion of event processing of user touches on a touch-sensitive display also applies to other forms of user input that utilize input devices to operate the multifunction device 100, and not all user input is initiated on the touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a touchpad, such as tapping, dragging, scrolling, etc.; stylus input; input based on real-time analysis of video images obtained by one or more cameras; movement of the device; verbal commands; detected eye movement; biometric input; and / or any combination thereof are optionally used as input corresponding to sub-events defining the event to be recognized.

[0139] Figure 2 A device having a touch screen (e.g., Figure 1AThe portable multifunction device 100 of the present invention is a touch-sensitive display system 112 of the present invention. The touch screen optionally displays one or more graphics within the user interface (UI) 200. In these embodiments and in other embodiments described below, the user can select one or more of the graphics by, for example, making gestures on the graphics using one or more fingers 202 (not drawn to scale in the figure) or one or more styluses 203 (not drawn to scale in the figure). In some embodiments, when the user interrupts contact with the one or more graphics, selection of the one or more graphics will occur. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, from right to left, up and / or down) and / or rolling of the fingers that have been in contact with the device 100 (from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with the graphics will not select the graphics. For example, when the gesture corresponding to the selection is a tap, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application.

[0140] The device 100 optionally also includes one or more physical buttons, such as a "home button" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen display.

[0141] In some embodiments, the device 100 includes a touch screen display, a menu button 204 (sometimes referred to as a home button 204), a push button 206 for powering the device on / off and for locking the device, a volume adjustment button 208, a user identity module (SIM) card slot 210, an earphone jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and keeping the button pressed for a predefined time interval; lock the device by pressing the button and releasing the button before the predefined time interval passes; and / or unlock the device or initiate an unlocking process. In some embodiments, the device 100 also accepts voice input for activating or deactivating certain functions through a microphone 113. The device 100 also optionally includes one or more contact strength sensors 165 for detecting contact strength on the touch-sensitive display system 112, and / or one or more tactile output generators 163 for generating tactile output for a user of the device 100.

[0142] Figure 3A300 is a block diagram of an exemplary multi-function device with a display and a touch-sensitive surface according to some embodiments. Device 300 does not have to be portable. In some embodiments, device 300 is a gaming system, a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a gaming system, or a control device (e.g., a home controller or an industrial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuits (sometimes referred to as chipsets) that interconnect system components and control communications between system components. Device 300 includes an input / output (I / O) interface 330 with a display 340, which is optionally a touch screen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on device 300 (e.g., similar to the above reference). Figure 1A The one or more tactile output generators 163 described above), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch sensors, and / or sensors similar to those described above) Figure 1A Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDRRAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located away from CPU 310. In some embodiments, memory 370 stores information related to portable multifunction device 100 ( Figure 1A ) or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100 ( Figure 1A )'s memory 102 optionally does not store these modules.

[0143] Figure 3AEach element in the above-mentioned identified element is optionally stored in one or more memory devices in the previously mentioned memory device. Each module in the above-mentioned identified module corresponds to the instruction set for performing the above-mentioned functions. The above-mentioned identified module or program (that is, instruction set) need not be implemented as an independent software program, process or module, so the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 370 optionally stores the subgroup of the above-mentioned modules and data structures. In addition, memory 370 optionally stores additional modules and data structures not described above.

[0144] Figure 3B is a block diagram of an exemplary wearable audio output device 301 according to some embodiments. In some embodiments, the wearable audio output device 301 is a single earphone. In some embodiments, the wearable audio output device 301 includes a pair of earphones (e.g., one earphone for each ear of the user's ears) or more. In some embodiments, the wearable audio output device 301 is one or more in-ear headphones, over-ear headphones, etc. In some embodiments, the wearable audio output device 301 includes one or more speakers 306 for providing audio output (e.g., provided to the user's ears). In some embodiments, the wearable audio output device 301 includes one or more placement sensors 304 to detect the positioning or placement of the wearable audio output device 301 relative to the user's ears, such as detecting the placement of the wearable audio output device 301 in the user's ears. In some embodiments, the wearable audio output device 301 conditionally outputs audio based on whether the wearable audio output device 301 is in the user's ear or near the user's ear (e.g., when the wearable audio output device 301 is not in the user's ear, the output audio is abandoned to reduce power usage). In some embodiments, the audio I / O logic component 312 determines the positioning or placement of the wearable audio output device 301 relative to the user's ear based on information received from the placement sensor 304, and in some embodiments, the audio I / O logic component 312 controls the resulting conditional audio output.

[0145] In some embodiments, the wearable audio output device 301 includes one or more microphones 302 for receiving audio input. In some embodiments, the microphone 302 detects speech from a user wearing the wearable audio output device 301 and / or ambient noise around the wearable audio output device 301. In some embodiments, the audio I / O logic component 312 detects or recognizes speech or ambient noise based on information received from the microphone 302. In some embodiments, the wearable audio output device 301 includes one or more other input devices 308, such as a touch-sensitive surface (for detecting touch input), an accelerometer and / or a posture sensor (for determining the posture of the wearable audio output device 301 relative to the physical environment and / or changes in the device posture) and / or other input devices through which a user can interact with the wearable audio output device 301 and provide input to the wearable audio output device. In some embodiments, input provided via one or more input devices 308 is processed by the audio I / O logic component 312. In some embodiments, the audio I / O logic component 312 communicates with independent devices (e.g., Figure 1A The independent device 100 communicates with the independent device 100, which provides instructions or content for audio output and optionally receives and processes input (or information about the input) provided via the microphone 302, the placement sensor 304 and / or the input device 308.

[0146] Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .

[0147] Figure 4A An exemplary user interface of an application menu on portable multifunction device 100 is shown according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0148] One or more signal strength indicators of one or more wireless communications, such as cellular signals and Wi-Fi signals;

[0149] ·time;

[0150] Bluetooth indicator;

[0151] Battery status indicator;

[0152] A tray 408 with icons for commonly used applications, such as:

[0153] o an icon 416 of the phone module 138 labeled “Phone”, the icon 416 optionally including an indicator 414 of the number of missed calls or voice messages;

[0154] o an icon 418 of the email client module 140 labeled “Mail”, the icon 418 optionally including an indicator 410 of the number of unread emails;

[0155] o an icon 420 labeled "Browser" of the browser module 147; and

[0156] o Icon 422 labeled “Music” of the video and music player module 152;

[0157] and

[0158] Icons for other apps, such as:

[0159] o Icon 424 labeled “Messages” of IM module 141;

[0160] o Icon 426 labeled “Calendar” of calendar module 148;

[0161] o Icon 428 labeled “Photos” of the image management module 144;

[0162] o Icon 430 labeled “Camera” of camera module 143;

[0163] o Icon 432 labeled “Online Video” of the online video module 155;

[0164] o Icon 434 labeled “Stock Market” of the Stock Market widget 149-2;

[0165] o an icon 436 labeled “MAP” of the map module 154;

[0166] o Icon 438 labeled “Weather” of the Weather widget 149-1;

[0167] o Icon 440 labeled “Clock” of the alarm clock widget 149-4;

[0168] o an icon 442 labeled “Fitness Support” of the fitness support module 142;

[0169] o Icon 444 labeled "Notepad" of the Notepad module 153; and

[0170] o An icon 446 of a settings application or module labeled “Settings” that provides access to settings for the device 100 and its various applications 136 .

[0171] It should be noted that Figure 4AThe icon labels shown in are exemplary only. For example, other labels are optionally used for various application icons. In some embodiments, the label of the corresponding application icon includes the name of the application corresponding to the corresponding application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0172] Figure 4B A touch-sensitive surface 451 (eg, Figure 3A A device (e.g., a tablet or touch pad 355) Figure 3A Although many of the examples that follow will be given with reference to input on a touch screen display 112 (where a touch-sensitive surface and a display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the main axis (for example, Figure 4B According to these embodiments, the device detects contact with the touch-sensitive surface 451 at a location corresponding to a corresponding location on the display (e.g., Figure 4B 460 and 462) (e.g., in Figure 4B 460 corresponds to 468 and 462 corresponds to 470). Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g., Figure 4B 450 in ) is separated, the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0173] In some embodiments, the response of the device to the input detected by the device depends on a criterion based on the contact strength during the input. For example, for some "light press" inputs, the strength of the contact exceeding the first strength threshold during the input triggers a first response. In some embodiments, the response of the device to the input detected by the device depends on a criterion including both the contact strength during the input and a time-based criterion. For example, for some "deep press" inputs, as long as a delay time passes between satisfying the first strength threshold and satisfying the second strength threshold, the strength of the contact exceeding the second strength threshold greater than the first strength threshold of the light press during the input triggers a second response. The duration of the delay time is typically less than 200ms (milliseconds) (e.g., 40ms, 100ms, or 120ms, depending on the magnitude of the second strength threshold, where the delay time increases as the second strength threshold increases). The delay time helps avoid accidentally identifying a deep press input. As another example, for some "deep press" inputs, a period of reduced sensitivity will occur after reaching the first strength threshold. During this period of reduced sensitivity, the second strength threshold increases. This temporary increase in the second intensity threshold also helps avoid accidental deep press inputs.For other deep press inputs, the response to detecting the deep press input does not depend on time-based criteria.

[0174] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of simultaneous inputs, user history, environmental factors (e.g., ambient noise), focus selector position, etc. Example factors are described in U.S. patent applications 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entirety.

[0175] For example, Figure 4C A dynamic intensity threshold 480 is shown that changes over time based in part on the intensity of touch input 476 over time. Dynamic intensity threshold 480 is the sum of two components: a first component 474 that decays over time after a predefined delay time p1 from when touch input 476 is initially detected, and a second component 478 that tracks the intensity of touch input 476 over time. The initial high intensity threshold of first component 474 reduces accidental triggering of a “deep press” response while still allowing an immediate “deep press” response if touch input 476 provides sufficient intensity. Second component 478 reduces unintentional triggering of a “deep press” response through gradual intensity fluctuations of the touch input. In some embodiments, when touch input 476 satisfies dynamic intensity threshold 480 (e.g., at Figure 4C ), triggering a "deep press" response.

[0176] Figure 4D Another dynamic intensity threshold 486 (eg, intensity threshold IT D ). Figure 4D Two other intensity thresholds are also shown: a first intensity threshold IT H and the second intensity threshold IT L .exist Figure 4D Although the touch input 484 satisfies the first intensity threshold IT before time p2, H and the second intensity threshold IT L , but no response is provided until a delay time p2 has elapsed at time 482. In addition, Figure 4D , the dynamic intensity threshold 486 decays over time, wherein the decay is from time 482 (when the trigger is in contact with the intensity threshold IT L This type of dynamic intensity threshold reduction immediately follows the trigger with a lower threshold intensity (such as the first intensity threshold IT H Or the second intensity threshold IT L ) is triggered after or simultaneously with the response associated with the dynamic intensity threshold IT D The associated response.

[0177] Figure 4E Another dynamic intensity threshold 492 (eg, intensity threshold IT D ).exist Figure 4E In the example, after a delay time p2 has passed since the touch input 490 was initially detected, the trigger and intensity threshold IT L At the same time, the dynamic intensity threshold 492 decays after a predefined delay time p1 has passed since the touch input 490 was initially detected. L Reducing the intensity of touch input 490 after the associated response, and then increasing the intensity of touch input 490 without releasing touch input 490 may trigger the same as intensity threshold IT D The associated response (eg, at time 494) is performed even when the intensity of touch input 490 is below another intensity threshold (eg, intensity threshold IT L ) is also the case.

[0178] User interface and associated processes

[0179] Attention is now turned to the Figure 1A The portable multifunction device 100 or Figure 3AEmbodiments of a user interface ("UI") and associated processes implemented on a device 300) that includes one or more audio output devices (e.g., one or more wearable audio output devices such as Figure 3B A wearable audio output device 301, optionally one or more sensors for detecting placement of the audio output device, optionally a display and / or a touch-sensitive surface, optionally one or more accelerometers and / or gesture sensors, and optionally one or more additional input devices (e.g., a rotatable input mechanism) (and / or communicating with these devices).

[0180] FIG. 5A to FIG. 5AD Exemplary audio notifications and exemplary user interfaces and interactions with audio notifications according to some embodiments are shown. The user interfaces in these figures are used to illustrate the processes described below, including FIG. 6A to FIG. 6E , 7A to 7C , FIG. 8A to FIG. 8D , 9A to 9C and FIG. 10A to FIG. 10B For ease of explanation, some of the embodiments will be discussed with reference to operations performed using a wearable output device that is in communication with and separate from a device that detects an event corresponding to an audio notification. However, in some cases, similar operations are optionally performed using an audio output device that is part of the device that detects the event.

[0181] FIG. 5A to FIG. 5AD Exemplary audio notifications and exemplary user interfaces (including verbal, auditory, and tactile interfaces) and interactions with audio notifications are shown according to some embodiments.

[0182] Figure 5A 500 is shown providing an audio notification in response to the user 500 wearing one or more headphones 502. The headphones 502 represent a single headphone, two headphones (one headphone in each ear or on each ear of the user 500), in-ear headphones, over-ear headphones, etc. In some embodiments, the headphones 502 correspond to the wearable audio output device 301 ( Figure 3B In some embodiments, the headset 502 is combined with a device that can be separated from or integrated with the headset 502 (e.g., Figure 1A device 100) communicates.

[0183] exist Figure 5A In the embodiment, when the earphone 502 is far from the ear of the user 500 and is positioned outside the range 504 around the ear of the user 500, no audio output is provided via the earphone 502.

[0184] When the earphone 502 is placed in the ear of the user 500 (or moved within its range 504), an audio summary 506 of the day is output via the earphone 502. In some embodiments, the audio summary 506 includes an audio notification at a device in communication with the earphone 502 (e.g., corresponding to an event that has occurred, is scheduled to occur, and / or is predicted to occur). In some embodiments, such as Figure 5A As shown, the audio summary 506 is provided only in response to the first detection of placement of the earphone 502 in the ear of the user 500 on a particular day. Figure 5A As shown, the audio summary 506 includes one or more of the following:

[0185] The current date (for example, stating “Today is Monday, September 24”)

[0186] The weather for the day (e.g., stating “Sunny, current temperature 65 degrees Fahrenheit, high 78 degrees Fahrenheit”);

[0187] A summary of events missed when earphone 502 was not in or within ear range of user 500 (e.g., stating “You have four messages and five emails”);

[0188] · Commuting information (e.g., stating “It will take 25 minutes to get to work now”); and

[0189] The first upcoming calendar event (eg, stating "Your first meeting 'Project Status Update' will take place at 10:30").

[0190] Figure 5B An example of differentiating audio notifications based on the attributes of detected events is shown. Figure 5B As shown, the detected events correspond to received messages 508-1 to 508-4. Message 508-1 is received from "Harold Smith" and includes text content (e.g., text "Hey, I'll be in the city from next Wednesday to Sunday."). Subsequently, messages 508-2 and 508-3 are received from "Barbara Lee (Mom)" (the user's mother) and include text content (e.g., text "Are you going home this weekend?" and text "Don't forget that it's Dad's birthday this weekend."). Then, message 508-4 is also received from "Harold Smith" and includes text content (e.g., text "Let's get together sometime.").

[0191] In response to receiving message 508-1, an audio notification 510 including the contents of message 508-1 is provided to user 500 via earphone 502 in verbal form (e.g., by converting the text of message 508-1 to speech for reading aloud). Figure 5BIn the example shown, audio notification 510 identifies the full name (e.g., both first and last name) of the message sender "Harold Smith". Figure 5B As shown, the contact's last name (e.g., "Smith") is included in addition to his first name (e.g., "Harold") because user 500 has not communicated with the contact for at least a predefined period of time (e.g., at least the past week, month, year, or other specified period of time). In some cases, including the contact's first and last name in an audio notification associated with a message or other event received from the contact is due to including another contact with the same name in user 500's address book (e.g., including another "Harold" in user 500's address book).

[0192] In response to receiving message 508-2, an audio notification 512 including the contents of message 508-2 is provided to user 500 via earphone 502 in verbal form. Figure 5B In the example shown, audio notification 512 identifies the alias "Mom" (e.g., as specified by user 500 in her address book) of message sender "Barbara Lee" (the mother of user 500) rather than her full name "Barbara Lee." Because user 500 has corresponded with "Mom" within a predefined time period (e.g., within the past week, month, year, or other specified time period), audio notification 512 does not identify "Mom" by her full name.

[0193] In addition, in response to receiving message 508-3, an audio notification 514 including the contents of message 508-3 is provided to user 500 in verbal form via earphone 502. Figure 5B In the example shown, because message 508-3 corresponding to audio notification 514 is immediately received after another message (e.g., message 508-2 corresponding to audio notification 512) that is also received from the same sender "Barbara Lee", audio notification 514 omits the name of the message sender "Barbara Lee".

[0194] In response to receiving message 508-4, an audio notification 516 including the contents of message 508-4 is provided to user 500 in verbal form via earphone 502. Figure 5B In the example shown, audio notification 516 omits the last name of the message sender "Harold Smith" because at this point user 500 has corresponded with "Harold Smith" within the predefined time period due to message 508-1 received from the same "Harold Smith".

[0195] Figure 5CAn example of outputting audio notifications at different simulated spatial locations based on the attributes of the corresponding events is shown. In some embodiments, providing audio notifications at different simulated spatial locations requires multiple audio output devices. Therefore, Figure 5C A user 500 is shown wearing a pair of headphones 502. Figure 5C As shown, multiple events, namely messages 508-4 to 508-6, have been received. Figure 5B As described, message 508-4 is a text message from "Harold Smith". Specifically, message 508-4 is sometimes referred to as a "private message" because message 508-4 is sent from "Harold Smith" to user 500 and is not sent to other recipients. Messages 508-5 and 508-6 are both received as part of a group message conversation between three or more participants and are identified as a "team" and are sometimes referred to as a "group message". Specifically, message 508-5 is received from participant "John Godfrey" in the "team" group message and includes text content (e.g., the text "Ten minutes late, sorry."). Next, message 508-6 is received from a different participant "Alice Young" in the same "team" group message and includes text content (e.g., the text "No problem, let's discuss other projects first.").

[0196] In response to receiving message 508-4, an audio notification 518 including the content of message 508-4 is provided to user 500 via headset 502 in verbal form. Because message 508-4 is a private chat message, audio notification 518 is output at a simulated spatial location 520 inside the head of user 500, so that audio notification 518 sounds as if it comes from or is playing at the spatial location 520 inside the head of user 500. In some embodiments, private chat messages such as audio notification 518 are output at one or more simulated spatial locations inside the head of user 500. In some embodiments, different spatial locations are associated with different contacts. For example, in some embodiments, audio notifications from senders other than "Harold Smith" are output at simulated spatial locations different from simulated spatial location 520 inside the head of user 500.

[0197] In response to receiving messages 508-5 and 508-6, audio notifications 522 and 526, respectively, are provided to user 500 via earphone 502. Audio notification 522 includes the contents of message 508-5 in verbal form, the audio notification beginning with an indication that the message is part of a group message and an identifier of the group message (e.g., the group message begins with "In group 'Team'," where "Team" is the group message identifier). In addition, audio notification 522 is output at a simulated spatial location 524 outside of the head of user 500, so that audio notification 522 sounds as if it is coming from or being played at the spatial location 524 of the head of user 500 (e.g., even if the sound is coming from earphone 502 in the ear of user 500). Next, audio notification 526 includes the contents of message 508-6 in verbal form. In some embodiments, such as in Figure 5C In the example shown, because message 508-6 corresponding to audio notification 526 is immediately followed by another message (e.g., message 508-5 corresponding to audio notification 522) that is also received as part of the same group message (e.g., the "Team" group message), audio notification 526 omits the group message beginning (e.g., "In group 'Team'"). Audio notification 526 is output using a different type of audio (e.g., a different audio tone or a different simulated voice) than audio notification 522 (e.g., as indicated by the italicized text representing audio notification 526 in contrast to the normal text representing audio notification 522). In addition, audio notification 526 is output at a simulated spatial location 528 outside of the head of user 500, so that audio notification 526 sounds as if it is coming from or being played at spatial location 528 outside of the head of user 500.

[0198] In some embodiments, such as Figure 5C As shown, because messages 508-5 and 508-6 are group messages rather than private messages, audio notifications 522 and 526 are output at simulated spatial locations (e.g., simulated spatial locations 524 and 528, respectively) outside the head of user 500. In addition, in some embodiments, as Figure 5C As shown, because messages 508 - 5 and 508 - 6 are messages from different senders (eg, different participants in a group message), audio notification 522 is output at a different simulated spatial location than audio notification 526 .

[0199] In some embodiments, messages of different types of events are distinguished by outputting corresponding audio notifications at different volumes. Figure 5CAs shown, audio notification 518 is output at a higher volume (as indicated by bold text) than audio notifications 522 and 526. In some embodiments, private chat messages (e.g., message 508-4 sent from "Harold Smith" to only user 500) are output at a higher volume relative to group messages (e.g., messages 508-5 and 508-6 in the group "team" between user 500 and at least "John Godfrey" and "Alice Young"). A person of ordinary skill will appreciate that many other ways of distinguishing events are possible, including, but not limited to, emphasizing audio notifications from specific contacts over other contacts (e.g., in response to a user specifying specific contacts, such as by creating a contact "whitelist"), emphasizing audio notifications associated with specific applications over other applications, emphasizing audio notifications associated with specific types of applications over other types of applications, emphasizing audio notifications that meet specific geographic or temporal criteria (e.g., emphasizing audio notifications of certain events that occur when the user is at or near a specific geographic location or during a specific time of day), and the like.

[0200] Figure 5D An example of differentiating audio notifications based on geographic context is shown. Figure 5D As shown, (for example, in a device such as Figure 1A A first event 530 is detected at device 100 of the present invention. Event 530 is a notification for an upcoming calendar event, which is a meeting at a specified location 533 (e.g., "Apollo conference room, Building 1, N. De Anza Blvd., Cupertino") at a specified time 532 (e.g., "10:30 a.m.") about a specified subject 531 (e.g., "project status update"). Event 530 is detected at a predefined time related to the calendar event (e.g., current time 534, "10:20 a.m.", ten minutes before the start of the upcoming calendar event). In response to detecting event 530, an audio notification 538 is provided via earphones 502 to notify user 500 of the upcoming meeting. Audio notification 538 includes information about event 530, including an indication of the event type (e.g., "meeting"), the event time (e.g., "at 10:30 a.m."), and the event subject (e.g., "project status update").

[0201] like Figure 5D As shown, audio notification 538 also includes event location information (e.g., "Apollo Conference Room"). Figure 5DIn the example shown, the current location 536 of user 500 is "Lobby of Building 1." Therefore, user 500 is already in the same building (e.g., "Building 1") as the meeting location associated with event 530. Therefore, audio notification 538 omits the name of the building (e.g., "Building 1") and any description of a larger geographic area such as a street name and a city name. Audio notification 538 includes location information for a geographic area that is only different from the current location 536 of user 500 (e.g., a geographic area within a building, such as a floor number or a conference room name). In some embodiments, in the case where user 500 is already in the same minimum geographic area (e.g., the same conference room) as the upcoming event, event location information is not provided in the corresponding audio notification.

[0202] Figure 5D It also indicates (e.g., in a device such as a Figure 1A A second event 540 is detected at device 100 of the present invention. Event 540 is a notification for an upcoming calendar event, which is a meeting on a specified topic 541 (e.g., "test bed evaluation") at a specified time 542 (e.g., "3:00 p.m.") at a specified location 543 (e.g., "Montara conference room, Building 5, S. Main St., Sunnyvale"). Event 540 is detected at a predefined time related to the calendar event (e.g., current time 544, "2:50 p.m.", ten minutes before the start of the upcoming calendar event). In response to detecting event 540, an audio notification 548 is provided via earphones 502 to notify user 500 of the upcoming meeting. Audio notification 548 includes information about event 540, including an indication of the event type (e.g., "meeting"), the event time (e.g., "at 3 o'clock"), and the event topic (e.g., "test bed evaluation").

[0203] In some embodiments, the audio notification 548 also includes event location information (e.g., "Montara conference room, Building 5, South Main Street, Sunnyvale"). Figure 5DIn the example shown, user 500's current location 546 is "Lobby of Building 1." Thus, user 500 is in a different city (e.g., "Cupertino") than the city (e.g., "Sunnyvale") where event 540 is about to occur, even though both cities are in the same state. Thus, audio notification 548 includes the name of the conference room (e.g., "Montara"), the name of the building (e.g., "Building 5"), the name of the street (e.g., "South Main Street"), and the name of the city (e.g., "Sunnyvale"), all of which are different geographic areas than user 500's current location 546. Audio notification 548 omits the name of the state where event 540 is about to occur because user 500 is in that same state.

[0204] Figure 5E An example of differentiating audio notifications based on the attributes of received events is shown. Specifically, Figure 5E Different sound effects and output volumes are shown for events indicating transactions of different values. In response to user 500 receiving a payment from "Travis" for a small amount of money (e.g., his "lunch" expense), an audio notification 550 describing the transaction includes a first "[ding-ding (coins falling)]" sound effect and is played at a low volume (as indicated by lowercase text) via headphones 502. In response to user 500 receiving a payment from "Jane" for a large amount of money (e.g., her "rent" expense), an audio notification 552 describing the transaction includes a second "[clack]" cash register sound effect that is different from the first sound effect and is played at a high volume (as indicated by bold capital letters) via headphones 502.

[0205] Fig. 5F 2 shows exemplary operations performed in response to user 500 interacting with headset 502 for at least a predefined period of time in the absence of received notifications or events. Specifically, Fig. 5F Indicates that no notifications have been received (e.g., by a device in communication with the headset 502) in the last ten minutes. Therefore, in response to detecting the tap input 554 on the headset 502, the headset 502 begins playing music, as indicated by the audio output 556.

[0206] Figure 5G to Figure 5H 5. The output of compressed audio notifications in response to detecting multiple events when the earphone 502 is not in the ear of the user 500 is shown, wherein the corresponding audio notifications of the detected events are generally voluminous and / or lengthy. Specifically, Figure 5GIt is shown that multiple events are detected when the earphone 502 is not in the ear of the user 500. The detected events correspond to messages 558-1 to 558-3 received from "Travis", messages 560-1 and 560-2 and event invitation 560-3 received from "Octavia", and messages 562-1 to 562-5 received from "Ellie". When the earphone 502 is not in the ear of the user 500, audio notifications corresponding to the detected events are not output via the earphone 502.

[0207] Figure 5H The earphone 502 is shown to have been placed in the ear of the user 500. In response, an audio notification corresponding to the detected event is provided. However, outputting the audio content associated with each of the detected events 558-1 to 558-3, 560-1 to 560-3, and 562-1 to 562-5 will generally require more than a predefined amount of time. In contrast, if Figure 5H As shown, a compressed audio notification is provided. Specifically, audio notification 564 summarizes all messages 558-1 to 558-3 received from "Travis" into the statement "Travis sent you three messages." Similarly, audio notification 566 summarizes all messages 560-1 and 560-2 and event invitation 560-3 received from "Octavia" into the statement "Octavia sent you two messages and an event invitation." Figure 5H In the example of , audio notifications 564 and 566 are output at simulated spatial location 568.

[0208] FIG. 5I to FIG. 5K The amount of content included in the audio notification (sometimes referred to as the "details" of the audio notification) is shown changing in response to user 500 interacting with earphones 502 via head movements. Fig.5I 560 - 1 to 560 - 3, the user 500 tilts his head toward the simulated spatial location 568 (e.g., toward the "source" of the sound) to hear more information about the audio notification that just played (or in some cases, the audio notification that is currently playing). Figure 5J, additional audio content associated with events 560-1 through 560-3 is provided in the form of audio notifications 566-1 through 566-3 (e.g., an expanded version of audio notification 566). Audio notification 566-1 includes the content of message 560-1 in verbal form, beginning only with the name of the sender, "Octavia," with whom user 500 has communicated "recently" (e.g., within a predefined time period, such as the last week, month, or year). Audio notification 566-2 includes the content of message 560-2 in verbal form, omitting any identifier of the sender (because message 560-2 immediately follows another event (message 560-1)) from the same sender. Audio notification 566-3 describes event invitation 560-3 (e.g., "Octavia invites you to an event today at 4:30 PM."). In some embodiments, because event invitation 560-3 follows immediately after another event (eg, message 560-2) from the same sender (eg, "Octavia"), audio notification 566-3 is rephrased to omit any identifier of the sender.

[0209] Figure 5K 566-3, the user 500 tilts his head away from the simulated spatial location 568 to hear less information about the subsequent audio notification (e.g., return to listening to the compressed audio notification). In response, a compressed audio notification 570 is provided at the simulated spatial location 568. Specifically, the audio notification 570 summarizes all messages 562-1 to 562-5 received from "Ellie" into the statement "Ellie sent you five messages."

[0210] Figure 5L to Figure 5R 2 shows example operations performed in response to user interaction with headset 502. Figure 5L , while audio notification 570 is being provided, or within a predefined time period (e.g., within a few seconds) after audio notification 570 is provided, user 500 performs tap input 572 on earphone 502. In response to detecting tap input 572, additional audio content associated with messages 562-1 to 562-5 (e.g., an extended version of audio notification 570) is provided in the form of audio notifications 570-1 to 570-5, such as Figure 5M to Figure 5P In some embodiments, during or “shortly after” the audio notification (e.g., within a predefined time period, such as a few seconds), a (single) tap input directed to the earphone 502 performs an operation related to the audio notification (e.g., providing additional information, such as Figure 5L to Figure 5M ), while tap inputs outside of the time period “soon after” the audio notification perform different actions unrelated to any audio notification (e.g., starting or stopping music playback, as shown in Fig. 5F shown).

[0211] exist Figure 5M , audio notification 570-1 includes the contents of message 562-1 in verbal form, beginning only with the name of the sender "Ellie," with whom user 500 has "recently" communicated. Audio notification 570-2 includes the contents of message 562-2 in verbal form. Specifically, because message 562-2 consists of a photo, audio notification 570-2 includes a verbal description of the contents of the photo (e.g., a verbal summary including one or more keywords associated with the photo determined using image processing). Thus, audio notification 570-2 includes the statement "Ellie sent a photo of a cat" describing the photo of a cat in message 562-2. In some embodiments, because message 562-2 immediately follows another message (e.g., message 562-1) from the same sender (e.g., "Ellie"), audio notification 570-2 is rephrased to omit any identifier of the sender.

[0212] exist Figure 5N , audio notification 570-3 includes the contents of message 562-3 in verbal form, omitting any identifier of the sender (e.g., "Ellie") because message 562-3 is immediately following another message 562-2 also from the same sender. After providing audio notification 570-3, headset 502 begins providing a first portion of audio notification 570-4, which includes a first portion of the contents of message 562-4 in verbal form (e.g., a partial statement "You want to return..."). While providing audio notification 570-4 (e.g., after providing the first portion, but before providing a second subsequent portion), user 500 performs a double-click input 574 on headset 502, as shown in FIG. Fig.5O In response, the headset 502 skips the remainder of the audio notification 570-4 corresponding to the unread portion of the message 562-4 and proceeds to output the audio notification 570-5 corresponding to the message 562-5, as shown. Figure 5P In some embodiments, during or “shortly after” the audio notification (e.g., within a predefined time period, such as a few seconds), a (single) tap input directed to the earphone 502 performs an operation related to the audio notification (e.g., providing additional information, such as Figure 5L to Figure 5M ), which is different from a double-click input directed to earphone 502 "shortly" during or after an audio notification (e.g., skipping the remainder of the audio notification, such as Figure 5O to Figure 5P shown).

[0213] In addition, Figure 5P In , audio notification 570-5 is provided at an initial time as indicated by timer 576-1. Figure 5Q, after a predefined period of time (e.g., more than 7 seconds to 10 seconds) has passed since audio notification 570-5 was provided, as indicated by timer 576-2, user 500 performs a double-click input 578 on earphone 502. In response to detecting the double-click input 578 after the predefined period of time has passed, a virtual assistant is invoked (e.g., on earphone 502 or a device in communication with earphone 502, such as a Figure 1A device 100). A sound effect 580 (e.g., a virtual assistant ringtone or tone, or another indication that the virtual assistant is ready to receive verbal commands) indicates the invocation of the virtual assistant. In some embodiments, during or “shortly after” the audio notification (e.g., within a predefined time period, such as a few seconds), a double-tap input directed to earpiece 502 performs an action related to the audio notification (e.g., skipping the remainder of the audio notification, such as Figure 5O to Figure 5P ), while double-tap input outside of the time period “soon after” the audio notification performs a different action unrelated to any audio notification (e.g., invoking a virtual assistant, such as Figure 5Q to Figure 5R shown).

[0214] Figures 5S to 5AB Prompting the user and receiving a response to the audio notification is shown. In response to receiving event 560-3, audio notification 566-3 is provided to user 500 via headset 502, namely a calendar event invitation for a "4:30 PM" meeting from "Octavia". Audio notification 566-3 includes information about event 560-3 in verbal form.

[0215] After providing the audio notification 566-3, the headset 502 plays a sound effect 582 (e.g., white noise) to indicate that the headset 502 or an electronic device in communication with the headset 502 is ready to receive a response to the audio notification 566-3 and prompt the user 500 to provide a response. In some embodiments, the electronic device in communication with the headset 502 is a smart watch (e.g., a smartwatch such as Figure 5T to Figure 5Y In some embodiments, the electronic device that communicates with the headset 502 is a smart phone (e.g., Figure 5Z to Figure 5AA shown).

[0216] Figure 5T Shows from Figure 5S transformation. Figure 5T An electronic device 100 is shown in communication with earphones 502. The device 100 includes a touch screen 112 (also referred to as a "touch-sensitive display"). Figure 5SIn response to detecting the touch input 584, the selection indicator 586 indicates the selection of the first response option of the one or more response options for responding to the calendar event invitation 560-3. In response to detecting the touch input 584, the device 100 initiates the audio playback of the response options for responding to the calendar event invitation 560-3. Specifically, Figure 5T 58, and select indicator 586 indicates a selection of an option (e.g., “Accept”) to accept calendar event invitation 560-3, and an audio notification 588-1 with a response option “Accept” is played to user 500 via headset 502. In some embodiments, such as in Figure 5T In the example shown, the response options are displayed on the touch screen 112 along with an audio playback of the response options. In some embodiments, the response options are not displayed on the touch screen 112, and the user 500 listens to the response options provided audibly without looking at the touch screen 112, so as to reduce power usage of the touch screen 112. In some embodiments, one or more of the response options are automatically played after the audio notification 566-3 is played and before the sound effect 582 is played to indicate that the device 100 is ready to receive a response (e.g., as described herein with reference to Figure 5AB In some embodiments, less than all of the response options are automatically played (e.g., Figure 5AB ), and may respond to input directed to the headset 502 (e.g., a tap input or a double-click input on the headset 502, or an audio input to the headset 502 via a microphone of the headset 502) or input directed to the device 100 (e.g., reference 1 to FIG. 1 ). Figure 5T to Figure 5V The touch input 584, or the reference in this article Figure 5W to Figure 5Y The unplayed response options are accessed and played by rotation of the dial 592).

[0217] Next, in Figure 5U , in response to further movement of touch input 584, selection indicator 586 changes to indicate selection of an option to decline calendar event invitation 560-3 (e.g., “Decline”), and audio notification 588-2 with response option “Decline” is played to user 500 via headset 502. Next, in response to further movement of touch input 584, selection indicator 586 changes to indicate selection of an option to tentatively accept calendar event invitation 560-3 (e.g., “Tentative”), and audio notification 588-3 with response option “Tentative” is played to user 500 via headset 502.

[0218] Next, in Figure 5V , after providing audio notification 588-3 to user 500 (and before providing a different audio notification with a different response option), and when selection indicator 586 indicates a selection of the option to tentatively accept calendar event invitation 560-3, touch input 584 is removed from touch screen 112 at an initial time as indicated by timer 590-1. After touch input 584 has been removed from touch screen 112 for at least a predefined time period (e.g., a few seconds) as indicated by timer 590-2, the selection of the option to tentatively accept calendar event invitation 560-3 is confirmed, and audio notification 588-4 is provided to indicate that calendar event invitation 560-3 has been tentatively accepted. In some embodiments, in response to a second input (e.g., a tap input on touch screen 112) detected after providing audio notification 566-3 and before providing a different audio notification with a different response option, the selection of the corresponding response option is confirmed. In some embodiments, in the case where the selected response option is displayed on touch screen 112, the selection of the response option is confirmed when the second input is detected while selection indicator 586 indicates the selection of the response option. In some embodiments, when one or more response options are displayed simultaneously on touch screen 112, the corresponding response option is selected and confirmed in response to a touch input (e.g., a tap input) at a location on touch screen 112 corresponding to the corresponding response option.

[0219] Figure 5W Shows from Figure 5S Alternative transformation. Figure 5W The electronic device 100 is shown in communication with the headset 502. The device 100 includes a dial 592 (e.g., a rotatable input mechanism such as a crown) and a display 112 (which is optionally a touch-sensitive display such as a Figure 5U touch screen 112). In response to hearing Figure 5S In response to detecting the rotation of dial 592, selection indicator 586 indicates selection of a first response option among one or more response options for responding to calendar event invitation 560-3. In response to detecting the rotation of dial 592, device 100 initiates audio playback of the response option for responding to audio notification 560-3. Specifically, Figure 5W 5. In some embodiments, such as in FIG. 5, selection indicator 586 indicates selection of an option (e.g., “Accept”) to accept calendar event invitation 560-3, and audio notification 588-1 with a response option “Accept” is played to user 500 via headset 502. Figure 5WIn the example shown, the response options are displayed on the display 112 along with an audio playback of the response options. In some embodiments, the response options are not displayed on the display 112, and the user 500 listens to the response options provided audibly without viewing the display 112, so as to reduce power usage of the display 112.

[0220] Next, in Figure 5X , in response to further rotation of dial 592, selection indicator 586 changes to indicate selection of an option to decline calendar event invitation 560-3 (e.g., “Decline”), and audio notification 588-2 with the response option “Decline” is played to user 500 via headset 502. Next, in response to further rotation of dial 592, selection indicator 586 changes to indicate selection of an option to tentatively accept calendar event invitation 560-3 (e.g., “Tentative”), and audio notification 588-3 with the response option “Tentative” is played to user 500 via headset 502.

[0221] Next, in Figure 5Y , once audio notification 588-3 is provided to user 500, and when selection indicator 586 indicates selection of the option to tentatively accept calendar event invitation 560-3, input 594 (e.g., tap input or press input) is detected via dial pad 592. In response to press input 594 via dial pad 592, selection of the option to tentatively accept calendar event invitation 560-3 is confirmed, and audio notification 588-4 is provided to indicate that calendar event invitation 560-3 has been tentatively accepted. In some embodiments, selection of the corresponding response option is confirmed when no further input is detected via dial pad 592 within at least a predefined time period (e.g., a few seconds) after detecting rotation of dial pad 592 that causes selection of the corresponding response option or after hearing playback of the corresponding response option. In some embodiments, in the case where the selected response option is displayed on display 112, selection of the response option is confirmed when selection indicator 586 remains on the corresponding response option for at least a predefined time period. In some embodiments, the device 100 includes a touch-sensitive surface, and when one or more response options are simultaneously displayed on the display 112, the corresponding response option is selected and confirmed in response to a touch input (e.g., a tap input) at a location on the touch-sensitive surface corresponding to the corresponding response option.

[0222] Figure 5Z Shows from Figure 5S Alternative transformation. Figure 5Z An electronic device 100 is shown in communication with a headset 502. The device 100 includes a display 112. In response to hearing Figure 5SIn the sound effect 582 that plays after the audio notification 566-3, the user 500 lifts the device 100 (e.g., lifts the top edge 100a of the device 100 relative to the bottom edge 100b of the device 100) so as to change the posture of the device 100 by at least a predefined amount (e.g., a change of at least a predefined degree, such as 30 degrees; from horizontal to at least a predefined degree above horizontal, such as 30 degrees above horizontal; or from within a predefined range of horizontal, such as within 30 degrees of horizontal, to outside the predefined range).

[0223] In response to detecting a change in posture of device 100, a set of response options (e.g., options "Accept," "Reject," "Tentative," and "Skip") for responding to audio notification 566-3 are displayed on display 112 of device 100. In some embodiments, the set of response options is displayed based on determining that device 100 is raised within a predefined time period (e.g., a few seconds) from the time when earphone 502 begins to output sound effect 582.

[0224] Next, in Figure 5AA , while viewing the response options on display 112, and while sound effects 582 are playing via earphones 502, user 500 responds to calendar event invitation 560-3 by stating aloud his desired response option as indicated by response (speech) 596 “Accept.” In response to detecting response 596 (e.g., via earphones 502), device 100 receives and processes the selection of the option to accept calendar event invitation 560-3. In response, audio notification 598 is provided to indicate that calendar event invitation 560-3 has been accepted.

[0225] In some embodiments, the headset 502 stops playing the sound effect 582 after a predefined period of time. In some embodiments, after the headset 502 stops playing the sound effect 582, the device 100 will not Figure 5T to Figure 5AA The input mechanism described above is used to receive a response to the calendar event invitation 560-3. In some such embodiments, the user 500 may alternatively respond to the calendar event invitation 560-3 by interacting with one or more user interfaces of the device 100 to navigate to, view, and manually respond to the visual notification corresponding to the event 560-3 on the touch-sensitive display of the device 100.

[0226] Figure 5AB FIG. 1 shows the adjustment of the audio output volume according to the ambient noise level. Figure 5AB 5, an audio notification 5100 including information about event 560-3 is provided to user 500 in verbal form via earphone 502. Ambient noise element 5102 indicates that the ambient noise level is high (e.g., compared to a Figure 5SIn some embodiments, the ambient noise level is detected via one or more microphones on the headset 502 or on an electronic device that communicates with the headset 502. Figure 5AB In the case where the ambient noise level is high, the output volume of the audio notification 5100 is increased so that the user 500 can hear the ambient noise (as indicated by the bold capital letters, Figure 5S The audio notification 5100 is heard in contrast to the normal text of the audio notification 566-3).

[0227] exist Figure 5AB In the example shown, a subset of response options (e.g., corresponding to the most likely responses) for responding to calendar event invitation 560-3 is also provided to user 500 via headset 502. Specifically, after information about event 560-3 is provided, user 500 is presented with a selection between two response options, "Accept" and "Decline" (e.g., stating "Accept or Decline?").

[0228] After providing the subset of response options, the earphone 502 plays a sound effect 5104 (e.g., white noise) to indicate that the earphone 502 or an electronic device in communication with the earphone 502 is ready to receive a response to the audio notification 5100 and prompt the user 500 to provide a response. Because the ambient noise level is high, the output volume of the sound effect 5104 is also increased so that the user 500 can hear the ambient noise (as indicated by the bold capital letters, Figure 5S The sound effect 5104 is heard by the user 500 in any manner as described herein, such as by voice ( Figure 5AA ), respond to the audio notification 5100. Alternatively, the user 500 may first request to be processed in any manner described herein, such as by providing input via the device 100 (e.g., as described herein with reference to Figure 5T to Figure 5Z described above), listen to the additional response options, and then provide or confirm the selection of the response option.

[0229] Figure 5AC 500 is shown as delaying audio notifications while user 500 is participating in a conversation. At event t0, one or more events are detected (e.g., one or more notifications are received). Additionally, at time t0, it is determined that user 500 is participating in the conversation (e.g., by detecting speech via one or more microphones on earphone 502 or on device 100 that communicates with earphone 502). Thus, while user 500 is participating in the conversation, output of audio notifications corresponding to the detected events is delayed. At time t1, user 500 is no longer participating in the conversation (e.g., earphone 502 or device 100 detects that the speech has ended). Thus, at time t2 after the end of the speech is detected, a delayed audio notification 5106 is provided to user 500 via earphone 502. In some embodiments, as Figure 5AC As shown, audio notification 5106 includes an audio summary of the one or more events (e.g., a summary of the content of the one or more events) detected while user 500 is participating in the conversation (e.g., using simulated voice, such as using text-to-speech conversion). In some embodiments, audio notification 5106 includes a beep or audio tone without providing an audio summary of the one or more events.

[0230] Figure 5AD Suppressing redundant audio notifications when a device includes or communicates with multiple audio output devices is illustrated. Figure 5AD , a map navigation event 5108 corresponding to turn-by-turn navigation is detected. When the earphone 502 is not worn in the ear of the user 500, an audio notification 5110 including verbal guidance is provided via one or more speakers of the device 100. Optionally, in conjunction with the audio notification 5110 being provided, a visual notification 5112 is displayed on the display 112 of the device 100. When the earphone 502 is worn in the ear of the user 500, an audio notification 5114 is provided via the earphone 502, optionally in conjunction with the visual notification 5112 being displayed.

[0231] FIG. 6A to FIG. 6E 6 is a flowchart illustrating a method 600 for providing an audio notification to an audio output device worn by a user according to some embodiments. The method 600 is performed in an electronic device (e.g., Figure 1A The portable multifunction device 100 or Figure 3A The method is performed at a device 300, the electronic device being connected (eg, via a wired or wireless connection) to one or more wearable audio output devices (eg, Figure 3B A wearable audio output device 301, such as an in-ear headset, a circum-ear headset, etc., which is separate from the electronic device, and one or more sensors (e.g., Figure 3B In some embodiments, the sensor is located on a wearable audio output device (e.g., Figure 3B In some embodiments, the sensor is located on the electronic device. In some embodiments, the device includes a display (e.g., Figure 1A Touch screen 112 or Figure 3A display 340) and a touch-sensitive surface (e.g., Figure 1A Touch screen 112 or Figure 3A In some embodiments, the device communicates with or includes one or more additional audio output devices (e.g., one or more speakers, such as Figure 1A The speaker 111 on the device 100 or Figure 3BIn some embodiments, the device includes one or more audio input devices (e.g., Figure 1A Microphone 113 or Figure 3B In some embodiments, the device includes a rotatable input mechanism (e.g., as a Figure 1A In some embodiments, the device includes one or more accelerometers or posture sensors (e.g., a portion of other input controllers 160) for detecting movement and / or posture changes of the device. Figure 1A In some embodiments, a wearable audio output device in communication with the device includes one or more accelerometers or posture sensors (e.g., as sensors for detecting movement and / or posture changes of the wearable audio output device) Figure 3B Some operations in method 600 are optionally combined, and / or the order of some operations is optionally changed.

[0232] As described below, method 600 automatically provides an audio notification of an event that occurs at an associated electronic device when the user is not wearing the wearable audio output device to a user who has worn one or more wearable audio output devices (such as headphones or in-ear headphones). Automatically providing an audio notification when a specific condition (e.g., a specific placement of the wearable audio output device) is met, and no additional user input is required, making it easy for the user to obtain information about missed events and reducing the degree of device use and processing, thereby enhancing the operability of the device, and making the user-device interface more efficient (e.g., by helping the user obtain the desired information and reducing user errors), reducing power usage, and extending the battery life of the device.

[0233] The device detects (602) occurrences of a plurality of events without outputting notifications corresponding to the events via the one or more wearable audio output devices when the one or more wearable audio output devices are not in corresponding positions relative to the user's ears.

[0234] After detecting the occurrence of the plurality of events, the device detects (604) via the one or more (placement) sensors that the one or more wearable audio output devices have been placed in respective positions relative to the user's ears (e.g., in, on, or near the user's ears, as described herein with reference to Figure 5A and Figure 5G to Figure 5H described above).

[0235] In response to detecting, via the one or more (placement) sensors, that the one or more wearable audio output devices have been placed in respective positions relative to the user's ears, the device outputs (606) information, via the one or more wearable audio output devices, the information comprising one or more audio notifications corresponding to one or more events of the plurality of events.

[0236] In some embodiments, the one or more audio notifications include (608) an audio summary of the plurality of events (e.g., Figure 5A Providing an audio summary of an event that occurred while the user was not wearing the wearable audio output device allows the user to more quickly and easily understand missed event information and reduces the need for the user to view visual notifications displayed on the display of the electronic device, which reduces the number of inputs that need to be processed and reduces the amount of time the display needs to be powered on, thereby enhancing the operability of the device, reducing power usage and extending the battery life of the device.

[0237] In some embodiments, based on determining that the length of the audio content representing the plurality of events does not exceed (610) a predefined length, the audio summary includes the audio content; and based on determining that the length of the audio content representing the plurality of events exceeds the predefined length, the audio summary includes less than the entirety of the audio content (e.g., Figure 5H The audio notifications 564 and 566 of the device summarize their corresponding events instead of outputting the entire contents of their associated audio content. When the total length of the audio content (e.g., the predicted total length) will be too long (e.g., longer than a predefined length), reducing the amount of content provided in the audio summary of the event reduces the intrusiveness of the audio summary and more effectively presents the event information while reducing the chance of user distraction. This reduces the need for the user to review the notification one or more times in an auditory or visual manner on the display of the device, which reduces the number of inputs that need to be processed and reduces the amount of time the display needs to be powered on, thereby enhancing the operability of the device, reducing power usage, and extending the battery life of the device.

[0238] In some embodiments, the information includes (612) one or more audio notifications corresponding to one or more events (e.g., received notifications, time-based reminders, distance-based reminders, calendar events, etc.) that are associated with (e.g., occur or are scheduled / predicted to occur during) a corresponding time period (e.g., during a particular day when one or more wearable audio output devices have been placed in a corresponding position relative to a user's ear). In some embodiments, the corresponding time period is a time period that begins at a time in the past (e.g., the beginning of a day or the last time the one or more wearable audio output devices were detected in a corresponding position relative to a user's ear) and extends to a current time (e.g., the time when the one or more wearable audio output devices have been placed in a corresponding position relative to a user's ear). In some embodiments, the corresponding time period includes (e.g., in addition to the aforementioned time periods) a time period that begins at a current time (e.g., the time when the one or more wearable audio output devices have been placed in a corresponding position relative to a user's ear) and extends to a future time (e.g., such as the end of a day). For example, the audio summary 506 ( Figure 5A ) includes audio notifications corresponding to events that were missed before the user 500 put on the headphones 502, as well as audio notifications corresponding to events on a particular day (including upcoming events).

[0239] Providing a summary of events associated with a given period of interest (such as a particular day when the audio output device has been placed in or near the user's ear, or a period before or after the audio output device was placed in the user's ear) allows the user to more quickly and easily understand event information (such as missed events or upcoming events) within the given period of time. This reduces the need for the user to review missed notifications or upcoming events audibly or visually on the device's display, which reduces the number of inputs that need to be processed and reduces the amount of time the display needs to be powered on, thereby enhancing the operability of the device, reducing power usage, and extending the battery life of the device.

[0240] In some embodiments, in response to detecting that the one or more wearable audio output devices have been placed in a corresponding position relative to the user's ear, according to determining that the one or more wearable audio output devices are not previously placed in the corresponding position relative to the user's ear during the corresponding time period (e.g., during the corresponding time period, the one or more wearable audio output devices have not been previously placed in the corresponding position relative to the user's ear, as described herein with reference to Figure 5AIn some embodiments, after detecting that the one or more wearable audio output devices have been placed in the corresponding position relative to the user's ears, the device detects via the one or more sensors that the one or more wearable audio output devices have subsequently been placed in the corresponding position again during the corresponding time period (for example, the one or more wearable audio output devices are placed in the corresponding position again after being removed from the corresponding position). In some embodiments, in response to detecting via the one or more sensors that the one or more wearable audio output devices have subsequently been placed in the corresponding position again during the corresponding time period, the device abandons outputting audio notifications corresponding to the one or more events, which are associated with the corresponding time period. For example, the audio summary 506 is not provided when the earphones 502 are removed and placed in the ears of the user 500 successively on the same day ( Figure 5A ) (although audio notifications of events missed during the intervening time when earphone 502 was not in user 500's ear may still be provided. Providing audio notification of an event only once within a predefined time period reduces redundancy of event information provided to the user, which reduces the intrusiveness of the audio notification and reduces device processing (e.g., by reducing the number of times the audio notification needs to be provided to the user), thereby enhancing the operability of the device and making the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0241] In some embodiments, the information includes (616) one or more audio notifications corresponding to one or more events that are scheduled or predicted to occur within a predefined time period after the one or more wearable audio output devices are placed in a corresponding position relative to the user's ears (e.g., Figure 5A The predefined time period corresponds to the remaining time of a particular day in which the one or more wearable audio output devices have been placed in corresponding positions relative to the user's ears. Automatically providing audio notifications of upcoming events reduces the need for the user to provide input to the device to review event information and reduces the need for the user to visually view event information on a display of the electronic device, which reduces the number of inputs that need to be processed and reduces the amount of time the display needs to be powered on, thereby enhancing the operability of the device, reducing power usage, and extending the battery life of the device.

[0242] In some embodiments, outputting the one or more audio notifications includes (618) outputting a first audio notification corresponding to a first event in the plurality of events. In some embodiments, based on determining that the electronic device is in a first context, the first audio notification includes a first audio content corresponding to the first context; and based on determining that the electronic device is in a second context different from the first context, the first audio notification includes a second audio content corresponding to the second context and different from the first audio content. For example, as described herein with respect to Figure 5D As described, and as further discussed below, for the first geographic location and the geographic area including the first geographic location, the first context and the second context can be different first geographic location and second geographic location, or a first geographic area and a second geographic area. Other examples are described below. The context-dependent presentation of event information in the audio notification provides the user with information that is useful for a given context of the electronic device (and may not be useful in a different context of the device), which strikes a balance between reducing the redundancy of event information provided to the user and providing sufficient information, thereby providing improved feedback to the user, which in turn enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user achieve expected results and reducing user errors when operating the device / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0243] In some embodiments, the first event corresponds to (620) a calendar event (and in some embodiments, has an event time and / or event title) occurring at a first geographic location, wherein the first geographic location includes corresponding location information at multiple levels in a location hierarchy. In some embodiments, determining that the electronic device is in a first context includes determining that the location of the electronic device is within a first geographic area (e.g., within the same building, within the same block, within the same city, etc.) that is the same as the first geographic location, the first geographic area corresponding to the first level in the location hierarchy. In some embodiments, the first audio content includes corresponding location information at a level lower than the first level in the location hierarchy. In some embodiments, the first range is the minimum range surrounding the first geographic location, which also includes the location of the electronic device. In some embodiments, the first audio content includes the event time and / or event title. In some embodiments, the first audio content excludes corresponding location information at the first level or at a level higher than the first level.

[0244] In some embodiments, determining that the electronic device is in the second context includes determining that the location of the electronic device is in a second geographic area that is the same as the first geographic location, wherein the second geographic area is different from the first geographic area and corresponds to a second level different from the first level in the position hierarchy. In some embodiments, the second audio content includes corresponding location information at a level lower than the second level in the position hierarchy. In some embodiments, the second geographic area is the smallest geographic area surrounding the first geographic location, which also includes the location of the electronic device. In some embodiments, in addition to the aforementioned corresponding location information at a level lower than the second level in the position hierarchy, the second audio content also includes event time and / or event title. In some embodiments, the second audio content excludes corresponding location information at the second level or at a level higher than the second level.

[0245] Providing information that helps the user navigate to the destination while the user of the device is within the same specific geographic area as the destination location (such as a meeting location) without providing irrelevant location information about the larger geographic area, striking a balance between reducing redundancy in event information provided to the user and providing sufficient information, thereby providing improved feedback to the user, which in turn enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user achieve intended results and reducing user errors in operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently. Figure 5D Context-dependent presentation of geographic information associated with an event is described in further detail.

[0246] In some embodiments, the first event corresponds to (622) a first message received from the first contact. In some embodiments, determining that the electronic device is in the first context includes determining that a first portion of an identifier (e.g., first name or first name) of the first contact is the same as a first portion of an identifier of at least one other contact in a contact list stored on the electronic device (or, in some embodiments, the user has not communicated with the first contact within a predefined amount of previous time, such as within the past year). In some embodiments, the first audio content includes a first portion of the identifier of the first contact, a second portion of the identifier of the first contact (e.g., last name or last name), and the content of the first message. In some embodiments, determining that the electronic device is in the second context includes determining that the first portion of the identifier of the first contact (e.g., first name or first name) is not the first portion of an identifier of any other contact in the contact list. In some embodiments, the second audio content includes the first portion of the identifier of the first contact and the content of the first message, but does not include the second portion of the identifier.

[0247] In some embodiments, determining that the electronic device is in the first context also includes determining that the user of the electronic device has communicated with the first contact within at least a predefined previous amount of time (e.g., the user has sent a message to the first contact or received a message from the first contact within the past year). In some embodiments, the first audio content excludes a second portion of an identifier (e.g., last name or surname) of the first contact, wherein other contacts in the user's contact list do not have the same first portion of the identifier (e.g., first name or given name). When the identity of the sender of the message can be inferred from specific identification information about the sender and from the context (e.g., because the user recently communicated with the sender, and / or because the sender has a unique name in the user's address book or another form of contact list or other source of contact information applicable to the user), only specific information about the sender is provided without irrelevant identification information, striking a balance between reducing the redundancy of event information provided to the user and providing sufficient information, thereby reducing device processing, which in turn enhances the operability of the device, reduces power usage and extends the battery life of the device. This article refers to at least FIG. 5B to FIG. 5C The inclusion / omission of portions of a contact name depending on the context is described in more detail.

[0248] In some embodiments or in some cases, the first event corresponds to (624) a first message from the first contact (e.g., the device determines that the first event corresponds to a corresponding message from the first contact). In some embodiments, determining that the electronic device is in the first context includes determining that the first event occurs discontinuously after the second event in the multiple events, and the second event corresponds to a second message from the first contact (e.g., a second message transmitted or received before the first message). In some embodiments, the first audio content includes an identifier of the first contact and the content of the first message. In some embodiments or in some cases, because (e.g., because the device determines): the first event is an initial event in the multiple events, the first event occurs discontinuously after the second event in the multiple events, and the second event corresponds to a second message from the first contact; the first event occurs continuously after the second event, but the second event does not correspond to the second message; or the first event and the second (e.g., previous) event both correspond to messages from the first contact, but one or more intervening events are detected between the first event and the second event. In some embodiments, the identifier of the corresponding contact includes a first part and optionally a second part, as described herein with reference to operation 622.

[0249] In some embodiments, determining that the electronic device is in the second context includes determining that a first event occurs consecutively after a second event in the multiple events (e.g., no intervening event is detected), the second event corresponding to a second message from the first contact (e.g., a second message transmitted or received before the first message). In some embodiments, the second audio content includes the content of the first message and does not include an identifier of the first contact. When the identity of the sender of a given message can be inferred from identification information provided for a previous message immediately preceding the given message (e.g., no intervening event occurs), omitting identification information of the sender of the given message reduces redundancy of event information provided to the user and reduces device processing, thereby enhancing the operability of the device, reducing power usage, and extending the battery life of the device. This article refers to at least FIG. 5B to FIG. 5C The inclusion / omission of portions of a contact name depending on the context is described in more detail.

[0250] In some embodiments or in some cases, (e.g., the device determines) that the first event corresponds to (626) a first message that is part of a group message conversation of the corresponding group. Figure 5C An example of a context determined based at least in part on a group message is described. In some embodiments, determining that the electronic device is in a first context includes determining that a first event in the multiple events occurs discontinuously after a second event, the second event corresponding to a second message that is part of a group message session (e.g., a second message that is transmitted or received before the first message and is part of a group message session). In some embodiments, the first audio content includes an identifier of the corresponding group or group message session and the content of the first message. In some embodiments, because (e.g., because the device determines): the first event in the multiple events is an initial event, the first event in the multiple events occurs discontinuously after a second event, the second event corresponds to a second message that is part of a group message session; the first event occurs continuously after the second event, but the second event does not correspond to a message that is part of a group message session; or the first event and the second (e.g., previous) event both correspond to messages that are part of a group message session but one or more intervening events are detected between the first event and the second event.

[0251] In some embodiments, the identifier of the group messaging session includes the name of the group messaging session. In some embodiments, the identifier of the group messaging session includes identifiers of one or more (or all) participants in the group messaging session. In some embodiments, the first message is from a corresponding contact, and the first audio content includes an identifier of the first contact. In some embodiments, the identifier of the corresponding contact includes a first part and optionally a second part, as described herein with reference to operation 622.

[0252] In some embodiments, determining that the electronic device is in the second context includes determining that a first event occurs consecutively after a second event in the multiple events (e.g., no intervening event is detected), and the second message corresponds to a second message that is part of a group messaging session (e.g., a second message that is transmitted or received before the first message and is part of a group messaging session). In some embodiments, the second audio content includes the content of the first message and does not include an identifier of the corresponding group or group messaging session. In some embodiments, the first message is from a first contact and the second message is from a second contact that is different from the first contact, so the first audio content includes an identifier of the first contact (e.g., including a first part and optionally a second part), as described herein with reference to operation 622. In some embodiments, the first message and the second message are from the same corresponding contact, so the first audio content does not include an identifier of the first contact, as described herein with reference to operation 622.

[0253] When a given message is received as part of a group messaging conversation, and the identity of the group can be inferred from the identification information provided for the previous message immediately preceding the given message (e.g., no intervening events have occurred), omitting the identification information of the group for the given message reduces redundancy of event information provided to the user and reduces device processing, thereby enhancing device operability, reducing power usage, and extending device battery life. FIG. 5B to FIG. 5C The context-dependent inclusion / omission of group identifiers is described in more detail.

[0254] In some embodiments, outputting one or more audio notifications includes (628) outputting a first corresponding audio notification corresponding to a first corresponding event in the multiple events. In some embodiments, the first corresponding event corresponds to a first application associated with a first notification category, and the first corresponding audio notification has a first structure defined by the first notification category. In some embodiments, the application specifies a corresponding notification category, and events associated with the application are associated with the corresponding notification category. In some embodiments, the corresponding structure associated with the corresponding notification category is defined by an application-independent module (e.g., by an operating system of an electronic device or by an application programming interface). In some embodiments, the first structure specifies a first set of audio content included in the corresponding audio notification. In some embodiments, based on determining that the second application associated with the second corresponding event is associated with a second notification category different from the first notification category, the second corresponding audio notification has a different second structure defined by the second notification category (e.g., a second set of audio content specified to be included in the second corresponding audio notification).

[0255] For example, the audio notifications 538 and 548 of calendar event invitations 530 and 540 can have a notification structure defined by a calendar event notification category, which defines the notification structure for calendar type events. Outputting audio notifications for events using a structure defined by a notification category associated with an event provides consistency in how event information is presented, making it easier for a user to understand the event information. In addition, using a normalized notification structure for audio notifications associated with a given notification category reduces the calculations required to determine which information to include in the audio notification and to output the audio notification, which enhances the operability of the device, reduces power usage, and extends the battery life of the device.

[0256] In some embodiments, outputting one or more audio notifications includes (630) outputting a second corresponding audio notification corresponding to a second corresponding event in the multiple events. In some embodiments, the second corresponding event corresponds to a second application, and the second corresponding audio notification has a second structure defined by the second application. In some embodiments, the second structure specifies a second set of audio content included in the second corresponding audio notification. In some embodiments, based on determining that a third corresponding event in the multiple events is associated with a third application different from the second application, the third corresponding audio notification has a different third structure defined by the third application (e.g., a third set of audio content specified to be included in the third corresponding audio notification).

[0257] For example, audio notifications 538 and 548 for calendar event invitations 530 and 540 can have a notification structure defined by the calendar application associated with the event. If the event is associated with different calendar applications, the event can have different notification structures defined separately by each calendar application. Outputting audio notifications for events using structures defined or specified by the applications associated with the event provides greater flexibility for each application in deciding which information to present in its associated audio notification, and takes into account application developer preferences and determines which information in the audio notification will be useful to the user. In addition, using an application-specified structure for audio notifications for a given application reduces the computation required to determine which information to include in the audio notification and to output the audio notification, which enhances device operability, reduces power usage, and extends the battery life of the device.

[0258] It should be understood that FIG. 6A to FIG. 6EThe specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein in conjunction with one or more other methods described herein (e.g., methods 700, 800, 900, and / or 1000) are also applicable in a similar manner to the details of the processes described above in conjunction with FIG. 6A to FIG. 6E Method 600 is described. For example, the device or components thereof, events, audio notifications, and inputs described above with reference to method 600 optionally have one or more of the features of the device or components thereof, events, audio notifications, and inputs described herein with reference to one or more of the other methods described herein (e.g., methods 700, 800, 900, and / or 1000). For the sake of brevity, these details are not repeated here.

[0259] 7A to 7C is a flowchart illustrating a method 700 of providing audio context to convey additional information about an event according to some embodiments. The method 700 is performed in an electronic device (e.g., Figure 1A The portable multifunction device 100 or Figure 3A The device 300 is executed at a device such as a phone, a computer, a headset, or one or more earbuds, the electronic device being connected to one or more audio output devices (e.g., one or more speakers such as Figure 1A The speaker 111 on the device 100 or Figure 3B In some embodiments, the device is one or more wearable audio output devices (e.g., speakers 306 on wearable audio output device 301) that are separate from the electronic device. Figure 3B In some embodiments, the device includes one or more sensors (e.g., Figure 3B In some embodiments, the device includes a display (e.g., Figure 1A Touch screen 112 or Figure 3A display 340) and / or a touch-sensitive surface (e.g., Figure 1A Touch screen 112 or Figure 3A In some embodiments, the device includes one or more audio input devices (e.g., Figure 1A Microphone 113 or Figure 3B In some embodiments, the device includes a rotatable input mechanism (e.g., as a Figure 1AIn some embodiments, the device includes one or more accelerometers or posture sensors (e.g., a portion of other input controllers 160) for detecting movement and / or posture changes of the device. Figure 1A In some embodiments where the audio output device is separate from and in communication with the device, the audio output device includes one or more accelerometers or posture sensors (e.g., as sensors) for detecting movement and / or posture changes of the wearable audio output device. Figure 3B Some operations in method 700 are optionally combined, and / or the order of some operations is optionally changed.

[0260] As described below, method 700 changes the audio output properties associated with different types of audio notifications (e.g., changing spatial position, volume, content, etc.), which are associated with events with different characteristics (e.g., changing sender / initiator, value, etc.). It should be noted that spatial position (sometimes referred to as simulated spatial position) is a perceptual property of audio output. Known audio synthesis techniques can be used to control or change spatial position so that the audio output is perceived as coming from a specific spatial position in three-dimensional space, which is different from the physical position of the speaker that produces the audio output. Generally speaking, at least two speakers are required to change the spatial position of the audio output. Changing the audio output properties of the audio notification based on the characteristics of the associated event helps the user distinguish between different types of events and different event information. Providing improved feedback enhances the operability of the device, and makes the user-device interface more effective (e.g., by helping the user achieve the desired result and reducing user errors when operating the device / interacting with the device), which in turn reduces power use and extends the battery life of the device by enabling the user to use the device more quickly and effectively.

[0261] The device detects (702) the occurrence of an event (eg, a request to play a recent notification, an incoming message, etc.).

[0262] In response to detecting the occurrence of the event, the device outputs (704) an audio notification via the one or more audio output devices, the audio notification including audio content representative of the event (e.g., the audio content includes content of a received message, an indication of a transaction between users, etc.).

[0263] Based on determining that the event is associated with a first feature in a first set of features (706) (e.g., the event is sent or initiated by a first user among a plurality of users; the event is associated with a first value in a value range (e.g., a first transaction value); etc.), the audio notification includes audio content representing the event and has a first audio output attribute selected based on the first feature (e.g., a first spatial position, a first magnitude, a first sound effect, etc.). Based on determining that the event is associated with a second feature in the first set of features that is different from the first feature (e.g., a second user among the plurality of users; a second value in the value range), the audio notification includes audio content representing the event and has a second audio output attribute selected based on the second feature that is different from the first output attribute (e.g., a second spatial position, a second magnitude, a second sound effect, etc.). For example, as described herein with reference to Figure 5C As described, audio notifications of messages from different senders may be associated with and output at different spatial locations to help users distinguish between messages.

[0264] In some embodiments, determining that an event is associated with a first feature in a first set of features includes (708) determining that the event was initiated by a first user in a plurality of users; and determining that the event is associated with a second feature in the first set of features includes determining that the event was initiated by a second user in the plurality of users. In some embodiments, the first audio output attribute includes a first spatial position associated with the first user. In some embodiments, the first spatial position is independent of the physical position of the first user (e.g., the first spatial position is selected without regard to the physical position of the first user). For example, the first spatial position can be a position related to the position of a pair of headphones 502, such as Figure 5C In some embodiments, the second audio output attribute includes a second spatial location that is different from the first location and is associated with the second user. In some embodiments, the second spatial location is independent of the second user's physical location (e.g., the second spatial location is selected without regard to the second user's physical location). In some embodiments, the spatial relationship between the first spatial location and the second spatial location is independent of the spatial relationship between the first user's physical location and the second user's physical location. For example, as described herein with reference to Figure 5C As described above, audio notifications of messages from different participants in a group message may be associated with and output at different spatial locations relative to the location of a pair of headphones 502, such as Figure 5C shown to help users distinguish messages.

[0265] Associating events initiated by different users (e.g., different senders of received messages) with different spatial locations (e.g., different simulated spatial locations) and outputting corresponding audio notifications of the events at different spatial locations helps users (e.g., recipient users) identify and distinguish the originators of audio notifications. In addition, in some cases, using different spatial locations for different event initiators / senders reduces the amount of information that needs to be provided to the user verbally, visually, or otherwise to help the user identify the sender of a particular event or message (e.g., one or more names of the sender may be omitted). These beneficial effects provide improved feedback to the user and reduce device processing (e.g., by reducing the amount of information that needs to be provided to the user), thereby enhancing the operability of the device, and making the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0266] In some embodiments, determining that the event is associated with a first feature in the first set of features includes (710) determining that the event was initiated by a first user in a plurality of users; and determining that the event is associated with a second feature in the first set of features includes determining that the event was initiated by a second user in the plurality of users. In some embodiments, the first audio output attribute includes a first audio tone associated with the first user; and the second audio output attribute includes a second audio tone that is different from the first audio tone and is associated with the second user. References herein Figure 5C Described is the audio notification from different senders being distinguished by changing the audio tone used to output the audio notification. Different audio tones are associated with events initiated by different users (e.g., different senders of received messages) and these different audio tones are combined into the corresponding audio notifications of the events, which helps the user to identify and distinguish the originator of the audio notification. In addition, in some cases, the use of different audio tones for different event initiators / senders reduces the amount of information that needs to be provided to the user verbally, visually or otherwise to help the user identify the sender of a particular event or message (e.g., one or more names of the sender can be omitted). These beneficial effects provide improved feedback to the user and reduce device processing (e.g., by reducing the amount of information that needs to be provided to the user), thereby enhancing the operability of the device, and making the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating the device / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0267] In some embodiments, determining that an event is associated with a first feature in a first set of features includes (712) determining that the event was initiated by a first user among a plurality of users; and determining that the event is associated with a second feature in the first set of features includes determining that the event was initiated by a second user among the plurality of users. In some embodiments, outputting an audio notification including audio content representing an event and having a first audio output attribute includes outputting an audio notification including audio content representing an event using a first simulated voice associated with the first user (e.g., without regard to whether the detected event includes a voice communication from the first user). In some embodiments, outputting an audio notification including audio content representing an event and having a second audio output attribute includes outputting an audio notification including audio content representing an event using a second simulated voice that is different from the first simulated voice and is associated with a second user (e.g., without regard to whether the detected event includes a voice communication from the second user). References herein Figure 5C Differentiating audio notifications from different senders by varying the simulated voice used to output the audio notifications is described.

[0268] Associating different simulated voices with events initiated by different users (e.g., different senders of received messages) and using different simulated voices to output corresponding audio notifications helps users identify and distinguish the originators of audio notifications. In addition, in some cases, using different simulated voices for different event initiators / senders reduces the amount of information that needs to be provided to the user verbally, visually, or otherwise to help the user identify the sender of a particular event or message (e.g., one or more names of the sender may be omitted). These beneficial effects provide improved feedback to the user and reduce device processing (e.g., by reducing the amount of information that needs to be provided to the user), thereby enhancing the operability of the device, and making the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0269] In some embodiments, determining that an event is associated with a first feature in the first set of features includes (714) determining that the event is associated with a first transaction value (e.g., of a transaction between a user of the device and one or more other users or devices); and determining that the event is associated with a second feature in the first set of features includes determining that the event is associated with a second transaction value that is different from the first transaction value. In some embodiments, the first audio output attribute includes a first sound effect representing the first transaction value; and the second audio output attribute includes a second sound effect that is different from the first sound effect and represents the second transaction value. In some embodiments, the first sound effect includes a corresponding sound effect played at a first volume representing the first transaction value, and the second sound effect includes the same corresponding sound effect played at a second volume representing the second transaction value. For example, a sound effect played at a higher volume represents a larger transaction value, while the same sound effect played at a lower volume represents a smaller transaction value. In some embodiments, the second sound effect includes a sound effect of a different type than the first sound effect. Reference herein Figure 5E Audio notifications for transactions of different values ​​are described that are differentiated by changing the sound effects and / or audio volume of the corresponding audio notifications.

[0270] Associating different sound effects and / or different volumes with events associated with different transaction values ​​and outputting corresponding audio notifications using different sound effects and / or different volumes helps the user identify and distinguish the value (or relative value) of the received transaction. In addition, in some cases, the amount of information that needs to be provided to the user verbally, visually, or otherwise to help the user identify the value of the transaction is reduced (e.g., the statement of the value of the transaction may be omitted). These benefits provide improved feedback to the user and reduce device processing (e.g., by reducing the amount of information that needs to be provided to the user), thereby enhancing the operability of the device, and making the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0271] In some embodiments, determining that the event is associated with a first feature in the first set of features includes (716) determining that the event is a first type of notification (e.g., a first type of message, or a notification associated with a first application or a first group of applications); and determining that the event is associated with a second feature in the first set of features includes determining that the event is a second type of notification different from the first type of notification (e.g., a second type of message, or a notification associated with a second application or a second group of applications). In some embodiments, the first audio output attribute includes a first background audio associated with the first type of notification; and the second audio output attribute includes a second background audio different from the first background audio and associated with the second type of notification. Associating different background noises with different types of notifications and combining these different background noises into corresponding audio notifications helps users recognize and distinguish between types of notifications. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0272] In some embodiments, determining that an event is associated with a first feature in a first set of features includes (718) determining that the event is associated with a first type of message (e.g., a message between three or more users, sometimes referred to as a "group message"); and determining that the event is associated with a second feature in the first set of features includes determining that the event is associated with a second type of message that is different from the first type of message (e.g., a message between two users, sometimes referred to as a "private chat message"). In some embodiments, the first audio output attribute includes a first spatial position (e.g., a first simulated spatial position) associated with the first type of message; and the second audio output attribute includes a second spatial position (e.g., a second simulated spatial position) that is different from the first position and associated with the second type of message. References herein Figure 5C Differentiating between message types (e.g., private chat messages and group messages) by outputting corresponding audio notifications at different spatial locations is described. Associating different types of messages with different spatial locations (e.g., different simulated spatial locations) and outputting corresponding audio notifications at different spatial locations helps users identify and distinguish between message types. Providing improved feedback to users enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping users achieve desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling users to use the device more quickly and efficiently.

[0273] In some embodiments, the first type of message corresponds to a message between a user of the electronic device and at least two other users (e.g., a "group message"); and the second type of message corresponds to a message between a user of the electronic device and at most one other user (e.g., a "private chat message").

[0274] In some embodiments, the first spatial location is (722) a simulated spatial location outside the user's head, and the second spatial location is a simulated spatial location inside the user's head. In some embodiments, outputting the audio notification at the simulated spatial location outside the user's head includes outputting associated audio content of the audio notification so that the audio content sounds as if it is coming from outside the user's head (or at a location at least a predefined distance from the user's head or ears) or is being played outside the user's head. In some embodiments, outputting the audio notification at the simulated spatial location inside the user's head includes outputting associated audio content of the audio notification so that the audio content sounds as if it is being played inside the user's head.

[0275] References Figure 5C Distinguishing private messages from group messages by outputting corresponding audio notifications at simulated spatial locations "inside" and "outside" the user's head, respectively, is described. Presenting private messages at simulated locations so as to sound as if they are being played "inside" the user's head, while presenting group messages to sound as if they are being played "outside" the user's head, helps the user distinguish between different types of messages and focus more on messages specific to the user, and reduces distractions from messages in a group messaging conversation that may not be specific to the user. Providing improved feedback to the user enhances the operability of the device and makes the user-device interface more effective (e.g., by helping the user achieve desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0276] In some embodiments where the electronic device includes a display and a touch-sensitive surface, the device displays (724) a user interface on the display. The user interface includes: a representation of one or more audio output devices; a first user interface element that represents a first type of message and is displayed at a first position in the user interface, wherein the first position in the user interface defines a first spatial position associated with the first type of message; and a second user interface element that represents a second type of message and is displayed at a second position in the user interface, wherein the second position in the user interface defines a second spatial position associated with the second type of message. In some embodiments, the device receives input corresponding to the first user interface element on the touch-sensitive surface (e.g., a drag input that includes a contact at a position on the touch-sensitive surface corresponding to the first user interface element and subsequent movement of the contact along the touch-sensitive surface). In some embodiments, in response to receiving the input, the device: moves the first user interface element to a third position in the user interface; and associates the first type of message with a third spatial position defined by the third position in the user interface (and in some embodiments, disconnects the first type of message from the first spatial position defined by the first position in the user interface).

[0277] In some embodiments, after associating the first type of message with the third spatial location, outputting an audio notification including audio content representing an event associated with the first type of message includes outputting the audio content so that the audio content sounds as if it comes from the third spatial location. In some embodiments, more generally, a user interface is provided that displays a corresponding user interface element for each event feature in a corresponding group of features, wherein each user interface element in a plurality of user interface elements is associated with a corresponding audio output attribute. In some embodiments, a user may provide input pointing to a corresponding user interface element on the user interface to change the corresponding audio output attribute associated with the corresponding event feature. For example, a user may change the spatial location, sound volume, and / or sound effect type associated with a particular event feature (e.g., initiating user, transaction value, message type, etc.). (Unless otherwise specified, the "spatial location" discussed herein is a simulated spatial location that corresponds to the audio output attribute of the corresponding audio notification.)

[0278] Providing a user with a user interface that the user can use to add or modify the spatial locations associated with different types of messages provides additional control options, and allows the user to customize the audio experience to be more familiar and recognizable, and reduces the amount of information that would otherwise need to be provided to the user to help the user understand the information presented in the audio notification. These benefits provide improved feedback to the user and reduce device processing (e.g., by reducing the amount of information that needs to be provided to the user), thereby enhancing the operability of the device, and making the user-device interface more efficient (e.g., by helping the user achieve desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0279] It should be understood that 7A to 7C The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein in conjunction with one or more other methods described herein (e.g., methods 600, 800, 900, and / or 1000) are also applicable in a similar manner to the details of the processes described above in conjunction with 7A to 7C Method 700 is described. For example, the devices or components thereof, events, audio notifications, and inputs described above with reference to method 700 optionally have one or more of the features of the devices or components thereof, events, audio notifications, and inputs described herein with reference to one or more of the other methods described herein (e.g., methods 600, 800, 900, and / or 1000). For the sake of brevity, these details are not repeated here.

[0280] FIG. 8A to FIG. 8D 800 is a flow chart illustrating a method 800 for performing an operation based on the timing of a user input related to an audio notification according to some embodiments. The method 800 is performed in an electronic device (e.g., Figure 1A The portable multifunction device 100 or Figure 3A The method is performed at a device 300, the electronic device being connected (eg, via a wired or wireless connection) to one or more wearable audio output devices (eg, Figure 3B A wearable audio output device 301, such as an in-ear headset, a circum-ear headset, etc., which is separate from the electronic device, and one or more sensors (e.g., Figure 3B In some embodiments, the sensor is located on a wearable audio output device (e.g., Figure 3BIn some embodiments, the sensor is located on the electronic device. In some embodiments, the device includes a display (e.g., Figure 1A Touch screen 112 or Figure 3A display 340) and a touch-sensitive surface (e.g., Figure 1A Touch screen 112 or Figure 3A In some embodiments, the device communicates with or includes one or more audio output devices (e.g., one or more speakers, such as Figure 1A The speaker 111 on the device 100 or Figure 3B In some embodiments, the device includes one or more audio input devices (e.g., Figure 1A Microphone 113 or Figure 3B In some embodiments, the device includes a rotatable input mechanism (e.g., as a Figure 1A In some embodiments, the device includes one or more accelerometers or posture sensors (e.g., a portion of other input controllers 160) for detecting movement and / or posture changes of the device. Figure 1A In some embodiments, a wearable audio output device in communication with the device includes one or more accelerometers or posture sensors (e.g., as sensors for detecting movement and / or posture changes of the wearable audio output device) Figure 3B Some operations in method 800 are optionally combined, and / or the order of some operations is optionally changed.

[0281] As described below, method 800 performs different operations in response to input directed to one or more wearable audio output devices based on whether the input is detected within or after a predefined time period, the predefined time period being related to one or more audio notifications output in response to detecting an event. In addition, performing operations associated with audio notifications in response to detecting input within a predefined time period related to the audio notification enables a user to quickly respond to or otherwise interact with the audio notification or corresponding event. Performing different operations based on the timing of input directed to the wearable audio output device and enabling the user to quickly respond to the notification provides the user with additional control options without requiring additional audio or visual output to notify the user of the additional control options, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0282] The device detects (802) the occurrence of a first event.

[0283] After detecting the occurrence of the first event, the device outputs (804) via the one or more wearable audio output devices one or more audio notifications corresponding to the first event.

[0284] After initiating output of the one or more audio notifications, the device detects (806) input directed to the one or more wearable audio output devices (e.g., speech input, touch input, device movement input such as accelerometer movement).

[0285] In response to detecting input directed to the one or more wearable audio output devices (808): based on determining that the input was detected within a predefined time period, the device performs a first operation, the predefined time period is related to the one or more audio notifications corresponding to the first event, wherein the first operation is associated with the one or more audio notifications corresponding to the first event; and based on determining that the input was detected after the predefined time period has passed, the device performs a second operation, the predefined time period is related to the one or more audio notifications corresponding to the first event, wherein the second operation is not associated with the one or more audio notifications corresponding to the first event. For example, a tap input 554 ( Fig. 5F ) causes an operation (e.g., playing music) to be performed that is associated with a tap input 572 ( Figure 5L ) causes different actions (e.g., listening to more information).

[0286] In some embodiments, the device detects (810) a second input directed to the one or more wearable audio output devices. In some embodiments, in response to detecting the second input directed to the one or more wearable audio output devices, based on determining that the second input was detected before the one or more audio notifications corresponding to the first event began to be output and based on determining that a predefined time period has passed, the device performs a second operation, the predefined time period being related to one or more audio notifications corresponding to the second event (e.g., an event before the first event) and output before the one or more audio notifications corresponding to the first event, wherein the second operation is not associated with the one or more audio notifications corresponding to the second event.

[0287] For example, a tap input 554 ( Fig. 5F ) will also cause the same operation as tap input 554 (e.g., play music) to be performed. Performing an operation other than the operation related to a specific audio notification in response to an input directed to the wearable audio output device outside of the predefined time period related to the specific audio notification provides the user with additional control options in addition to notification control using the same input device (e.g., for interacting with other functions or applications of the electronic device), which enhances the operability of the device and makes the user-device interface more efficient (e.g., by enabling the user to achieve a greater number of intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0288] In some embodiments, after detecting the occurrence of the first event, the device detects (812) the occurrence of a third event. In some embodiments, after detecting the occurrence of the third event, and after outputting the one or more notifications corresponding to the first event, the device outputs one or more audio notifications corresponding to the third event via the one or more wearable audio output devices. In some embodiments, in response to detecting an input directed to the one or more wearable audio output devices, and based on determining that the input is detected after starting to output the one or more audio notifications corresponding to the third event and within a predefined time period, the device performs a first operation, the predefined time period being related to the one or more audio notifications corresponding to the third event, wherein the first operation is associated with the one or more audio notifications corresponding to the third event.

[0289] For example, movement of the head of user 500 detected within a predefined time period of its corresponding audio notification (e.g., audio notifications 566 and 570, respectively) Fig.5I ) and tap Enter 572( Figure 5L ) both cause the same action to be performed for each of their corresponding audio notifications (e.g., listening to more information). Performing an action associated with a particular audio notification in response to input directed to the wearable audio output device within a predefined time period associated with the particular audio notification (e.g., including while the particular notification is being output) provides the user with additional control options to respond to or interact with the particular audio notification using the same input device, even if the audio notification is one of a sequence of audio notifications being provided. Providing the user with improved control options enhances the operability of the device and makes the user-device interface more efficient (e.g., by enabling the user to achieve a greater number of intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0290] In some embodiments, outputting the one or more audio notifications corresponding to the respective event includes (814) outputting a portion of the audio content corresponding to the respective event; and performing a first operation associated with the one or more audio notifications corresponding to the respective event includes abandoning outputting a remaining portion of the one or more audio notifications corresponding to the respective event. Figure 5N to Figure 5P Skipping the remainder of an audio notification is described. Enabling a user to skip the remainder of the one or more audio notifications after listening to the first portion reduces the likelihood of presenting unnecessary or undesirable information to the user, and may also reduce the amount of time that the audio output device and / or the electronic device needs to be activated, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0291] In some embodiments, outputting the one or more audio notifications corresponding to the corresponding event includes (816) outputting a first portion of audio content corresponding to the corresponding event; and performing a first operation associated with the one or more audio notifications corresponding to the corresponding event includes outputting a second (e.g., additional) portion of the audio content corresponding to the corresponding event, the second portion being different from the first portion. In some embodiments, the first portion of the audio content corresponds to an abbreviated form of the audio notification corresponding to the corresponding event, and the second portion of the audio content corresponds to an expanded form of the audio notification, the expanded form including audio content excluded from the abbreviated audio notification (such as the summarized audio notification, compressed audio notification, or truncated audio notification described in reference method 600 herein). Reference herein FIG. 5I to FIG. 5J and Figure 5L to Figure 5M Described is playing an expanded form of an audio notification originally presented in compressed form in response to user input directed to a wearable audio device during or “shortly after” (e.g., within a predefined amount of time relative to the audio notification) playing a compressed audio notification.

[0292] Enabling a user to listen to an extended form of audio notification in response to a user input directed to a wearable audio output device enables the user to quickly obtain additional information about a detected event without requiring the user to view the information on a display of the electronic device. Reducing the number of inputs required to listen to more information and reducing the amount of time the display needs to be powered on enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve desired results more quickly and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0293] In some embodiments, performing a second operation not associated with the one or more audio notifications corresponding to the corresponding event includes (818) invoking a virtual assistant or controlling playback of media content (e.g., playing, pausing, resuming, stopping, rewinding, or fast forwarding playback of media content different from the one or more audio notifications corresponding to the corresponding event), as described herein with reference to Fig. 5F and Figure 5Q to Figure 5RThe performing of operations other than operations related to a specific audio notification in response to input detected outside of a predefined time period related to the specific audio notification provides the user with additional control options beyond notification control using the same input device (e.g., for interacting with other functions or applications of the electronic device), which enhances the operability of the device and makes the user-device interface more efficient (e.g., by enabling the user to achieve a greater number of intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0294] In some embodiments, the first operation associated with the one or more audio notifications corresponding to the respective event is based (820) on an application associated with the respective event (e.g., specified in application settings for the application). The operation performed based on the audio notification on the application associated with the event corresponding to the audio notification enables a user to provide a relevant response to the event without requiring the user to view the event on a display of the electronic device. This provides the user with additional control options and reduces the amount of time the display needs to be powered on, which enhances the operability of the device and makes the user-viewer more efficient (e.g., by helping the user achieve a greater number of desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0295] In some embodiments, detecting the corresponding event includes (822) receiving media content other than text content, and performing the first operation associated with the one or more audio notifications corresponding to the corresponding event includes providing a verbal description of the media content other than the text content. For example, in response to receiving an image, the device provides one or more audio notifications including a verbal description of the image, as described herein with reference to Figure 5M Providing a verbal description of the content of non-text content (such as image content) provides the user with information about the additional type of content without requiring the user to view the content, such as on a display of the electronic device, which is beneficial for visually impaired users. Providing improved audio feedback to the user and reducing the amount of time the display needs to be powered on enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve desired results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0296] In some embodiments, based on determining that an input is detected while outputting a first audio notification of the one or more audio notifications, a first operation related to the first audio notification is performed (824). In some embodiments, the input is detected after outputting a first portion of the one or more audio notifications, and in response, the device abandons outputting (e.g., skips) a remaining portion of the one or more audio notifications (e.g., as described herein with reference to operation 814). In some embodiments, based on determining that an input is detected while outputting a second audio notification of the one or more audio notifications that is different from the first audio notification, a first operation related to the second audio notification is performed.

[0297] For example, if movement of the user 500's head is detected when the corresponding audio notification 566 is output ( Fig.5I ), then trigger (e.g., cause) execution of an operation associated with the audio notification 566, and, if a tap input 572 ( Figure 5L ), then trigger (e.g., cause) the performance of an operation related to the audio notification 570. Performing an operation related to a particular audio notification in response to input directed to the wearable audio output device during that particular audio notification enables the user to provide a response or otherwise interact with the particular audio notification, even if the audio notification is one of a sequence of audio notifications being output. Providing improved control options to the user enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0298] In some embodiments, based on determining that the input is a first type of input (e.g., a single click), the first operation is (826) a first type of operation (e.g., a reply operation or a "play more" operation); and based on determining that the input is a second type of input (e.g., a double click) different from the first type of input (e.g., a single click), the first operation is a second type of operation different from the first type of operation (e.g., a "skip" operation). Figure 5L to Figure 5M and Figure 5N to Figure 5PPerforming different types of operations in response to different types of input received during or within a predefined time period associated with an audio notification is described. Performing different types of operations associated with a particular audio notification based on the type of input directed to a wearable audio output device within a predefined time period associated with the particular notification (e.g., including when the particular notification is output) provides the user with additional control options for responding to or interacting with the particular notification using the same input device, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by enabling the user to achieve a greater number of intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0299] In some embodiments, based on determining that the input is (828) a first type of input (e.g., a single click), the second operation is a third type of operation (e.g., a "play / pause" operation); and based on determining that the input is a second type of input (e.g., a double click) that is different from the first type of input (e.g., a single click), the second operation is a fourth type of operation that is different from the third type of operation (e.g., a "call virtual assistant" operation). Fig. 5F and Figure 5Q to Figure 5R Performing different types of operations in response to different types of input received outside of a predefined time period associated with any audio notification is described. Performing different types of operations in addition to operations associated with a particular audio notification based on the type of input directed to a wearable audio output device outside of a predefined time period associated with the particular audio notification provides the user with additional control options beyond the notification controls (e.g., for interacting with other functions or applications of the electronic device), which enhances the operability of the device and makes the user-device interface more efficient (e.g., by enabling the user to achieve a greater number of intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0300] In some embodiments, detecting input directed to the one or more wearable audio output devices includes (830) detecting a tap input (e.g., a first tap input) on the one or more wearable audio output devices. In some embodiments, detecting the input includes detecting a double-tap input (e.g., a second tap input within a predefined time period related to the first tap input). Inputting and performing operations in response to one or more taps directed to the wearable audio output device provides the user with control options using intuitive user interactions, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve desired results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0301] In some embodiments where the one or more wearable audio output devices include one or more accelerometers, detecting input directed toward the one or more wearable audio output devices includes (832) detecting movement of the one or more wearable audio output devices via the one or more accelerometers. FIG. 5I to FIG. 5K Performing operations in response to accelerometer inputs is described. Performing operations in response to accelerometer inputs provided via movement of a wearable audio output device provides the user with control options using intuitive user interactions (e.g., movement of the user's head when the wearable audio output device is worn), which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0302] In some embodiments, outputting the one or more audio notifications corresponding to the corresponding event includes (834) outputting a first portion of the audio content corresponding to the corresponding event. In some embodiments, based on detecting movement of the one or more wearable audio output devices in a first direction (e.g., moving backward or tilting, or away from a simulated spatial location where the one or more audio notifications are output) via the one or more accelerometers, performing the first operation includes abandoning outputting a second portion of the audio content corresponding to the corresponding event. In some embodiments, based on detecting movement of the one or more wearable audio output devices in a second direction opposite to the first direction (e.g., moving forward or tilting, or toward a simulated spatial location where the one or more audio notifications are output) via the one or more accelerometers, performing the first operation includes outputting a second portion of the audio content corresponding to the corresponding event. In some embodiments, the second portion of the audio content includes additional audio content relative to the audio content, and if input directed to the one or more wearable audio devices does not occur, the additional audio content will be output. References herein FIG. 5I to FIG. 5K Providing different amounts of audio content in response to movement of a user's head toward or away from a simulated spatial location of a sound is described.

[0303] Changing the amount of audio content provided based on the direction of movement of the wearable audio output device provides the user with control options for listening to additional information using intuitive user interactions (e.g., providing additional audio content when the user is tilted in a particular direction, such as toward a simulated location / audio notification source, and providing less audio content when the user is tilted in a different direction, such as away from a simulated location / audio notification source), which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0304] In some embodiments, after detecting the occurrence of the corresponding event, the device abandons (836) outputting one or more notifications (e.g., audio, visual, and / or tactile notifications) corresponding to the corresponding event other than the one or more audio notifications output via the one or more wearable audio output devices. In some embodiments, the device includes or communicates with one or more audio output devices other than the one or more wearable audio output devices, and the device abandons outputting an audio notification corresponding to the first event via the one or more other audio output devices. In some embodiments, the device includes one or more tactile output generators, and the device abandons outputting a tactile output notification corresponding to the first event via the one or more tactile output generators. In some embodiments, the device includes a display, and the device abandons outputting a visual notification corresponding to the first event. References herein Figure 5AD Describes suppressing redundant audio notifications.

[0305] Suppressing other types of notifications for an event for which an audio notification is or will be provided avoids redundant feedback that could distract or confuse the user, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the wearable audio output device and the electronic device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0306] It should be understood that FIG. 8A to FIG. 8D The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein in conjunction with one or more other methods described herein (e.g., methods 600, 700, 900, and / or 1000) are also applicable in a similar manner to the details of the processes described above in conjunction with FIG. 8A to FIG. 8D Method 800 is described. For example, the devices or components thereof, events, audio notifications, and inputs described above with reference to method 800 optionally have one or more of the features of the devices or components thereof, events, audio notifications, and inputs described herein with reference to one or more of the other methods described herein (e.g., methods 600, 700, 900, and / or 1000). For the sake of brevity, these details are not repeated here.

[0307] 9A to 9C is a flow chart illustrating a method 900 of prompting a user and receiving a response to an audio notification according to some embodiments. The method 900 is performed in an electronic device (e.g., Figure 1AThe portable multifunction device 100 or Figure 3A The method is performed at a device 300, the electronic device being connected (eg, via a wired or wireless connection) to one or more wearable audio output devices (eg, Figure 3B A wearable audio output device 301, such as an in-ear headset, a circum-ear headset, etc., which is separate from the electronic device, and one or more sensors (e.g., Figure 3B In some embodiments, the sensor is located on a wearable audio output device (e.g., Figure 3B In some embodiments, the sensor is located on the electronic device. In some embodiments, the device includes a display (e.g., Figure 1A Touch screen 112 or Figure 3A display 340) and a touch-sensitive surface (e.g., Figure 1A Touch screen 112 or Figure 3A In some embodiments, the device communicates with or includes one or more audio output devices (e.g., one or more speakers, such as Figure 1A The speaker 111 on the device 100 or Figure 3B In some embodiments, the device includes one or more audio input devices (e.g., Figure 1A Microphone 113 or Figure 3B In some embodiments, the device includes a rotatable input mechanism (e.g., as a Figure 1A In some embodiments, the device includes one or more accelerometers or posture sensors (e.g., a portion of other input controllers 160) for detecting movement and / or posture changes of the device. Figure 1A In some embodiments, a wearable audio output device in communication with the device includes one or more accelerometers or posture sensors (e.g., as sensors for detecting movement and / or posture changes of the wearable audio output device) Figure 3B Some operations in method 900 are optionally combined, and / or the order of some operations is optionally changed.

[0308] As described below, method 900 outputs one or more audio notifications via one or more wearable audio output devices (e.g., worn by a user), the one or more audio notifications corresponding to an event detected at an associated electronic device, and then outputs an audio indication that the electronic device is ready to receive a response to the detected event. Outputting the audio notification and audio indication via the wearable audio output device prompts the user to respond, and also indicates that the response can be provided via the wearable audio output device, which provides an alternative input mechanism that reduces the need for the user to interact with the electronic device, in particular by reducing the need for the user to view and respond to the detected event using the display of the device. This reduces the number of inputs that need to be processed and reduces the amount of time the display needs to be powered on, thereby enhancing the operability of the device and making the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating the device / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0309] The device detects (902) the occurrence of a first event.

[0310] After detecting the occurrence of the first event, the device outputs (904) one or more audio notifications corresponding to the first event via the one or more wearable audio output devices.

[0311] After outputting the one or more audio notifications corresponding to the first event, the device outputs (906) a corresponding audio output separate from the one or more audio notifications indicating that the electronic device is ready to receive a response to the first event.

[0312] While outputting the corresponding audio output, the device detects (908) an input corresponding to a response to the first event and generates a response to the first event based on the input.

[0313] In some embodiments, after outputting (or starting to output) the one or more audio notifications corresponding to the first event, the device detects (910) a second input via the electronic device or the one or more wearable audio output devices, the second input corresponding to a request for a response to the first event (e.g., a tap input directed to the electronic device or the one or more wearable audio output devices). In response to detecting the second input, an output indicates that the electronic device is ready to receive a corresponding audio output in response to the first event. In some embodiments, based on determining that the second input is detected when the one or more audio notifications corresponding to the first event are output or within a predefined time period related to the one or more audio notifications (e.g., relative to the start of outputting the one or more audio notifications, or relative to the end of outputting the one or more audio notifications), the corresponding audio output is output.

[0314] Requiring the user to provide input corresponding to the request for a response to the first event provides confirmation that the user intends to provide a response to the first event, and, if the confirmation has not been received, avoids unnecessarily processing unintentional input and thereby avoids performing unintentional operations on the electronic device. Providing improved input processing and control options enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0315] In some embodiments, the corresponding audio output includes (912) white noise (e.g., as described herein with reference to Figures 5S to 5AB The corresponding audio output is composed of white noise. Providing white noise to indicate that the device is ready to accept a response to an event for which an audio notification is output prompts the user to respond using intuitive sounds, which enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide the requested input and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0316] In some embodiments in which the electronic device communicates with one or more audio input devices (e.g., one or more microphones as part of the electronic device or as part of the one or more wearable audio output devices), detecting an input corresponding to a response to the first event includes (914) detecting an audio input via the one or more audio input devices (e.g., as described herein with reference to Figure 5AA Receiving responses to detected events via one or more audio input devices provides the user with an alternative input mechanism to accepting verbal input (e.g., voice), which can provide input in a more intuitive and natural manner and reduces the need for the user to use another input device (such as a touch-sensitive surface or display of the electronic device) to view and respond to detected events, which reduces the amount of time those devices need to be activated or powered on and assists visually impaired users, thereby enhancing the operability of the device and making the user-device interface more effective (e.g., by helping the user provide appropriate input and achieve intended results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0317] In some embodiments where the electronic device includes a touch-sensitive surface, while outputting a corresponding audio output, and before detecting an input corresponding to a response to a first event (916), the device: detects a first touch input via the touch-sensitive surface; and in response to detecting the first touch input via the touch-sensitive surface, outputs one or more response options to the first event via the one or more wearable audio output devices. In some embodiments, the input is a drag input, and different response options are output as the input moves on the touch-sensitive surface. In some embodiments, detecting the input corresponding to the response to the first event includes detecting a second touch input (e.g., a tap input) via the touch-sensitive surface that selects a corresponding response option among the one or more response options. In some embodiments, if a second input is detected when the corresponding response option is output or within a predefined time after the output (and in some embodiments, before starting to output a subsequent response option), the second input selects the corresponding response option. In some embodiments, the second input is a lift-off of the first touch input combined with the end of a predefined time period after the first touch input is lifted, wherein no subsequent touch input is detected during the predefined time period. In some embodiments, the generated response to the first event corresponds to the selected corresponding response option. References herein Figure 5T to Figure 5V Outputting response options in response to touch input is described.

[0318] Outputting response options for responding to detected events via a wearable audio output device in response to input on a touch-sensitive surface of an electronic device allows the user to listen to the response options before responding, and in some cases, does not have to view the response options on a display of the electronic device, which reduces the amount of time the display needs to be powered on and assists visually impaired users, thereby enhancing the operability of the device and making the user-device interface more effective (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0319] In some embodiments where the electronic device includes a rotatable input mechanism, while outputting a corresponding audio output, and before detecting an input corresponding to a response to a first event (918), the device: detects a first input (e.g., rotation of the rotatable input mechanism) via the rotatable input mechanism; and in response to detecting the first input via the rotatable input mechanism, outputs one or more response options to the first event via the one or more wearable audio output devices. In some embodiments, detecting the input corresponding to a response to the first event includes detecting a second input (e.g., a tap input) that selects a corresponding response option from the one or more response options via the rotatable input mechanism (or via another input device or sensor of the electronic device, such as a touch-sensitive surface or accelerometer). In some embodiments, if the second input is detected when the corresponding response option is output or within a predefined time after the output (and in some embodiments, before starting to output a subsequent response option), the second input selects the corresponding response option. In some embodiments, the generated response to the first event corresponds to the selected corresponding response option. Reference herein Figure 5W to Figure 5Y Outputting response options in response to rotation of a rotatable input mechanism is described.

[0320] Outputting response options for responding to detected events via the wearable audio output device in response to input via the rotatable input mechanism allows the user to listen to the response options before responding, and in some cases, does not have to view the response options on the display of the electronic device, which reduces the amount of time the display needs to be powered on and assists visually impaired users, thereby enhancing the operability of the device and making the user-device interface more efficient (e.g., by helping the user achieve intended results and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0321] In some embodiments in which the electronic device includes a display and one or more posture sensors (e.g., one or more cameras, gyroscopes, inertial measurement units, or other sensors for detecting a change in orientation of the electronic device), before detecting an input corresponding to a response to a first event, the device detects (920) a change in posture (e.g., orientation) of the electronic device via the one or more posture sensors (e.g., corresponding to an upward tilt of the electronic device so as to lift a top edge of the electronic device relative to a bottom edge of the electronic device). In some embodiments, in response to detecting a change in posture of the electronic device, based on determining that a change in posture of the electronic device is detected within a predefined time period (e.g., when the one or more audio notifications are output or within a predefined time after the output), the predefined time period being related to the one or more audio notifications corresponding to the first event, the device displays one or more response options to the first event on the display (e.g., as described herein with reference to Figure 5Z to Figure 5AA In some embodiments, detecting an input corresponding to a response to the first event includes detecting an input selecting a corresponding response option among the one or more response options.

[0322] Displaying response options for responding to a detected event on a display of an electronic device in response to the electronic device being raised allows the user to quickly view response options before responding to the electronic device with a minimal number of inputs, without having to wait for the options to be read aloud in sequence via the wearable audio output device. This reduces the number of inputs that need to be processed and reduces the amount of time the wearable audio output device needs to be activated, thereby enhancing the operability of the device and making the user-device interface more efficient (e.g., by helping the user achieve the desired result more quickly and reducing user errors when operating / interacting with the device), which in turn reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0323] In some embodiments, after outputting the one or more audio notifications corresponding to the first event, and before outputting a corresponding audio output indicating that the electronic device is ready to receive a response to the first event, the device outputs (922) via the one or more wearable audio output devices prompting a user of the electronic device to provide multiple response options for a response to the first event (e.g., as described herein with reference to Figure 5ABAutomatically outputting a response option for responding to a detected event via the wearable audio output device after outputting a corresponding audio notification reduces the number of inputs that need to be processed. Outputting a corresponding audio output indicating that the device is ready to receive a response after outputting the response option allows the user to listen to the response option before being prompted to respond, which reduces the need for the user to view the response option on the display of the device, thereby reducing the amount of time the display needs to be powered on and helping visually impaired users. These beneficial effects enhance the operability of the device and make the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the device), which in addition reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0324] In some embodiments, the first event is a calendar event invitation, and the plurality of response options include an option to accept the calendar event invitation and an option to decline the calendar event invitation (e.g., as described herein with reference to Figure 5AB In some embodiments, the plurality of response options further include an option to postpone processing the invitation (e.g., postpone responding to the invitation until a later time) and / or an option to tentatively accept the invitation. In some embodiments, the detected input corresponding to the response to the first event includes a selection (e.g., an audio narration) of a corresponding option in the plurality of response options.

[0325] In some embodiments in which the electronic device communicates with one or more audio input devices, the device detects (926) an ambient noise volume relative to the one or more audio input devices; and adjusts the output volume of the one or more audio notifications based on the ambient noise volume (e.g., as described herein with reference to Figure 5AB The invention relates to an audio output device that is configured to receive an audio output in response to an event. The audio output of the notification is adjusted based on the ambient noise volume (and / or indicates that the device is ready to receive an audio output in response to a corresponding event) to help the user listen to the audio output in a noisy environment by increasing the output volume, and reduce damage to the user's ears by reducing the output volume in a quieter environment, where the user does not need a higher volume to listen to the audio output through the ambient noise, thereby providing improved audio feedback to the user. In addition, reducing the output volume when a higher volume is not needed reduces the power usage of the audio output device. These beneficial effects enhance the operability of the device and make the user-device interface more efficient (e.g., by helping the user achieve the desired results and reducing user errors when operating / interacting with the device), which in addition reduces power usage and extends the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0326] It should be understood that 9A to 9CThe specific order in which the operations have been described herein is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize a variety of ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein in conjunction with one or more other methods described herein (e.g., methods 600, 700, 800, and / or 1000) are also applicable in a similar manner to the details of the processes described above in conjunction with 9A to 9C Method 900 is described. For example, the devices or components thereof, events, audio notifications, and inputs described above with reference to method 900 optionally have one or more of the features of the devices or components thereof, events, audio notifications, and inputs described herein with reference to one or more of the other methods described herein (e.g., methods 600, 700, 800, and / or 1000). For the sake of brevity, these details are not repeated here.

[0327] FIG. 10A to FIG. 10B 1 is a flow chart illustrating a method 1000 for conditionally outputting an audio notification based on user activity and context according to some embodiments. The method 1000 is performed on an electronic device (e.g., Figure 1A The portable multifunction device 100 or Figure 3A The method is performed at a device 300, the electronic device being connected (eg, via a wired or wireless connection) to one or more wearable audio output devices (eg, Figure 3B A wearable audio output device 301, such as an in-ear headset, a circum-ear headset, etc., which is separate from the electronic device, and one or more sensors (e.g., Figure 3B In some embodiments, the sensor is located on a wearable audio output device (e.g., Figure 3B In some embodiments, the sensor is located on the electronic device. In some embodiments, the device includes a display (e.g., Figure 1A Touch screen 112 or Figure 3A display 340) and a touch-sensitive surface (e.g., Figure 1A Touch screen 112 or Figure 3A In some embodiments, the device communicates with or includes one or more audio output devices (e.g., one or more speakers, such as Figure 1A The speaker 111 on the device 100 or Figure 3B In some embodiments, the device includes one or more audio input devices (e.g., Figure 1A Microphone 113 or Figure 3BIn some embodiments, the device includes a rotatable input mechanism (e.g., as a Figure 1A In some embodiments, the device includes one or more accelerometers or posture sensors (e.g., a portion of other input controllers 160) for detecting movement and / or posture changes of the device. Figure 1A In some embodiments, a wearable audio output device in communication with the device includes one or more accelerometers or posture sensors (e.g., as sensors for detecting movement and / or posture changes of the wearable audio output device) Figure 3B Some operations in method 1000 are optionally combined, and / or the order of some operations is optionally changed.

[0328] As described below, if the user is currently participating in a conversation (e.g., as detected via a microphone on the electronic device or on a wearable output device that communicates with the electronic device), the method 1000 delays outputting an audio notification from the electronic device. This reduces the intrusiveness of the audio notification and also increases the likelihood that the user will listen to and understand the content of the audio notification, which reduces the need for the user to review the notification one or more times, either audibly or visually, on the display of the device, thereby reducing the number of inputs that need to be processed and the amount of time that the display needs to be powered on, thereby enhancing the operability of the device, and reducing power usage and extending battery life by enabling the user to use the device more quickly and efficiently.

[0329] While the user is wearing the one or more wearable audio output devices, the device detects (1002) an occurrence of a first event.

[0330] In response to detecting the occurrence of a first event, the device outputs (1004) one or more audio notifications corresponding to the first event via the one or more wearable audio output devices, including: based on determining that the user of the electronic device is currently participating in a conversation (e.g., based on the detected voice of the user or another user), delaying the output of the one or more audio notifications corresponding to the first event until the conversation has ended; and based on determining that the user of the electronic device is not currently participating in the conversation, outputting the one or more audio notifications corresponding to the first event without delaying the output. Figure 5AC Describes delaying the output of an audio notification based on whether a user is engaged in a session.

[0331] In some embodiments where the electronic device is in communication with one or more audio input devices, determining that a user of the electronic device is currently participating in a conversation includes (1006) detecting speech via the one or more audio input devices (e.g., as described herein with reference to Figure 5AC In some embodiments, the one or more audio input devices include one or more microphones. In some embodiments, the one or more audio input devices are part of the electronic device. In some embodiments, the one or more audio input devices are part of the one or more wearable audio output devices. Automatically detecting speech via one or more audio input devices such as microphones allows the device to quickly and easily determine whether the user is participating in the conversation without using alternative methods that may require the device to prompt the user to provide one or more inputs. Automatically performing speech detection reduces the number of inputs required, which enhances the operability of the device, reduces power usage, and extends the battery life of the device.

[0332] In some embodiments, outputting the one or more audio notifications includes (1008) outputting audio content corresponding to the first event using simulated speech (e.g., as described herein with reference to Figure 5AC Outputting an audio notification using simulated speech to output audio content corresponding to a detected event provides the user with substantive information about the detected event (e.g., the sender or initiator of the event, the content of a received message, the time and location information of a calendar or other event invitation, etc.) without requiring the user to provide additional input to hear the substantive information or requiring the user to review the event information visually on the display of the device. This reduces the number of inputs that need to be processed and reduces the amount of time the display needs to be powered on, thereby enhancing the operability of the device, reducing power usage, and extending the battery life of the device.

[0333] In some embodiments, outputting the one or more audio notifications includes (1010) outputting one or more audio tones (e.g., as described herein with reference to Figure 5AC The audio notification 5106 is described above). Outputting an audio notification using one or more audio tones (e.g., outputting audio content without using simulated voice) notifies the user that an event has been detected, while reducing the intrusiveness of the notification and the length of the output audio content, which reduces the amount of time the wearable audio output device needs to be activated or powered on, thereby enhancing the operability of the device, reducing power usage, and extending the battery life of the device.

[0334] In some embodiments, the electronic device includes or communicates with one or more second audio output devices (e.g., one or more speakers as part of the electronic device), and the one or more second audio output devices are different from the one or more wearable audio output devices. In some embodiments, in conjunction with outputting language information corresponding to the second event via the one or more second audio output devices, the device abandons (1012) outputting an audio notification corresponding to the second event via the one or more wearable audio output devices. In some embodiments, in conjunction with outputting the one or more audio notifications corresponding to the first event via the one or more wearable audio output devices, the device abandons outputting language information (e.g., audio notification) corresponding to the first event via the one or more second audio output devices. That is, in some embodiments, the device outputs language information (e.g., audio notification) via the one or more wearable audio output devices or via the one or more second audio output devices, but not both.

[0335] For example, for turn-by-turn navigation, when outputting verbal directions via one or more speakers on or in communication with the electronic device, the device abandons outputting verbal directions via the one or more wearable audio output devices. In another example, when outputting verbal directions via the one or more wearable audio output devices, the device abandons outputting verbal directions via the one or more speakers on or in communication with the device. Figure 5AD These examples are described further below.

[0336] Giving up outputting the same audio notification via some audio output devices while outputting the same audio notification via other audio output devices avoids redundant audio output, thereby reducing the overall power usage of the various audio output devices, and also reduces user confusion, thereby enhancing device operability and reducing user errors, which in turn reduces overall power usage and extends the battery life of the device.

[0337] In some embodiments, based on determining that the metadata of the first event associates the first event with an audio notification, performing (1014) outputting the one or more audio notifications corresponding to the first event. In some embodiments, the metadata specifies that the notification corresponding to the first event should be presented to the user verbally. In some embodiments, based on determining that the metadata of the first event does not associate the first event with an audio notification, the device abandons outputting the one or more audio notifications corresponding to the first event. In some embodiments, the metadata of the first event specifies one or more attributes of the first event (e.g., an application, an application type, an event type, and / or a contact for the first event), and the metadata or settings stored on the electronic device associate the corresponding attributes of the first event with the audio notification.

[0338] In some embodiments, the metadata associates an application with an audio notification, and the application is associated with the first event (e.g., the metadata specifies that notifications corresponding to a particular application should be presented verbally to the user). In some embodiments, the first event is associated with an application that is a particular type of application, and the metadata associates the application type of the application with the audio notification (e.g., the metadata specifies that notifications corresponding to the particular type of application should be presented verbally to the user). For example, the first event is associated with a messaging type application, and the metadata specifies that notifications corresponding to an instant messaging application should be presented verbally to the user (e.g., via the one or more wearable audio output devices). Other application types include calendar applications, fitness applications, gaming applications, and the like.

[0339] In some embodiments, the first event is a first type event, and the metadata associates the event type of the event with an audio notification. For example, the first event is a calendar event, and the metadata specifies that a notification corresponding to the calendar event should be presented to the user verbally. In some embodiments, the first event corresponds to a notification from a specific contact, and the metadata associates the notification from the specific contact with the audio notification. For example, the first event corresponds to a message received from a specific sender, and the metadata specifies that a message from the specific sender should be presented to the user verbally. In some embodiments, the specific contact is one of a plurality of contacts on a contact whitelist stored on the electronic device.

[0340] Conditionally outputting an audio notification for a detected event based on metadata of the detected event indicating that the event is associated with an audio notification reduces the intrusiveness and frequency of audio notifications by requiring each event to specify that an audio notification should be provided for the event, and provides flexibility in enabling or disabling audio notifications for specific events. Providing improved control over the presentation of audio notifications enhances device operability, reduces power usage, and extends device battery life.

[0341] In some embodiments, based on determining that the electronic device is in a corresponding mode that allows audio notifications, performing (1016) outputting the one or more audio notifications corresponding to the first event. In some embodiments, based on determining that the electronic device is in a corresponding mode that does not allow audio notifications, the device abandons outputting the one or more audio notifications corresponding to the first event, or delays outputting the one or more audio notifications corresponding to the first event until the device is in a mode that does allow audio notifications.

[0342] In some embodiments, one or more device modes can be defined in a settings user interface on an electronic device. In some embodiments, the one or more device modes are defined by the user (e.g., the user specifies a mode based on the situation where the user wishes to receive audio notifications or does not wish to receive audio notifications). In some embodiments, the one or more device modes are defined based on volume settings, time of day, geographic location of the device, and / or calendar availability of the user, etc. An exemplary mode that allows audio notifications is a "work" mode, which is defined based on working hours and / or the location of the device being present at or near the user's workplace. An exemplary mode that does not allow audio notifications is a "gym" mode, which is defined based on scheduled time and / or the location of the device being present at or near the user's gym. In some embodiments, a specific mode allows specific types of audio notifications while not allowing other types of audio notifications (e.g., a "work" mode allows language descriptions of calendar notifications, but does not allow language descriptions of game notifications).

[0343] Conditionally outputting audio notifications of detected events based on determining the current mode or context of the electronic device allows audio notifications to reduce the intrusiveness and frequency of audio notifications in certain modes, while allowing audio notifications to be provided in other modes that are appropriate for a particular device mode, and provides users with the flexibility to enable or disable audio notifications for particular modes (e.g., time of day, location, type of notification, etc.). Providing users with improved control over the presentation of audio notifications enhances device operability, reduces power usage, and extends device battery life.

[0344] In some embodiments, outputting (1018) the one or more audio notifications corresponding to the first event is performed based on previous user behavior monitored by a virtual assistant executing on the electronic device. In some embodiments, the virtual assistant monitors user behavior on the electronic device to determine situations where the user has allowed audio notification output (e.g., geographic location, time of day, volume setting, sender, etc.) and situations where the user has suppressed audio notification output (e.g., by choosing not to wear the one or more wearable audio output devices, by skipping the audio notification being presented, etc.). In some embodiments, based on determining that the device and / or the one or more wearable audio output devices are in a situation where the user has previously allowed audio notifications, the one or more audio notifications corresponding to the first event are output. In some embodiments, based on determining that the device and / or the one or more wearable audio output devices are in a situation where the user has previously suppressed audio notifications, the device abandons outputting the one or more audio notifications corresponding to the first event.

[0345] Conditionally and automatically outputting audio notifications of detected events based on previous user behavior learned by the virtual assistant reduces the number and extent of control options that need to be presented to the user via a displayed settings interface, and reduces the amount of input that the user needs to provide to the electronic device to specify the desired notification behavior, thereby enhancing the operability of the device, reducing power usage and extending the battery life of the device.

[0346] In some embodiments in which an electronic device communicates with one or more audio input devices, before outputting the one or more audio notifications corresponding to a first event, the device detects (1020) via the one or more audio input devices a speech input indicating a corresponding event attribute (e.g., event type, application, application type, etc.) of interest to the user. In some embodiments, outputting the one or more audio notifications corresponding to the first event is performed based on determining that the first event is associated with the corresponding event attribute. For example, in response to speech input from the user indicating that the user is interested in traffic-related notifications, the device outputs the one or more audio notifications if the first event includes traffic information (e.g., public transportation scheduling information) or if the first event is associated with a navigation application. In some embodiments, the event attributes of interest to the user are inferred from speech input to the electronic device (e.g., by a virtual assistant executing on the device).

[0347] Conditionally outputting audio notifications of detected events based on their association with event attributes specified by the user using previous speech input provides the user with the flexibility to enable or disable (or emphasize or de-emphasize) notifications associated with certain types of events without requiring the user to specify the event attributes of interest via one or more displayed control options, thereby reducing the amount of time the display needs to be powered on, thereby enhancing device operability, reducing power usage and extending the battery life of the device.

[0348] It should be understood that FIG. 10A to FIG. 10B The specific order in which the operations in the description are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations may be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein. In addition, it should be noted that the details of other processes described herein in conjunction with one or more other methods described herein (e.g., methods 600, 700, 800, and / or 900) are also applicable in a similar manner to the details of the processes described above in conjunction with FIG. 10A to FIG. 10BMethod 1000 is described. For example, the devices or components thereof, events, audio notifications, and inputs described above with reference to method 1000 optionally have one or more of the features of the devices or components thereof, events, audio notifications, and inputs described herein with reference to one or more of the other methods described herein (e.g., methods 600, 700, 800, and / or 900). For the sake of brevity, these details are not repeated here.

[0349] For the purpose of explanation, the foregoing description is described by reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the invention to the disclosed precise form. According to the above teachings, many modifications and variations are possible. The embodiments are selected and described in order to best illustrate the principles of the present invention and its practical application, so that other technical personnel in the field can best use the present invention and the various described embodiments with various modifications suitable for the specific purpose contemplated.

[0350] As described above, one aspect of the present technology is to collect and use data from various sources to provide audio notifications to users to inform them of events that may be of interest to them. The present disclosure contemplates that in some instances, these collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, twitter IDs, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying information or personal information.

[0351] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit users. For example, personal information data can be used to deliver event information as part of an audio notification that is of greater interest to the user, while reducing the amount of event information delivered that is not of interest to the user or has no value to the user. Therefore, the use of such personal information data enables event information to be delivered to the user that is customized to the user's interests and takes into account information known to the user. In addition, the present disclosure also contemplates other uses of personal information data that are beneficial to the user. For example, health and fitness data can be used to provide insights into the user's overall health, or can be used as positive feedback to individuals who use technology to pursue health goals.

[0352] This disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of these legitimate uses. In addition, such collection / sharing should be performed after receiving informed consent from the user. In addition, such entities should consider taking any necessary steps to defend and secure access to such personal information data and ensure that others who have access to personal information data comply with their privacy policies and processes. In addition, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to specific types of personal information data collected and / or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the United States, the collection or access of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained in each country for different types of personal data.

[0353] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware elements and / or software elements may be provided to prevent or block access to such personal information data. For example, with respect to an event notification delivery service, the technology of the present invention may be configured to allow a user to choose to "opt in" or "opt out" at any time during or after registration for the service to participate in the collection of personal information data. For another example, a user may choose not to provide contact identification data, location data, data of received messages (such as phone calls, text messages, emails, calendar invitations or events, etc.) to the target event notification delivery service. In yet another example, a user may choose to limit the length of time that the data is retained, or completely prohibit the development of user activity profiles based on these data. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application, and then the user may be reminded again just before the personal information data is accessed by the application.

[0354] In addition, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data between users), and / or other methods when appropriate.

[0355] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, the various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, event notifications may be selected and delivered to a user by inferring preferences based on non-personal information data or an absolute minimum of personal information (e.g., content requested by a device associated with the user, other non-personal information available to an event notification delivery service, or publicly available information).

Claims

1. A method comprising: At an electronic device in communication with one or more wearable audio output devices: detecting an occurrence of one or more first events; In response to detecting an occurrence of the one or more first events, outputting audio content corresponding to the one or more first events via the one or more wearable audio output devices; detecting movement of the one or more wearable audio output devices after outputting the audio content corresponding to the one or more first events; as well as In response to detecting the movement of the one or more wearable audio output devices, and based on determining that a first movement of the one or more wearable audio output devices satisfies a first movement criterion, outputting additional audio content corresponding to one or more events via the one or more wearable audio output devices. 2 . The method of claim 1 , wherein the additional audio content comprises additional audio content for the one or more first events.

3. The method of claim 1 , wherein the audio content corresponding to the one or more first events comprises a simulated spatial location associated with the audio content corresponding to the one or more first events, and wherein, The first movement criteria include criteria related to movement of the one or more wearable audio output devices to the simulated spatial location.

4. The method according to claim 1, wherein the audio content corresponding to the one or more first events includes a first audio content portion corresponding to the one or more first events and a second audio content portion corresponding to the one or more first events; and the method comprises: In response to detecting the movement of the one or more wearable audio output devices, and based on determining that the first movement of the one or more wearable audio output devices satisfies a second movement criterion, outputting the first audio content portion of the audio content corresponding to the one or more first events via the one or more wearable audio output devices without outputting the second audio content portion of the audio content corresponding to the one or more first events.

5. The method of claim 4, wherein the audio content corresponding to the one or more first events comprises a simulated spatial location associated with the audio content corresponding to the one or more first events, and wherein, The second movement criteria include criteria related to movement of the one or more wearable audio output devices away from the simulated spatial location.

6. The method of claim 5, wherein the second movement criteria comprises criteria related to movement of the one or more wearable audio output devices away from the simulated spatial location during output of the audio content corresponding to the one or more first events.

7. The method of claim 4, wherein the first movement of the one or more wearable audio output devices that satisfies the second movement criterion comprises movement backward, tilting backward, or movement away from a simulated spatial location associated with the audio content corresponding to the one or more first events.

8. The method according to claim 1, comprising: In response to detecting a second movement of the one or more wearable audio output devices, and based on determining that the second movement of the one or more wearable audio output devices satisfies a second movement criterion, reducing a level of detail of subsequent audio content corresponding to one or more second events.

9. The method of claim 8, wherein the audio content corresponding to the one or more first events comprises a simulated spatial location associated with the audio content corresponding to the one or more first events, and wherein, The second movement criteria include criteria related to movement of the one or more wearable audio output devices away from the simulated spatial location.

10. The method of claim 9, wherein the second movement criteria comprises criteria related to movement of the one or more wearable audio output devices away from the simulated spatial location after outputting the audio content corresponding to the one or more first events.

11. The method of claim 8, wherein the second movement of the one or more wearable audio output devices comprises moving backward, leaning backward, or moving away from a simulated spatial location associated with the audio content corresponding to the one or more first events.

12. The method of claim 1, wherein the first movement of the one or more wearable audio output devices comprises moving forward, leaning forward, or moving toward a simulated spatial location associated with the audio content corresponding to the one or more first events.

13. The method of claim 1, wherein the one or more wearable audio output devices include one or more accelerometers, and wherein: Detecting the movement of the one or more wearable audio output devices is based on movement detected by the one or more accelerometers.

14. An electronic device that communicates with one or more wearable audio output devices, the electronic device comprising: one or more processors; and A memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for: detecting an occurrence of one or more first events; In response to detecting an occurrence of the one or more first events, outputting audio content corresponding to the one or more first events via the one or more wearable audio output devices; detecting movement of the one or more wearable audio output devices after outputting the audio content corresponding to the one or more first events; as well as In response to detecting the movement of the one or more wearable audio output devices, and based on determining that a first movement of the one or more wearable audio output devices satisfies a first movement criterion, outputting additional audio content corresponding to one or more events via the one or more wearable audio output devices.

15. The electronic device of claim 14, wherein the one or more programs include instructions for executing the method of any one of claims 2 to 13.

16. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by an electronic device in communication with one or more wearable audio output devices, cause the electronic device to: detecting an occurrence of one or more first events; In response to detecting an occurrence of the one or more first events, outputting audio content corresponding to the one or more first events via the one or more wearable audio output devices; detecting movement of the one or more wearable audio output devices after outputting the audio content corresponding to the one or more first events; as well as In response to detecting the movement of the one or more wearable audio output devices and based on determining that a first movement of the one or more wearable audio output devices satisfies a first movement criterion, outputting additional audio content corresponding to one or more events via the one or more wearable audio output devices.

17. The computer-readable storage medium of claim 16, comprising instructions for performing the method of any one of claims 2 to 13.

18. A method comprising: At an electronic device in communication with one or more wearable audio output devices and with one or more sensors for detecting placement of the one or more wearable audio output devices: detecting occurrences of a plurality of events without outputting notifications corresponding to the events via the one or more wearable audio output devices when the one or more wearable audio output devices are not in respective positions relative to the user's ears; after detecting said occurrences of said plurality of events, detecting, via said one or more sensors, that said one or more wearable audio output devices have been placed in said respective positions relative to said user's ears; as well as in response to detecting, via the one or more sensors, that the one or more wearable audio output devices have been placed in the respective positions relative to the user's ears, outputting information via the one or more wearable audio output devices, The information includes one or more audio notifications corresponding to one or more events of the plurality of events.

19. A method comprising: At an electronic device having one or more audio output devices: Detecting the occurrence of an event; as well as In response to detecting the occurrence of the event, outputting an audio notification via the one or more audio output devices, the audio notification comprising audio content representative of the event, wherein: Based on determining that the event is associated with a first feature in a first set of features, the audio notification includes the audio content representative of the event and has a first audio output attribute selected based on the first feature; as well as Based on determining that the event is associated with a second feature of the first set of features that is different from the first feature, the audio notification includes the audio content representative of the event and has a second audio output attribute different from the first output attribute selected based on the second feature.

20. A method comprising: At an electronic device in communication with one or more wearable audio output devices and with one or more sensors for detecting placement of the one or more wearable audio output devices: detecting an occurrence of a first event; after detecting said occurrence of said first event, outputting, via said one or more wearable audio output devices, one or more audio notifications corresponding to said first event; After initiating output of the one or more audio notifications, detecting input directed to the one or more wearable audio output devices; as well as In response to detecting the input directed to the one or more wearable audio output devices: performing a first operation based on determining that the input is detected within a predefined time period, the predefined time period being related to the one or more audio notifications corresponding to the first event, wherein the first operation is associated with the one or more audio notifications corresponding to the first event; as well as A second operation is performed based on determining that the input is detected after the predefined time period has passed, the predefined time period being related to the one or more audio notifications corresponding to the first event, wherein the second operation is not associated with the one or more audio notifications corresponding to the first event.

21. A method comprising: At an electronic device in communication with one or more wearable audio output devices: detecting an occurrence of a first event while the user is wearing the one or more wearable audio output devices; as well as In response to detecting the occurrence of the first event, outputting, via the one or more wearable audio output devices, one or more audio notifications corresponding to the first event, comprising: in response to determining that the user of the electronic device is currently participating in a conversation, delaying outputting the one or more audio notifications corresponding to the first event until the conversation has ended; as well as Based on determining that the user of the electronic device is not currently participating in a session, The one or more audio notifications corresponding to the first event are output without delaying the outputting.

Images