Devices, methods, and user interfaces for adaptively providing audio output

Abstract

An electronic device includes one or more pose sensors to detect a pose of a user of the electronic device relative to a first physical environment, and is in communication with one or more audio output devices. The electronic device provides audio content at a first simulated spatial location relative to the user when a first pose of the user satisfies first presentation criteria. The electronic device detects a change in the pose of the user from the first pose to a second pose. In response to detecting the change in the pose of the user, and in accordance with a determination that the second pose of the user does not satisfy the first presentation criteria, the electronic device provides audio content at a second simulated spatial location relative to the user that is different from the first simulated spatial location.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to electronic devices with audio output devices, such as wearable audio output devices, including but not limited to electronic devices that adaptively provide audio output based on changes in the context of the audio output device. BACKGROUND

[0002] Audio output devices, including wearable audio output devices such as headphones and earphones, are widely used to provide audio output to users. But conventional approaches to providing audio output are cumbersome, inefficient, and limited. In some cases, conventional approaches provide audio output in ways that hinder the user's ability to interact with their surrounding physical environment. In some cases, audio output is provided in a static manner regardless of the user's location, orientation, or movement. In some cases, audio output is provided in a static manner regardless of changes in ambient audio, thereby interfering with the user's ability to understand when to speak, particularly when wearing headphones or earphones. Moreover, conventional approaches take longer and require more user interaction than is necessary to adjust audio output for such contextual changes, thereby wasting energy. This latter consideration is particularly important in battery-powered devices. SUMMARY

[0003] Accordingly, there is a need for electronic devices with improved methods and interfaces for adaptively providing audio output based on changes in the context of an audio output device. Such methods and interfaces optionally supplement or replace conventional methods of providing audio output. Such methods and interfaces reduce the number, extent, and / or nature of the inputs from the user and produce a more efficient human-machine interface. For battery-powered devices, such methods and interfaces can conserve power and increase the time between battery charges.

[0004] The above-mentioned deficiencies and other problems associated with providing audio output are reduced or eliminated by the disclosed devices. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the device is a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the device has one or more audio output devices (e.g., a wearable audio output device, such as an in-ear earphone, an earbud, an over-ear headphone, etc.) (and / or is in communication with one or more audio output devices). In some embodiments, the device has a trackpad (and / or is in communication with a trackpad). In some embodiments, the device has a touch-sensitive display (also known as a "touch screen" or "touch-screen display") (and / or is in communication with a touch-sensitive display). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs or sets of instructions stored in memory for performing various functions. In some embodiments, a user interacts with the GUI primarily through stylus and / or finger contacts and gestures on the touch-sensitive surfaces. In some embodiments, these functions optionally include image editing, drawing, presenting, word processing, spreadsheet making, game playing, telephoning, video conferencing, e-mailing, instant messaging, workout support, digital photographing, digital videoing, web browsing, digital music / audio playback, note taking, and / or digital video playback. Executable instructions for performing these functions are, optionally, included in a non-transitory computer-readable storage medium or other computer program product that is configured for execution by one or more processors.

[0005] According to some embodiments, a method is performed at an electronic device that includes one or more posture sensors to detect a posture of a user of the electronic device relative to a first physical environment. The electronic device is in communication with one or more wearable audio output devices. The method includes providing audio content at a first simulated spatial location relative to the user when a first posture of the user satisfies first presentation criteria. The method includes detecting a change in the posture of the user from the first posture to a second posture; and in response to detecting the change in the posture of the user, and in accordance with a determination that the second posture of the user does not satisfy the first presentation criteria: providing audio content at a second simulated spatial location relative to the user that is different from the first simulated spatial location.

[0006] According to some embodiments, a method is performed at one or more wearable audio output devices that are in a respective physical environment and in communication with an electronic device. The method includes providing audio output corresponding to a first audio criteria when one or more audio properties of the respective physical environment satisfy the first audio criteria. The audio output includes audio corresponding to audio content from the electronic device at a first device content audio level and audio corresponding to environmental sound from the respective physical environment at a first environmental sound audio level. The method includes detecting a change in the one or more audio properties of the respective physical environment and providing audio corresponding to the environmental sound from the respective physical environment at a second environmental sound audio level different from the first environmental sound audio level in response to detecting the change in the one or more audio properties of the respective physical environment.

[0007] According to some embodiments, an electronic device includes or is in communication with one or more audio output devices, optionally one or more posture sensors (e.g., to detect a posture of the electronic device, a posture of the audio output device, or a posture of a user of the electronic device relative to a first physical environment), optionally one or more audio input devices, optionally a display, optionally a touch-sensitive surface or other input device, one or more processors, and memory storing one or more programs configured to be executed by the one or more processors and include instructions for performing or causing performance of the operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium has stored therein instructions which, when executed by an electronic device as described herein, cause the device to perform or cause performance of the operations of any of the methods described herein. According to some embodiments, a graphical user interface on an electronic device as described herein includes one or more of the elements displayed in any of the methods described herein that are updated in response to input, as described in any of the methods described herein. According to some embodiments, an electronic device as described herein includes means for performing or causing performance of the operations of any of the methods described herein. According to some embodiments, an information processing apparatus, such as for use in an electronic device described herein, includes means for performing or causing performance of the operations of any of the methods described herein.

[0008] Thus, electronic devices that include one or more audio output devices or that are in communication with one or more audio output devices, that include optional one or more posture sensors (e.g., to detect a posture of the electronic device, a posture of the audio output device, or a posture of a user of the electronic device relative to a first physical environment), that include optional one or more audio input devices, that include an optional display, and that include an optional touch-sensitive surface or other input device, are provided improved methods and interfaces for adaptively providing audio output, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods and interfaces can complement or replace conventional methods for providing audio output. BRIEF DESCRIPTION OF DRAWINGS

[0009] For a better understanding of various described implementations, reference will be made to the following detailed description in conjunction with the accompanying drawings, in which like reference numbers designate corresponding parts throughout the several figures.

[0010] FIG. 1A is a block diagram illustrating a portable multifunction device having a touch-sensitive display in accordance with some embodiments.

[0011] FIG. 1B is a block diagram illustrating example components for event processing in accordance with some embodiments.

[0012] FIG. 2 shows a portable multifunction device with a touch screen in accordance with some embodiments.

[0013] FIG. 3A is a block diagram of an example multifunction device with a display and a touch-sensitive surface in accordance with some embodiments.

[0014] FIG. 3B is a block diagram of an example wearable audio output device in accordance with some embodiments.

[0015] FIG. 3C shows example audio control by a wearable audio output device in accordance with some embodiments.

[0016] FIG. 4A shows an example user interface for an application menu on a portable multifunction device in accordance with some embodiments.

[0017] FIG. 4B shows an example user interface for a multifunction device with a display separate from a touch-sensitive surface in accordance with some embodiments.

[0018] FIG. 5A through FIG. 5L shows example adaptive audio output in response to example user interactions with a wearable audio output device in accordance with some embodiments.

[0019] FIG. 6A through FIG. 6JAn example adaptive audio output responsive to an example change in audio properties of a surrounding physical environment is shown in accordance with some embodiments.

[0020] FIG. 7A through FIG. 7B is a flowchart of a process for adaptively changing a simulated spatial position of audio output in response to a change in user posture in accordance with some embodiments.

[0021] FIG. 8A through FIG. 8B is a flowchart of a process for adaptively changing an audio output level in response to a change in audio properties of a surrounding physical environment in accordance with some embodiments. DETAILED DESCRIPTION

[0022] As described above, audio output devices such as wearable audio output devices are widely used to provide audio output to a user. Many audio output devices provide audio output in a static manner that does not adapt to changes in user posture or changes in audio properties of a surrounding physical environment. The methods, devices, and user interfaces / interactions described herein improve how audio output is provided in a variety of ways. For example, the embodiments disclosed herein describe ways in which audio output is adaptively provided in the context of an audio output device based on changes such as changes in user posture or changes in audio properties of a surrounding physical environment so that a user can better interact with his surrounding physical environment.

[0023] Below, FIG. 1A through FIG. 1B , FIG. 2 and FIG. 3A through FIG. 3B provide descriptions of examples of example devices and their operation. FIG. 4A through FIG. 4B An example user interface of an example device on which embodiments disclosed herein are implemented is shown. FIG. 5A through FIG. 5L An example adaptive audio output responsive to an example user interaction with a wearable audio output device is shown. FIG. 6A through FIG. 6J An example adaptive audio output responsive to an example change in audio properties of a surrounding physical environment is shown. FIG. 7A through FIG. 7B A flowchart of a method for adaptively changing a simulated spatial position of audio output in response to a change in user posture is shown. FIG. 8A through FIG. 8B A flowchart of a method for adaptively changing an audio output level in response to a change in audio properties of a surrounding physical environment is shown. FIG. 5A through FIG. 5L and FIG. 6A through FIG. 6J user interfaces are used to show FIG. 7A through FIG. 7B , FIG. 8A through FIG. 8B processes in

[0024] Exemplary Device

[0025] Reference will now be made in detail to implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description of implementations, numerous specific details are set forth in order to provide a thorough understanding of the various described implementations. However, it will be apparent to one of ordinary skill in the art that the various described implementations can be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the implementations.

[0026] It will also be understood that, although the terms“first,”“second,” etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first audio output could be termed a second audio output, and, similarly, a second audio output could be termed a first audio output, without departing from the scope of the various described implementations. The first audio output and the second audio output are both audio outputs, but they are not the same audio output, unless the context clearly indicates otherwise.

[0027] The terminology used in the description of the various described implementations herein is for the purpose of describing particular implementations only and is not intended to be limiting. As used in the description of the various described implementations and the appended claims, the singular forms“a,”“an,” and“the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term“and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms“comprises” and / or“comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0028] As used herein, the term“if’ is, optionally, interpreted as meaning“when” or“while” or “in response to a determination” or“in response to a detection,” depending on the context. Similarly, the phrase“if it is determined” or“if [a stated condition or event] is detected” is, optionally, interpreted as meaning“upon being determined” or “in response to determining” or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event],” depending on the context.

[0029] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described herein. In some embodiments, the device is a portable communications device, such as a mobile telephone, that also contains other functions, such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, without limitation, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. Other exemplary embodiments can include a iPod and devices. Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touch-screen displays and / or touchpads), are, optionally, used. It should also be understood that, in some embodiments, the device is not a portable communications device, but is a desktop computer with a touch-sensitive surface (e.g., a touch-screen display and / or a touchpad).

[0030] In the discussion that follows, a device that includes a display and a touch-sensitive surface is described. It should be understood, however, that the device optionally includes one or more other physical user-interface devices, such as a physical keyboard, a mouse and / or a joystick.

[0031] The device typically supports a variety of applications, such as one or more of the following: a notes application, a drawing application, a presentation application, a word processing application, a website creation application, a disk authoring application, a spreadsheet application, a game application, a telephone application, a video conferencing application, an e-mail application, an instant messaging application, a workout support application, a photo management application, a digital camera application, a digital

[0032] The various application programs executed on the device optionally make use of at least one common physical user-interface device, such as the touch- sensitive surface. One or more functions of the touch-sensitive surface, as well as

[0033] Attention is now directed towards embodiments of portable devices with touch- sensitive displays. FIG. 1Ais a block diagram illustrating portable multifunction device 100 having touch-sensitive display system 112 in accordance with some embodiments. Touch-sensitive display system 112 is sometimes called a "touch screen" for convenience, and is sometimes simply called a touch-sensitive display. Device 100 includes memory 102 (which optionally includes one or more computer-readable storage mediums), memory controller 122, one or more processing units (CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control devices 116, and external port 124. Device 100 optionally includes one or more optical sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting intensity of contacts on device 100 (e.g., a touch-sensitive surface such as the touch- sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile outputs on device 100 (e.g., generating tactile outputs on a touch-sensitive surface such as the touch-sensitive display system 112 of device 100 or the touchpad 355 of device 300). These components optionally communicate over one or more communication buses or signal lines 103.

[0034] As used in the specification and claims, the term “tactile output” refers to physical displacement of a device relative to a previous position of the device, physical displacement of a component (e.g., a touch-sensitive surface) of a device relative to another component (e.g., a housing) of the device, or displacement of a component relative to a center of mass of the device that will be detected by a user with the user’s sense of touch. For example, in situations where the device or the component of the device is in contact with a surface (e.g., a surface of a finger, palm, or other part of a user’s hand), the tactile output generated by the physical displacement will be interpreted by the user as an object that is felt on the surface of the device or the component of the device. In some situations, the tactile output generated by physical displacement of a device or a component of a device will be interpreted by a user as an object or change in sensation that is felt on the surface of the device or the component of the device. In some situations, the tactile output generated by physical displacement of a device or a component of a device is interpreted by a user as a change in friction between an object and the device or the component of the device. In some situations, the tactile output generated by physical displacement of a device or a component of a device is interpreted by a user as having effects on the user’s physical sensations, such as touch, pain, pressure, and / or the like. While tactile outputs generated by physical displacement of a device or a component of a device are ultimately sensed by a user, it is important to note that to “generate” a tactile output does not require that the user perceive the tactile output. Indeed, a tactile output can be generated without the user being present or even without the user being aware that the tactile output is being generated. Although a tactile output is not required to be perceived by the user, in some situations, the tactile output is generated to produce a tactile effect that is perceived by the user.

[0035] In some embodiments, a tactile output pattern specifies a characteristic of a tactile output, such as an amplitude of the tactile output, a shape of a motion waveform of the tactile output, a frequency of the tactile output, and / or a duration of the tactile output.

[0036] When a device generates haptic output having different haptic output patterns (e.g., via one or more haptic output generators that generate haptic output via moving movable masses), the haptic output can produce different tactile sensations in a user holding or touching the device. While the user's senses are based on the user's perception of the haptic output, most users will be able to recognize changes in the waveform, frequency, and amplitude of the haptic output generated by the device. Thus, the waveform, frequency, and amplitude can be adjusted to indicate to the user that a different operation has been performed. In this way, haptic output having haptic output patterns designed, selected, and / or arranged to simulate characteristics (e.g., size, material, weight, stiffness, smoothness, etc.) of objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment having virtual boundaries and virtual objects, a real physical environment having physical boundaries and physical objects, and / or a combination of any of the above); behaviors (e.g., oscillation, displacement, acceleration, rotation, stretching, etc.); and / or interactions (e.g., collision, adhesion, repulsion, attraction, friction, etc.) of objects in the given environment will, in some cases, provide helpful feedback to the user that reduces input errors and improves the user's efficiency in operating the device. Additionally, haptic output is optionally generated to correspond to feedback unrelated to simulated physical characteristics such as input thresholds or object selection. Such haptic output will, in some cases, provide helpful feedback to the user that reduces input errors and improves the user's efficiency in operating the device.

[0037] In some embodiments, haptic output having suitable haptic output patterns serves as a cue that an event of interest occurred in a user interface or behind a screen in a device. Examples of events of interest include activation of an affordance (e.g., a real or virtual button, or a toggle switch) provided on the device or in the user interface, success or failure of a requested operation, reaching or crossing a boundary in the user interface, entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detecting or recognizing a type of input or gesture, and so on. In some embodiments, haptic output is provided to serve as a warning or cue about an impending event or outcome that will occur unless a change in direction or interruption of the input is timely detected. Haptic output is also used in other contexts to enrich the user experience, improve accessibility of the device for users with visual or motor difficulties or other accessibility needs, and / or improve the efficiency and functionality of the user interface and / or device. Haptic output is optionally compared with audio input and / or visual user interface changes, which further enhances the user's experience when the user interacts with the user interface and / or device, facilitates better transmission of information about the state of the user interface and / or device, and reduces input errors and improves the user's efficiency in operating the device.

[0038] It should be appreciated that device 100 is only one example of a portable multifunctional device, and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. The various components shown here are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing and / or application specific integrated circuits. FIG. 1A The various components shown here are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing and / or application specific integrated circuits.

[0039] Memory 102 optionally includes high-speed random access memory and also optionally includes nonvolatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other nonvolatile solid-state storage devices. Access to memory 102 by other components of device 100 is, optionally, controlled by a memory controller 122.

[0040] Peripheral interface 118 can be used to couple input and output peripherals of the device to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or sets of instructions stored in memory 102 to perform the various functions of device 100 and to process data.

[0041] In some embodiments, peripheral interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0042] RF (radio frequency) circuitry 108 receives and sends RF signals, also called electromagnetic signals. RF circuitry 108 converts electrical signals to / from electromagnetic signals and communicates with communications networks and other communications devices, via the electromagnetic signals. The RF circuitry 108, optionally, includes well-known circuitry for detecting and for resolving signal frequencies and for tuning out (at least partially) unwanted signal frequencies, such as those produced within an electronic device, power supply, and / or communications network. The RF circuitry 108, optionally, includes a plurality of antennas. The plurality of antennas, optionally, includes one or more diversity antennas and / or one or more omnidirectional antennas. The plurality of antennas, optionally, includes one or more directional antennas (e.g., dBi, diH, hi-gain antennas). The plurality of antennas, optionally, includes one or more low-profile antennas. The plurality of antennas, optionally, includes one or more monopole antennas. The plurality of antennas, optionally, includes one or more dipole antennas. The plurality of antennas, optionally, includes one or more helical antennas. The plurality of antennas, optionally, includes one or more microstrip antennas. The plurality of antennas, optionally, includes one or more patch antennas. The plurality of antennas, optionally, includes one or more turnstile antennas. The plurality of antennas, optionally, includes one or more

[0043] Audio circuitry 110, speaker 111, and microphone 113 provide an audio interface between a user and device 100. Audio circuitry 110 receives audio data from peripherals interface 118, converts the audio data to an electrical signal, and transmits the electrical signal to speaker 111. Speaker 111 converts the electrical signal to human-audible sound waves. Audio circuitry 110 also receives electrical signals converted by microphone 113 from sound waves. Audio circuitry 110 converts the electrical signal to audio data and transmits the audio data to peripherals interface 118 for processing. Audio data is, optionally, retrieved from and / or transmitted to memory 102 and / or RF circuitry 108 by peripherals interface 118. In some embodiments, audio circuitry 110 also includes a headset jack (e.g., 212 in FIG. 2). The headset jack provides an interface between audio circuitry 110 and removable audio input / output peripherals, such as output-only headphones or a headset with both output (e.g., stereo FIG. 2

[0044] I / O subsystem 106 couples input / output peripherals on device 100, such as touch-sensitive display system 112 and other input or control devices 116, with peripherals interface 118. I / O subsystem 106 optionally includes display controller 156, optical sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from / to other input or control devices 116. The other input or control devices 116 optionally include physical buttons (e.g., 208 in FIG. 2, such as push buttons, rocker buttons, and / or the like), dials, slide controls, joysticks, click wheels, and so forth. In some alternate embodiments, input controller(s) 160 are, optionally, coupled with any FIG. 2 FIG. 2

[0045] ​​​The touch-sensitive display system 112 provides an input interface and an output interface between the device and a user. The display controller 156 receives and / or sends electrical signals from / to the touch-sensitive display system 112. In some embodiments, the touch-sensitive display system 112 or the display controller 156, or any combination thereof, are considered display generation components of the device 100. The touch-sensitive display system 112 renders visual output onto the touch-sensitive display 112. The visual output optionally includes graphical information, text, icons, video, and any combination thereof (collectively termed "graphics"). In some embodiments, some or all of the visual output corresponds to user-interface objects. As used herein, the term "affordance" refers to a user-interactive graphical user-interface object (e.g., a graphical user-interface object that is configured to respond to inputs directed at the graphical user-interface object). Examples of user-interactive graphical user-interface objects include, without limitation, a button, a slider, an icon, a selectable menu item, a switch, a hyperlink, or other user-interface control.

[0046] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from the user based on haptic and / or tactile contact. The touch-sensitive display system 112 and the display controller 156 (along with any associated modules and / or sets of instructions in memory 102) detect contact (and any movement or breaking of the contact) on the touch-sensitive display system 112 and convert the detected contact into interaction with user-interface objects (e.g., one or more soft keys, icons, web pages, or images) that are displayed on the touch-sensitive display system 112. In some embodiments, a point of contact between the touch-sensitive display system 112 and the user corresponds to a finger or stylus of the user.

[0047] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light-emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 112 and the display controller 156 optionally detect contact and any movement or breaking thereof using any of a plurality of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112. In some embodiments, projected mutual capacitance sensing technology is used, such as that found in the iPhone® iPod and Touch® devices from Apple Inc. (Cupertino, California) and is not limited to Apple’s iPhone® or iPod touch® devices.

[0048] Touch-sensitive display system 112 optionally uses LCD (Liquid Crystal Display) technology, LPD (Light Emitting Polymer Display) technology, or LED (Light Emitting Diode) technology, although other display technologies are used in other implementations. Viewable images generated by the display are incident on the touch-sensitive display system, which includes a touch-sensitive surface capable of receiving input from an object such as a finger, stylus, or any suitable input instrument or object. The multi-touch touch-sensitive display system includes electrical circuitry configured to sense

[0049] In some embodiments, in addition to the touch screen, device 100 optionally includes a touchpad (not shown) for activating or deactivating particular functions. In some embodiments, the touchpad is a touch-sensitive area of the device that does not display visual output to the user, unlike the touch screen. The touchpad optionally is a touch-sensitive surface that is separate from the touch screen 112, or is an extension of the touch-sensitive surface formed by the touch screen.

[0050] Device 100 also includes power system 162 for powering the various components of device 100. Power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power for device 100.

[0051] Device 100 optionally also includes one or more optical sensors 164. FIG. 1A Optical sensor(s) 164 are shown coupled to optical sensor controller 158 in I / O subsystem 106. Optical sensor(s) 164 optionally include charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) photoreceptor(s). Optical sensor(s) 164 receive light from the environment, projected through one or more lenses, and converts the light to data representing an image. In conjunction with imaging module 143 (also called a camera module), optical sensor(s) 164 optionally capture still images and / or video. In some embodiments, an optical sensor is located on the back of device 100, opposite touch-sensitive display system 112 on the front of the device, so that the touch screen is enabled for use as a viewfinder for still and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device so that the user’s image is obtained (e.g., for selfies, for videoconferencing, etc.).

[0052] Device 100 optionally also includes one or more contact intensity sensors 165. FIG. 1A Contact intensity sensor(s) 165 are coupled with intensity sensor controller 159 in I / O subsystem 106. One or more contact intensity sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electric force sensors, piezoelectric force sensors, optical force sensors, capacitive touch surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch sensitive surface).

[0053] Device 100 optionally also includes one or more proximity sensors 166. FIG. 1A Proximity sensor(s) 166 are coupled with peripherals interface 118. Alternatively, proximity sensor 166 is coupled with input controller 160 in I / O subsystem 106. In some embodiments, proximity sensor 166 switches off and disables touch- sensitive display system 112 when the multifunction device is placed near the user’s ear (e.g., when the user is making a phone call).

[0054] Device 100 optionally also includes one or more tactile output generators 167. FIG. 1A Tactile output generators are coupled with haptic feedback controller 161 in I / O subsystem 106. In some embodiments, tactile output generator(s) 167 include one or more electroacoustic devices such as speakers or other audio components and / or electromechanical devices such as a motor, a solenoid, an electroactive polymer, a piezoelectric actuator, an electrostatic actuator, or other tactile output generating components (e.g., components used to convert electrical signals into tactile outputs on the device). Tactile output generator(s) 167 receive tactile feedback generation instructions from haptic feedback module 133 and generate tactile outputs on device 100 that are capable of being sensed by a user of device 100. In some embodiments, at least one tactile output generator is collocated with, or proximate to, a touch sensitive surface (e.g., touch sensitive display system 112) and, optionally, generates a tactile output by moving the touch sensitive surface vertically (e.g., in / out of a surface of device 100) or laterally (e.g., back and forth in the same plane as a surface of device 100). In some embodiments, at least one tactile output generator sensor is collocated with, or proximate to, the back of the device 100, opposite to where touch sensitive display system 112 is located on the device 100.

[0055] Device 100 optionally also includes one or more accelerometers 168. FIG. 1A Accelerometer 168 is coupled to peripherals interface 118. Alternatively, accelerometer 168 is coupled to an input controller 160 in I / O subsystem 106. In some embodiments, information is displayed on the touch screen display in a portrait view or a landscape view based on a determination of the orientation of device 100 using the accelerometer 168. Device 100 optionally includes a magnetometer (not shown) and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information concerning the location and orientation (e.g., portrait or landscape) of device 100.

[0056] In some embodiments, software components stored in memory 102 include an operating system 126, a communication module (or set of instructions) 128, a contact / motion module (or set of instructions) 130, a graphics module (or set of instructions) 132, a tactile feedback module (or set of instructions) 133, a text input module (or set of instructions) 134, a Global Positioning System (GPS) module (or set of instructions) 135, and applications (or sets of instructions) 136. Moreover, in some embodiments, memory 102 stores a device / global internal state 157, as shown in Figures 1A and 3. Device / global internal state 157 includes one or more of: active application state, indicating which applications, if any, are currently active; display state, indicating what, if anything, is currently being displayed by the touch screen display units 112; sensor state, including information obtained from the device's various sensors and input or control devices 116; and location and / or orientation state, indicating the current location and / or attitude of the device.

[0057] Operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates wireless communications between various hardware and software components.

[0058] Communication module 128 facilitates communication with other devices over one or more external ports 124 and also includes various software components for handling data received by RF circuitry 108 and / or external port 124. External port 124 (e.g., Universal Serial Bus (USB), FIREWIRE, etc.) is adapted for coupling directly to other devices or indirectly over a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as, or similar to, and / or compatible with, the 30-pin connector used in some models of the iPhone®, iPod Touch®, and iPad® devices. In some embodiments, the external port is a Lightning connector that is the same as, or similar to, and / or compatible with, the Lightning connector used in some models of the iPhone®, iPod Touch®, and iPad® devices. iPod and devices. In some embodiments, the external port is a Lightning connector that is the same as, or similar to, and / or compatible with, the Lightning connector used in some models of the iPhone®, iPod Touch®, and iPad® devices. iPod and devices. In some embodiments, the external port is a Lightning connector that is the same as, or similar to, and / or compatible with, the Lightning connector used in some models of the iPhone®, iPod Touch®, and iPad® devices.

[0059] Contact / motion module 130 optionally detects contact with touch-sensitive display system 112 (in conjunction with display controller 156) and other touch-sensitive devices (e.g., a trackpad or physical click wheel). Contact / motion module 130 includes various software components for performing various operations related to detection of contact (e.g., by a finger or a stylus) with touch-sensitive display system 112. Contact / motion module 130 includes software components for determining if contact has occurred (e.g., detecting a finger-down event), determining an intensity of the contact (e.g., a pressure or force of the contact or a substitute for a pressure or force of the contact), determining if there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-dragging events), and determining if the contact has ceased (e.g., detecting a finger-up event or a break in contact). Contact / motion module 130 receives contact data from the touch-sensitive surface. Determining movement of the point of contact, which is represented by a series of contact data, optionally includes determining speed (magnitude), velocity (magnitude and direction), and / or acceleration (a change in magnitude and / or direction) of the point of contact. These operations are, optionally, applied to single contact events (e.g., one finger events) or to multiple simultaneous contact events (e.g., "multitouch" events).

[0060] Contact / motion module 130 optionally detects contact events with objects, such as the contact 222 with the contacts module 142 on the touch-sensitive display 112. The

[0061] In some embodiments, detecting a finger tap gesture depends on detecting a length of time between a finger down event and a finger up event, but is independent of a strength of finger contact between the finger down event and the finger up event. In some embodiments, a tap gesture is detected in accordance with a determination that the length of time between the finger down event and the finger up event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether a strength of the finger contact during the tap reaches a given strength threshold (greater than a nominal contact detection strength threshold), such as a light press or deep press strength threshold. Thus, a finger tap gesture can satisfy a particular input criterion that does not require a characteristic strength of the contact to satisfy a given strength threshold to satisfy the particular input criterion. For clarity, finger contact in a tap gesture typically needs to satisfy a nominal contact detection strength threshold to detect a finger down event, below which contact would not be detected. Similar analysis applies to detecting tap gestures by stylus or other contact. Where a device is capable of detecting hovering finger or stylus contact above a touch-sensitive surface, the nominal contact detection strength threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0062] The same concepts apply in an analogous manner to other types of gestures. For example, a swipe gesture, a pinch gesture, an expand gesture, and / or a long press gesture can optionally be detected based on satisfying criteria that is independent of the intensity of contacts included in the gesture or that does not require one or more contacts performing the gesture to reach an intensity threshold in order to be recognized. For example, a swipe gesture is detected based on an amount of movement of one or more contacts; a pinch gesture is detected based on movement of two or more contacts toward each other; an expand gesture is detected based on movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of a contact on the touch-sensitive surface that has less than a threshold amount of movement. Thus, a statement that a particular gesture recognition criterion does not require a contact intensity to satisfy a corresponding intensity threshold in order to satisfy the particular gesture recognition criterion means that the particular gesture recognition criterion can be satisfied with contacts in a gesture that do not reach the corresponding intensity threshold, and also can be satisfied with one or more contacts in a gesture that reach or exceed the corresponding intensity threshold. In some embodiments, a tap gesture is detected based on a determination that a finger down event and a finger up event are detected within a predefined time period, regardless of whether the contact is above or below a corresponding intensity threshold during the predefined time period, and a swipe gesture is detected based on a determination that a contact moves more than a predefined amount, even if the contact is above the corresponding intensity threshold at the end of the contact movement. Even in implementations in which detection of a gesture is affected by the intensity of a contact performing the gesture (e.g., a device detects a long press more quickly when the intensity of the contact is above an intensity threshold, or a device delays detection of a tap input when the intensity of the contact is higher), detection of these gestures does not require the contact to reach a particular intensity threshold as long as the criteria for recognizing the gesture can be satisfied with a contact that does not reach the particular intensity threshold (e.g., even if the amount of time required to recognize the gesture changes).

[0063] In certain instances, the contact intensity threshold, the duration threshold, and the movement threshold are combined in various different combinations in order to create heuristic algorithms to distinguish between two or more different gestures for the same input element or region, such that multiple different interactions with the same input element can provide a richer set of user interactions and responses. The statement that a particular gesture recognition criterion does not require the intensity of the contact to satisfy a corresponding intensity threshold in order for the particular gesture recognition criterion to be satisfied does not preclude other intensity-dependent gesture recognition criteria from being simultaneously evaluated to identify other gestures that have criteria that are satisfied when the gesture includes a contact having an intensity above the corresponding intensity threshold. For example, in certain instances, a first gesture recognition criterion of a first gesture (which does not require the intensity of the contact to satisfy a corresponding intensity threshold in order for the first gesture recognition criterion to be satisfied) competes with a second gesture recognition criterion of a second gesture (which depends on a contact meeting a corresponding intensity threshold). In such a competition, if the second gesture recognition criterion of the second gesture is first satisfied, then the gesture is optionally not identified as satisfying the first gesture recognition criterion of the first gesture. For example, if the contact meets the corresponding intensity threshold before the contact moves a predefined movement amount, then a deep press gesture is detected instead of a swipe gesture. Conversely, if the contact moves the predefined movement amount before the contact meets the corresponding intensity threshold, then a swipe gesture is detected instead of a deep press gesture. Even in such instances, the first gesture recognition criterion of the first gesture still does not require the intensity of the contact to satisfy the corresponding intensity threshold in order for the first gesture recognition criterion to be satisfied, because if the contact remains below the corresponding intensity threshold until the end of the gesture (e.g., a swipe gesture with a contact that does not increase in intensity above the corresponding intensity threshold), the gesture would be identified as a swipe gesture by the first gesture recognition criterion. Thus, a particular gesture recognition criterion that does not require the intensity of the contact to satisfy a corresponding intensity threshold in order for the particular gesture recognition criterion to be satisfied will either (A) in certain instances, ignore the contact intensity relative to the intensity threshold (e.g., for a tap gesture) and / or (B) in certain instances, be unable to be satisfied if a set of competing intensity-dependent gesture recognition criteria (e.g., for a deep press gesture) identify the input as corresponding to an intensity-dependent gesture before the particular gesture recognition criterion identifies the input as corresponding to a gesture (e.g., for a long press gesture), in the sense that the particular gesture recognition criterion still depends on the contact intensity relative to the intensity threshold (e.g., for a long press gesture that competes for recognition with a deep press gesture).

[0064] Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual property) of graphics that are displayed. As used herein, the term "graphics" includes any object that can be displayed to a user, including, without limitation, text, web pages, icons (such as user-interface objects including soft keys), digital images, videos, animations, and the like.

[0065] In some embodiments, graphics module 132 stores data representing graphics to be used. Each graphic is, optionally, assigned a corresponding code. Graphics module 132 receives, from applications etc., one or more codes specifying graphics to be displayed, and then generates screen image data for a

[0066] Haptic feedback module 133 includes various software components for generating instructions used by tactile feedback controller 161 to induce tactile feedback to a user (e.g., instructions to vibrate haptic feedback controller 161 at different frequencies to produce different tactile sensations to user).

[0067] Text input module 134, which is, optionally, a component of graphics module 132, provides soft keyboards for entering text in various applications (e.g., contacts 137, e-mail 140, IM 141, browser 147, and any other application that needs text input).

[0068] GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to telephone 138 for use in location-based dialing; to camera 143 as picture / video metadata; and to applications such as weather widget, local yellow page widget, and map / navigation widget).

[0069] Applications 136, optionally, include the following modules (or sets of instructions):

[0070] • contacts module 137 (sometimes referred to as an address book or contact list);

[0071] • telephone module 138;

[0072] • video conference module 139;

[0073] • e-mail client module 140;

[0074] • instant messaging (IM) module 141;

[0075] • a fitness support module 142;

[0076] • a camera module 143 for still and / or video images;

[0077] • an image management module 144;

[0078] • a browser module 147;

[0079] • a calendar module 148;

[0080] • a widget module 149 that optionally includes one or more of the following widgets: a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets acquired by the user and widgets created by the user 149-6;

[0081] • a widget creator module 150 for forming the widgets 149-6 created by the user;

[0082] • a search module 151;

[0083] • a video and music player module 152, optionally including a video player module and a music player module;

[0084] • a memo module 153;

[0085] • a map module 154; and / or

[0086] • an online video module 155.

[0087] Examples of other application programs 136 that can optionally be stored in the memory 102 include other word processing applications, other image editing applications, a drawing application, a presentation application, a JAVA-enabled application, an encryption, a digital rights management, a voice recognition and a voice replication.

[0088] In conjunction with the touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the contacts module 137 includes executable instructions to manage an address book or contact list (e.g., the contacts module 137's application internal state 192 stored in the memory 102 or in memory 370) including: adding an name to the address book; deleting a name from the address book; associating a phone number, e-mail address, physical address or other information with a name; associating an image with a name; categorizing and sorting names; providing telephone numbers and / or e-mail addresses to initiate and / or facilitate communications by telephone 138, video conference 139, e-mail 140, or instant message 141; and so forth.

[0089] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, telephone module 138 includes executable instructions to

[0090] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conference module 139 includes executable instructions to initiate, conduct, and terminate a video conference between a user and one or more other participants in accordance with user instructions.

[0091] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, e-mail client module 140 includes executable instructions to create, send, receive, and manage e-mail in response to user instructions. In conjunction with image management module 144, e-mail client module 140 makes it very easy to create and send e-mails with still or video images taken with camera module 143.

[0092] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the instant messaging module 141 includes executable instructions to enter a sequence of characters corresponding to an instant message, modify previously entered characters, transmit a respective instant message (for example, using a Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for telephony-based instant messages or using XMPP, SIMPLE, Apple Push Notification Service (APNs) or IMPS for Internet-based instant messages), receive instant messages and view received instant messages. In some embodiments, transmitted and / or received instant messages optionally include graphics, photos, audio files, video files and / or other attachments as are supported in a MMS and / or an Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both telephony-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0093] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and video and music player module 152, the workout support module 142 includes executable instructions to create workouts (e.g., with time, distance, and / or calorie burning goals); communicate with workout sensors (in sports devices and smart watches); receive workout sensor data; calibrate sensors used to monitor a workout; select and play music for a workout; and display, store, and transmit workout data.

[0094] In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, the camera module 143 includes executable instructions to capture still images or video (including a video stream) and store them into memory 102, modify characteristics of a still image or video, and / or delete a still image or video from memory 102.

[0095] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, the image management module 144 includes executable instructions to arrange, modify (e.g., edit), or otherwise manipulate, label, delete, present (e.g., in a digital slide show or album), and store still and / or video images.

[0096] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, browser module 147 includes executable instructions to browse the Internet in accordance with user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0097] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, e-mail client module 140, and browser module 147, calendar module 148 includes executable instructions to create, display, modify, and store calendars and data associated with calendars (e.g., calendar entries, to-do lists, etc.) in accordance with user instructions.

[0098] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop widget module 149 is, optionally, a downloadable executable file (e.g., a Web-based application), an executable file providing a desktop interface, or an executable file providing a mobile application interface. In some embodiments, a desktop widget comprises an HTML file, a CSS file, and a JavaScript file. In some embodiments, a desktop widget includes an XML file and a JavaScript file (e.g., Yahoo! Widgets).

[0099] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, desktop widget creator module 150 includes executable instructions to create desktop widgets (e.g., turning a user-specified portion of a web page into a desktop widget).

[0100] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, the search module 151 includes executable instructions to search for text, music, sound, image, video, and / or other files in memory 102 that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0101] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow the user to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, and executable instructions to display, present or otherwise play back videos (e.g., on touch-sensitive display 112, or on an external, wireless-enabled display via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (trademark of Apple Inc.).

[0102] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, the notes module 153 includes executable instructions to create and manage notes, to-do lists, and the like in accordance with user instructions.

[0103] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, the map module 154 includes executable instructions to receive, display, modify, and store maps and data associated with maps (e.g., driving directions; data on stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0104] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, e-mail client module 140, and browser module 147, online video module 155 includes executable instructions that allow the user to access, browse, receive (e.g., by streaming and / or download), play back (e.g., on the touch screen 112, or on an external, wireless-enabled display via external port 124), send an e-mail with a link to a particular online video, and otherwise manage one or more online videos in one or more formats, such as H.264. In some embodiments, instant messaging module 141, rather than e-mail client module 140, is used to send a link to a particular online video.

[0105] Each of the above identified modules and applications correspond to a set of executable instructions for performing one or more functions described above and the methods described in this application (e.g., the computer-implemented methods described herein and other information processing methods). These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules are, optionally, combined or otherwise re-arranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 102 optionally stores additional modules and data structures not described above.

[0106] In some embodiments, device 100 is a device for which a predefined set of functions of the device are operated exclusively through the touch screen and / or touchpad. By using the touch screen and / or touchpad as the primary input control devices for operating device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on device 100 is optionally reduced.

[0107] The predefined set of functions that are operated exclusively through the touch screen and / or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by the user, navigates device 100 from any user interface displayed on device 100 to a main menu, home menu, or root menu. In such embodiments, the touchpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a touchpad.

[0108] FIG. 1B FIG. 17 is a block diagram illustrating example components of a device for event processing, in accordance with some embodiments. In some embodiments, memory 102 (FIG. 1) or memory 370 (FIG. 3) includes event classifier 170 (e.g., in operating system 126) and corresponding application programs 136-1 (e.g., any of the aforementioned application programs 136, 137-155, 380-390). FIG. 1A

[0109] ​The event sorter 170 receives event information and determines the application 136-1 to which events are directed, and an application view 191 within the application 136-1 that should be updated. The event sorter 170 includes an event monitor 171 and a event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192, that indicates to the event sorter 170 which one or more application views 191 are currently displayed on the touch-sensitive display 112 when the application is active or executing. In some embodiments, device / global internal state 157 is used by the event sorter 170 to determine which application(s) is currently active, and application internal state 192 is used to determine which application views 191 should be updated by the newly arrived event information.

[0110] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resume information to be used when the application 136-1 is resumed; user interface state information to be displayed or ready for use by the application 136-1; a state queue to enable the user to return to a previous state or view of the application 136-1; and a redo / undo queue of previous actions taken by the user.

[0111] The event monitor 171 receives event information from the peripheral interface 118. The event information includes information about sub-events (e.g., user touches on touch-sensitive display 112, motion sensor events such as free fall, and / or audio information such as song titles). The peripheral interface 118 transmits information it receives to the event sorter 170. The event sorter 170 receives information for a sub-event and determines the application 136-1 to which the sub-event pertains, as well as any application views 191 within the application 136-1 that should be updated. The event sorter 170 generates an event based on the sub-event information and the active application view 191, and provides the event to the event dispatcher module 174.

[0112] In some embodiments, the event monitor 171 sends a request to the peripheral interface 118 at pre-determined intervals. In response, the peripheral interface 118 transmits information about the event or state changes that have occurred since the last request. In other embodiments, a less frequent report is used. In other embodiments, events are not reported until they occur (i.e., the peripheral interface 118 waits to generate an event from the sub-event information unless or until a sub-event occurs).

[0113] In some embodiments, the event sorter 170 also includes a hit view determination module 172 and / or an active event recognizer determination module 173.

[0114] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides a software process for determining where within one or more views a sub-event has occurred. A view is composed of controls and other elements that a user can see on the display.

[0115] Another aspect of a user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the respective application) in which a touch is detected optionally corresponds to a programmed level within a programmed or view hierarchy of the application. For example, the lowest level view in which a touch is detected is optionally referred to as the hit view, and the set of events that are correctly entered are optionally determined based at least in part on the hit view of an initial touch that begins a touch-based gesture.

[0116] The hit view determination module 172 receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, the hit view determination module 172 identifies the lowest view in the hierarchy that should be handled for a sub-event. In most cases, the hit view is the lowest level view in which an initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once the hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0117] The active event recognizer determination module 173 determines which view or views within a view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, the active event recognizer determination module 173 determines that all views that include the physical location of a sub-event are actively engaged views, and thus determines that all actively engaged views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is entirely confined to an area associated with one particular view, higher views in the hierarchy will remain actively engaged views.

[0118] The event dispatcher module 174 dispatches event information to event recognizers (e.g., event recognizers 180). In embodiments that include an active event recognizer determination module 173, the event dispatcher module 174 delivers event information to event recognizers determined by the active event recognizer determination module 173. In some embodiments, the event dispatcher module 174 stores event information in an event queue that is retrieved by respective event receiver modules 182.

[0119] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application programs 136-1 include event classifier 170. In yet another embodiment, event classifier 170 is a standalone module, or is part of another module stored in memory 102, such as contact / motion module 130.

[0120] In some embodiments, application programs 136-1 include a plurality of event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events that occur within a respective view of the user interface of the application program. Each application view 191 of application programs 136-1 includes one or more event recognizers 180. Typically, a respective application view 191 includes multiple event recognizers 180. In other embodiments, one or more of event recognizers 180 are part of a higher level object, such as a user interface toolkit (not shown) or a higher level object from which application programs 136-1 inherit methods and other attributes. In some embodiments, a respective event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update the application internal state 192. Alternatively, one or more of application views 191 includes one or more respective event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in a respective application view 191.

[0121] A respective event recognizer 180 receives event information (e.g., event data 179) from event classifier 170 and recognizes events from the event information. Event recognizers 180 include event receiver 182 and event comparator 184. In some embodiments, event recognizers 180 also include at least a subset of metadata 183 and event delivery instructions 188, which optionally include sub-event delivery instructions.

[0122] The event receiver 182 receives event information from the event classifier 170. The event information includes information about a sub-event, e.g., a touch or a touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves motion of a touch, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., rotating from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0123] The event comparator 184 compares the event information to predefined event or sub-event definitions, and determines the event or sub-event, or determines or updates the state of the event or sub-event, based on the comparison. In some embodiments, the event comparator 184 includes event definitions 186. The event definitions 186 contain definitions of events (e.g., predefined sequences of sub-events), e.g., event 1 (187-1), event 2 (187-2), and others. In some embodiments, the sub-events in the events 187 include, e.g., touch down, touch up, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double tap on a displayed object. For example, a double tap includes a first touch (touch down) on a displayed object for a predetermined duration, a first lift off (touch up) for a predetermined duration, a second touch (touch down) on the displayed object for a predetermined duration, and a second lift off (touch up) for a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, a drag includes a touch (or contact) on a displayed object for a predetermined duration, movement of the touch on the touch-sensitive display system 112, and lift off of the touch (touch up). In some embodiments, the events also include information for one or more associated event handlers 190.

[0124] In some embodiments, the event definitions 187 include definitions of events for respective user interface objects. In some embodiments, the event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view that displays three user interface objects on the touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, the event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a respective event handler 190, the event comparator uses the results of the hit test to determine which event handler 190 should be activated. For example, the event comparator 184 selects the event handler that is associated with the sub-event and the object that triggered the hit test.

[0125] In some embodiments, the definition of a respective event 187 also includes deferred actions that delay delivery of event information until it has been determined that a sequence of sub-events does or does not correspond to an event type of the event recognizer.

[0126] When a respective event recognizer 180 determines that a sequence of sub-events does not match any event in the event definitions 186, the respective event recognizer 180 enters an event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0127] In some embodiments, a respective event recognizer 180 includes metadata 183 with configurable attributes, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery to actively engaged event recognizers. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists that indicate how event recognizers interact or are able to interact with each other. In some embodiments, the metadata 183 includes configurable attributes, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0128] In some embodiments, when one or more particular sub-events of an event are recognized, the respective event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the respective event recognizer 180 delivers event information associated with the event to the event handler 190. Activating an event handler 190 is distinct from sending (and deferring sending) sub-events to a respective hit view. In some embodiments, an event recognizer 180 throws a token associated with a recognized event, and an event handler 190 associated with the token picks up the token and performs a pre-defined procedure.

[0129] In some embodiments, the event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver event information to an event handler associated with the sequence of sub-events or to an actively engaged view. The event handler associated with the sequence of sub-events or with the actively engaged view receives the event information and performs a pre-determined procedure.

[0130] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates the telephone number used in contacts module 137, or stores a video file used by the video and music player module 152. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates a new user-interface object or updates the position of a user-interface object. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends it to graphics module 132 for display on the touch screen.

[0131] In some embodiments, event handler 190 includes data updater 176, object updater 177, and GUI updater 178, or has access to them and utilizes them as needed. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in the single module of application 136-1 or application manager 191. In other embodiments, they are included in two or more software modules.

[0132] It should be understood that the above discussion regarding event handling of user touches on touch-sensitive displays also applies to other forms of user inputs to multifunctional devices 100 not involving touch. For example, mouse input, keyboard or keypad input, telephone receiver or keypad input, etc. can be used instead of touch input. The multifunctional device 100 can include one or more physical buttons, such as, for example, push buttons, rocker buttons, etc. that are used in combination with the touch screen, for example, as described in U.S. Patent No. 6,301,021, which is incorporated by reference herein in its entirety.

[0133] FIG. 2 Figure 1A shows a portable multifunctional device 100 having a touch screen in accordance with some embodiments. Figure 1B shows a portable multifunctional device 100 having front and back touch screens in accordance with some embodiments. Figures 1C and 1D show portable multifunctional devices 100 and 100' in accordance with some embodiments. FIG. 1AA portable multi-functional device 100 (a touch-sensitive display system 112) is described. The touchscreen optionally displays one or more graphics within a user interface (UI) 200. In these embodiments and other embodiments described below, a user can select one or more graphics by gesturing over the graphics, for example, using one or more fingers 202 (not drawn to scale in the figures) or one or more styluses 203 (not drawn to scale in the figures). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the graphics. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or scrolling (from right to left, from left to right, up and / or down) of a finger already in contact with the device 100. In some embodiments or in some cases, unintentional contact with a graphic does not select the graphic. For example, a swipe gesture over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0134] Device 100 optionally also includes one or more physical buttons, such as a "home button" or menu button 204. As previously described, menu button 204 is optionally used to navigate to any application 136 of a set of applications optionally executed on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display.

[0135] In some embodiments, device 100 includes a touchscreen display, a menu button 204 (sometimes referred to as a home button 204), a push-button 206 for powering on / off the device and locking the device, a volume control button 208, a SIM card slot 210, a headphone jack 212, and a docking / charging external port 124. The push-button 206 is optionally used to power on / off the device by pressing the button and holding it in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing it before the predefined time interval has elapsed; and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts voice input via microphone 113 for activating or deactivating certain functions. Device 100 also optionally includes one or more contact strength sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more haptic output generators 167 for generating haptic outputs for a user of device 100.

[0136] FIG. 3Ais a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an education device (such as a child's learning toy), a gaming system, or a control device (e.g., a home- or industrial- control panel). Device 300 typically includes one or more processing units (CPU's) 310, one or more network or other communications interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication buses 320 optionally include circuitry (sometimes called a chipset) that interconnects and controls communications between system components. Device 300 includes input / output (I / O) interface 330 comprising display 340, which is typically a touch screen display. In some embodiments, display 340 is referred to as a display generation component. I / O interface 330 also optionally includes keyboard and / or mouse (or other pointing device) 350 and touchpad 355, tactile output generation FIG. 1A one or more tactile output generators 167 described above with reference to FIG. 1A touch intensity sensors 165 described above with reference to FIG. 1A Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. Memory 370 optionally includes one or more storage devices remotely located from CPU(s) 310. In some embodiments, memory 370 stores programs, modules, and data structures similar to the programs, modules, and data structures stored in memory 102 of portable multifunction device 100 FIG. 1A In some embodiments, memory 370 stores additional programs, modules, and data structures not present in the memory 102 of portable multifunction device 100 FIG. 1A In some embodiments, memory 370 stores additional programs, modules, and data structures not present in the memory 102 of portable multifunction device 100

[0137] FIG. 3AEach of the above identified elements can optionally be stored in one or more of the aforementioned memory devices, and corresponds to a set of instructions for performing a function described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules can optionally be combined or otherwise re-arranged in various embodiments. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. Furthermore, memory 370 optionally stores additional modules and data structures not described above.

[0138] FIG. 3B is a block diagram of an example wearable audio output device 301 in accordance with some embodiments. In some embodiments, the wearable audio output device 301 is one or more in-ear earphones, earbuds, over-ear headphones, etc. In some examples, the wearable audio output device 301 is a single earphone or earbud. In some examples, the wearable audio output device 301 includes a pair of earphones or earbuds (e.g., one earphone or earbud for each ear of a user). In some examples, the wearable audio output device 301 includes over-ear headphones (e.g., headphones with two over-ear earcups to be placed over the ears of a user and optionally connected by a headband). In some embodiments, the wearable audio output device 301 includes one or more speakers 306 for providing audio output (e.g., to the ears of a user). In some embodiments, the wearable audio output device 301 includes one or more pose sensors 304 (e.g., including an accelerometer and / or a gyroscope) to detect a pose (e.g., a position and / or an orientation) and a change in pose of the wearable audio output device 301 or a user (e.g., a wearer) of the wearable audio output device 301 relative to a physical environment. In some embodiments, the wearable audio output device 301 conditionally outputs audio based on a pose of the wearable audio output device 301 determined by the pose sensor 304. In some embodiments, the audio I / O logic 312 determines a pose of the wearable audio output device 301 or a wearer of the wearable audio output device 301, and in some embodiments, the audio I / O logic 312 controls the resulting conditional audio output.

[0139] In some embodiments, the wearable audio output device 301 includes one or more microphones 302 for receiving audio input. In some embodiments, the microphone 302 detects speech from a user wearing the wearable audio output device 301 and / or ambient noise around the wearable audio output device 301. In some embodiments, as described herein with reference to FIG. 3, the microphone 302 detects speech from a user wearing the wearable audio output device 301 and / or ambient noise around the wearable audio output device 301. FIG. 3CIn more detail, the plurality of microphones of the microphone(s) 302 are positioned at different locations on the wearable audio output device 301 to measure speech and / or ambient noise at different locations around the wearable audio output device 301. In some embodiments, the audio I / O logic 312 detects or identifies speech or ambient noise based on information received from the microphone(s) 302.

[0140] In some embodiments, the wearable audio output device 301 includes one or more other input devices 308, such as a touch-sensitive surface (to detect touch input), one or more placement sensors to detect positioning or placement of the wearable audio output device 301 relative to a user’s ear (such as to detect placement of the wearable audio output device 301 in or on a user’s ear), and / or other input devices by which a user can interact with and provide input to the wearable audio output device 301. In some embodiments, input provided via the one or more input devices 308 is processed by the audio I / O logic 312. In some embodiments, the audio I / O logic 312 is in communication with a separate device (e.g., the device 100 of FIG. 1A or the device 300 of FIG. 3A ) that provides instructions or content for audio output, and optionally receives and processes input (or information about input) provided via the microphone(s) 302, the posture sensor(s) 304, and / or the input devices 308 or via one or more input devices of a separate device. In some embodiments, the audio I / O logic 312 is located in the device 100 (e.g., as part of the peripheral device interface 118 of FIG. 1A ) or the device 300 (e.g., as part of the I / O interface 330 of FIG. 3A ), rather than in the device 301, or alternatively is partially located in the device 100 and partially located in the device 301, or partially located in the device 300 and partially located in the device 301.

[0141] FIG. 3C Exemplary audio control by a wearable audio output device is shown, in accordance with some embodiments. In some embodiments, when a wearable audio output device with over-ear earcups is worn on a user’s ear, the earcups act as a physical barrier that blocks at least some ambient sound from the surrounding physical environment from reaching the user’s ear. For example, in FIG. 3C , the wearable audio output device 301 is worn by a user such that the earcups 314 are on the user’s left ear. In some embodiments, (e.g., as shown in FIG. 3BA first microphone (or, in some embodiments, a first set of one or more microphones) 302-1 of the microphones 302 of the wearable audio output device 301 is positioned on the wearable audio output device 301 so as to detect environmental sound represented by waveform 322 in a region 316 of the physical environment surrounding the ear cover 314 (e.g., outside the ear cover). In some embodiments, the ear cover 314 blocks some, but not necessarily all, environmental sound in the ambient physical environment from reaching the user's ear. In some embodiments, the first microphone 302-1 is positioned on the wearable audio output device 301 so as to detect environmental sound in a region 316 of the physical environment surrounding the ear cover 314 (e.g., outside the ear cover). In some embodiments, the first microphone 302-1 is positioned on the wearable audio output device 301 so as to detect environmental sound in a region 316 of the physical environment surrounding the ear cover 314 (e.g., outside the ear cover). FIG. 3B A second microphone (or, in some embodiments, a second set of one or more microphones) 302-2 of the microphones 302 of the wearable audio output device 301 is positioned on the wearable audio output device 301 so as to detect any environmental sound represented by waveform 324 that is not completely blocked by the ear cover 314 and that is audible in a region 318 within the ear cover 314. Thus, in some cases in which the wearable audio output device 301 does not produce a noise-cancelling (also referred to as "anti-phase") audio signal to cancel out (e.g., attenuate) environmental sound from the ambient physical environment (as indicated by waveform 326-1), the environmental sound waveform 324 can be perceived by the user (as indicated by waveform 328-1). In some cases in which the wearable audio output device 301 produces an anti-phase audio signal to cancel out environmental sound (as indicated by waveform 326-2), the environmental sound waveform 324 can not be perceived by the user (as indicated by waveform 328-2).

[0142] In some embodiments, the environmental sound waveform 322 is compared to the attenuated environmental sound waveform 324 (e.g., by the wearable audio output device 301 or a component of the wearable audio output device 301 such as the audio I / O logic 312, or by an electronic device in communication with the wearable audio output device 301) to determine the amount of passive attenuation provided by the wearable audio output device 301. In some embodiments, the amount of passive attenuation provided by the wearable audio output device 301 is taken into account when providing an anti-phase audio signal to cancel out environmental sound from the ambient physical environment. For example, the anti-phase audio signal waveform 326-2 is configured to cancel out the attenuated environmental sound waveform 324, rather than the unattenuated environmental sound waveform 322.

[0143] Attention is now directed towards embodiments of user interfaces ("UI") that are optionally implemented on portable multifunction devices 100.

[0144] FIG. 4A An example user interface that shows an application menu on a portable multifunction device 100 in accordance with some embodiments is shown. Similar user interfaces are optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0145] • one or more signal strength indicators for one or more wireless communications such as cellular signals and Wi-Fi signals;

[0146] • time;

[0147] • a Bluetooth indicator;

[0148] • a battery status indicator;

[0149] • a tray 408 with icons for commonly used applications, such as:

[0150] o an icon 416 for the phone module 138 labeled "Phone," which optionally includes an indicator 414 of the number of missed calls or voicemails;

[0151] o an icon 418 for the email client module 140 labeled "Mail," which optionally includes an indicator 410 of the number of unread emails;

[0152] o an icon 420 for the browser module 147 labeled "Browser"; and

[0153] o an icon 422 for the video and music player module 152 labeled "Music";

[0154] and

[0155] • icons for other applications, such as:

[0156] o an icon 424 for the IM module 141 labeled "Messages";

[0157] o an icon 426 for the calendar module 148 labeled "Calendar";

[0158] o an icon 428 for the image managing module 144 labeled "Photos";

[0159] o an icon 430 for the camera module 143 labeled "Camera";

[0160] o an icon 432 for the online video module 155 labeled "Online Videos";

[0161] o an icon 434 for the stock market widget 149-2 labeled "Stocks";

[0162] o an icon 436 for the map module 154 labeled "Map";

[0163] o an icon 438 for the weather widget 149-1 labeled "Weather";

[0164] o an icon 440 for the clock widget 149-4 labeled "Clock";

[0165] o an icon 442 labeled "Workout Support" for the workout support module 142;

[0166] o an icon 444 labeled "Notes" for the notes module 153; and

[0167] o an icon 446 for setting up applications or modules that provides access to settings for the device 100 and its various applications 136.

[0168] It should be noted that FIG. 4A the icon labels shown in FIG. 4B are merely exemplary. For example, other labels are optionally used for the various application icons. In some embodiments, the label for a respective application icon includes the name of the application that corresponds to the respective application icon. In some embodiments, the label for a particular application icon is different from the name of the application that corresponds to the particular application icon.

[0169] FIG. 4B An exemplary user interface on a device (e.g., device 300 in FIG. 3) with a touch- sensitive surface 451 (e.g., trackpad or touchpad 355 in FIG. 3) separate from the display 450 is shown. Although in some examples, input can be received on a touch screen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, as shown in FIG. 4B FIG. 4C. In some embodiments, the touch-sensitive surface (e.g., 451 in FIG. 4B) has a principal axis (e.g., 452 in FIG. 4B) that corresponds to a principal axis (e.g., 453 in FIG. 4C) on the display (e.g., 450). According to these embodiments, the device detects contacts with the touch-sensitive surface 451 at locations that correspond to respective locations on the display (e.g., 460 and 462 in FIG. 4C) (e.g., 460 corresponds to 468 and 462 corresponds to 470 in FIG. 4C). In this way, when the touch-sensitive surface (e.g., 451 in FIG. 4B) is separate from the display (e.g., 450) of a multifunction device, user inputs (e.g., contacts 460 and 462 and movements thereof) that are detected by the device on the touch-sensitive surface are used by the device to manipulate user interfaces on the display. It should be understood that similar methods are optionally used for other user interfaces described herein. FIG. 4B FIG. 4B FIG. 4B FIG. 4B FIG. 4B FIG. 4B User Interface and Associated Processes

[0170] FIG. 5A through FIG. 5L

[0171] ​​​​​​​Attention is now directed to embodiments of user interfaces ("UIs") and associated processes that can be implemented on an electronic device such as portable multifunction device 100 or device 300 with a display, a touch-sensitive surface, (optional) one or more tactile output generators for generating tactile outputs, and (optional) one or more sensors for detecting intensities of contacts with the touch-sensitive surface.

[0172] FIG. 6A through FIG. 6J Exemplary adaptive audio output in response to exemplary user interactions with a wearable audio output device is shown in accordance with some embodiments. FIG. 7A through FIG. 7B Exemplary adaptive audio output in response to exemplary changes in audio properties of a surrounding physical environment is shown in accordance with some embodiments. The audio output, user interactions, and environmental audio changes in these figures are used to illustrate processes described below, including the processes in FIG. 8A through FIG. 8B and FIG. 5A through FIG. 5L For ease of explanation, some of the embodiments will be discussed with reference to operations performed using a wearable audio output device that is worn by a user and that is in communication with an electronic device that has a touch-sensitive display system 112 or display 340 separate from a touch-sensitive input device such as touchpad 355. In some embodiments, the operations are performed in response to instructions received by the wearable audio output device from the electronic device based on processing performed at the electronic device. In some embodiments, the operations are performed by the wearable audio output device based on processing performed at the wearable audio output device. However, in some cases, similar operations are optionally performed using an audio output device that is part of a device that has a display generation component and / or a touch-sensitive input device (e.g., a wearable device that integrates one or more audio output devices with a display and / or a touch-sensitive input device, such as a headset or earphones).

[0173] FIG. 5A through FIG. 5E Exemplary adaptive audio output in response to exemplary user interactions with a wearable audio output device, including changes in user posture, is shown in accordance with some embodiments.

[0174] FIG. 5A Changes in audio output provided by a headset worn by a user in response to changes in posture of the user between a seated position and a standing position are shown. FIG. 5B through FIG. 5C An expanded view 501 shows the headset 504 playing music so that the music sounds as if it is coming from an area of simulated space 506 around the head of the user 502.

[0175] FIG. 5A Changes in audio output provided by a headset worn by a user in response to changes in posture of the user between a seated position and a standing position are shown. FIG. 5Btransition. In particular, FIG. 5B The expanded view 501 in FIG. 5A illustrates the headset 504 moving the simulated spatial location of the music being played so that the music sounds as if it is coming from a simulated spatial location 508 above the head of the user 502 that is different from a simulated spatial region 506 surrounding the head of the user 502 and reducing the volume at which the music is played in response to the change in posture of the user 502 from sitting FIG. 5B ) to standing FIG. 5A .

[0176] In some embodiments, the headset 504 fades out the music playback entirely in response to the change in posture of the user 502 from sitting FIG. 5B ) to standing FIG. 5C . Thus, FIG. 5B illustrates an optional further transition in which the headset 504 stops playing the music entirely (e.g., the music volume is reduced to zero or paused / stopped) after the simulated spatial location of the music is moved above the head of the user 502 and after the volume at which the music is played is reduced (as shown in FIG. 5C and as described above), as shown in the expanded view 501 in FIG. 5B through FIG. 5C .

[0177] In other words, in FIG. 5A , in response to standing up from a seated position (or, in other embodiments, in response to sitting up or standing up from a reclined or lying down position), the user 502 is provided with an immersive less and thus less obstructive audio experience to make it easier for the user 502 to interact with their surrounding physical environment as the change in posture can indicate that the user 502 wants to interact with their surrounding physical environment. In contrast, in FIG. 5D through FIG. 5E , the user 502 is provided with an immersive more audio experience while seated (or, in other embodiments, while reclined or lying down) in which case the user 502 can be less likely to interact with their surrounding physical environment.

[0178] FIG. 5B through FIG. 5C illustrates a transition from FIG. 5D . In particular, FIG. 5A illustrates the user 502 having resumed the same seated posture as in FIG. 5C . In response to the change in posture of the user 502 from standing FIG. 5D ) to sitting FIG. 5A , the headset 504 resumes playing the music, as shown in FIG. 5D through FIG. 5E . In some embodiments, as shown in FIG. 5D , for example, by initially playing the music at the simulated spatial location 508 above the head of the user 502 (e.g., at the simulated spatial location 508 above the head of the user 502), the headset 504 resumes playing the music at the same simulated spatial location 508 above the head of the user 502 at which the music was playing before the change in posture of the user 502 from sitting to standing.FIG. 5E And then, moving the analog spatial position of the music, the headphones 504 gradually resume music playback, making the music sound as if it were coming from the analog spatial area 506 around the user's head 502. FIG. 5A Furthermore, in some implementations, the headphones 504 gradually increase the music volume from zero to an initial level relative to the volume. FIG. 5D The original volume shown is reduced to a smaller volume (e.g.) FIG. 5A (as shown in the expanded view 501), and later enlarged to FIG. 5E The original volume shown (e.g.) FIG. 5C (As shown in extended view 501). In some embodiments, the analog spatial position of the music gradually changes in conjunction with the gradual increase in music volume. In some embodiments where the headphones 504 do not respond to changes in user posture and completely fade out of music playback (e.g., in...), FIG. 5B In an implementation where the scenario shown does not occur, user 502 returns to a seated position, causing... FIG. 5E The scene changed to FIG. 5D The scenario without first through FIG. 5F through FIG. 5G The intermediate scene transition.

[0179] FIG. 5F The audio output provided by the headset 504 is shown to change in response to changes in the posture of the user 502, which includes movement between different types of spaces. FIG. 5F The image shows user 502 standing in public space 510. FIG. 5G The extended view 501 shows that when user 502 is standing in public space 510, headphones 504 quietly play music and make the music sound as if it is coming from a simulated spatial location 508 above user 502's head. FIG. 5G This indicates that user 502 has moved to private space 512. FIG. 5F The extended view 501 shows that, in response to user 502 moving from public space 510 to private space 512, headphones 504 increase the volume of music playback and move the analog spatial position of the music so that the music sounds as if it is coming from the analog spatial area 506 around user 502's head.

[0180] In other words, in FIG. 5G In this context, a less immersive and therefore less obstructive audio experience is provided to user 502 in a public space (e.g., public space 510), making it easier for user 502 to interact with their surrounding physical environment. In contrast, in... FIG. 5H through FIG. 5L In this context, a more immersive audio experience is provided to user 502 in a private space (e.g., private space 512), where user 502 is unlikely to interact with their surrounding physical environment.

[0181] FIG. 5H Changes in audio output provided by headphones 504 worn by user 502 are shown, as well as changes in visual feedback provided on a display of an electronic device in communication with headphones 504 in response to changes in the posture of user 502 between a reclined position and a seated position. FIG. 5H Reclined position. Expanded view 501 shows headphones 504 playing music such that the music sounds as if it is coming from an analog spatial region 506 around the head of user 502. Expanded view 503 shows a view of physical environment 500 from the perspective of user 502, including a view of device 514 and a portion of plant 518 visible behind device 514. Device 514 displays a full-screen user interface for playing video 516. In FIG. 5I The example shown, the music played by headphones 504 is the audio track for the video content of video 516.

[0182] FIG. 5J It is shown that, as user 502 continues to watch video content 516, notification 520 is displayed on device 514. FIG. 5K It is shown that, in response to notification 520, user 502 sits up, changing the posture from reclined to upright seated.

[0183] FIG. 5J It is shown that, in accordance with some embodiments, a transition from FIG. 5K is made. In FIG. 5B through FIG. 5C response to the change in posture of user 502 from reclined to upright seated, the level of immersion relative to both video 516 displayed on device 514 and the music provided by headphones 504 is reduced. In particular, device 514 pauses playback of video 516, reduces the scale of video 516 such that video 516 is displayed in an inset window (e.g., while paused), and moves the inset window to a corner of the display of device 514. Device 514 also displays a live view 522 of physical environment 500 (e.g., from one or more cameras of device 514). In addition, headphones 504 move the analog spatial location of the music such that the music sounds as if it is coming from an analog spatial location 508 above the head of user 502, and reduce the volume at which the music is played. In some embodiments, headphones 504 fade out music playback entirely, as described herein with reference to FIG. 5L .

[0184] FIG. 5J It is shown that, in accordance with some embodiments, an alternative transition from FIG. 5K is made. As with FIG. 5L , in FIG. 5LIn response to user 502 changing their posture from reclining to sitting upright, playback of video 516 is partially paused via device 514, and the simulated spatial position of the music is moved via headphones 504. This reduces the immersion level of both the video 516 displayed on device 514 and the music provided by headphones 504, making the music sound as if it's coming from a simulated spatial position 508 above user 502's head, and also reduces the volume of the music. However, in FIG. 6A through FIG. 6J Instead of shrinking video 516 to an embedded window and moving video 516 to a corner of the display, device 514 increases the transparency of video 516 and displays at least a portion of a live view 522 of the physical environment 500 (e.g., from one or more cameras of device 514), making the live view of the physical environment 500 appear as if it were being viewed through the partially transparent video 516. Furthermore, device 514 displays beacons 524-1 and 524-2, respectively, highlighting user 502's wallet 526 and key 528, to help user 502 respond to notification 520 (e.g., when reminding user 502 to leave for a meeting instructed by notification 520 regarding items to take).

[0185] FIG. 6A through FIG. 6D An exemplary adaptive audio output is shown, according to some implementations, in response to exemplary changes in the audio properties of the surrounding physical environment.

[0186] FIG. 6A The diagram illustrates the variation in audio output provided by a user-worn headset in response to the increase and decrease of ambient sounds (particularly speech) in the user's surrounding physical environment. FIG. 1A The illustration shows a user 602 seated and wearing a headset 604 in a physical environment 600. The headset 604 is connected to electronic devices (e.g., FIG. 3C The device 100 communicates with the device and provides an audio output corresponding to the audio content represented by the device content waveform 606 from the device. Ambient sounds present in the physical environment 600 are represented by the ambient sound waveform 608. Sounds from the physical environment 600 are not completely blocked by the headset 604 (e.g., due to the imperfect passive attenuation of the headset 604, as referenced herein). FIG. 6A The ambient sound that can be heard inside the earcups of the headphones 604 is represented by an attenuated ambient sound waveform 610. FIG. 6B through FIG. 6CIn the illustrated example, the ambient sound present in the physical environment 600 is deemed to satisfy the predefined audio output criteria (e.g., the audio output criteria associated with providing device audio content at, for example, a volume level selected by the user 602 or automatically determined by the headphones 604 or by a device in communication with the headphones 604). In addition, active noise control is enabled for the headphones 604, and thus a noise cancellation signal represented by the inverted waveform 612 is provided to cancel the attenuated ambient sound waveform 610. As a result of the active noise control, the user 602 does not perceive the ambient sound, as indicated by the perceived ambient sound waveform 614.

[0187] FIG. 6A A transition from FIG. 6B is shown. In particular, FIG. 6B through FIG. 6C Another person 616 is shown to have approached the user 602 and to have begun speaking with the user 602. The increase in ambient sound in the physical environment 600 due to the person 616 speaking is shown by the ambient sound waveform 608 and the attenuated ambient sound waveform 610. As a result, the ambient sound present in the physical environment 600 is no longer deemed to satisfy the predefined audio output criteria (e.g., the audio output criteria associated with providing device audio content). In some embodiments, FIG. 6C The ambient sound present in the physical environment 600 in is deemed to satisfy a different, second predefined audio output criteria (e.g., the audio output criteria associated with a bias toward ambient sound). Initially, after the person 616 begins speaking, the headphones 604 continue to output audio content from the device, as shown by the device content waveform 606, and also continue to cancel ambient sound, as shown by the inverted waveform 612. As a result, the user 602 continues to hear the device content and not the ambient sound.

[0188] FIG. 6C A modification of the audio output provided by the headphones 604 in response to detecting an increase in ambient sound in the physical environment 600 (here, due to the person 616 speaking) and in response to the ambient sound in the physical environment 600 no longer satisfying the predefined audio output criteria is shown. FIG. 6B The device content waveform 606 in shows a decrease in the volume level of the audio content from the device relative to the previous FIG. 6C shown by the device content waveform 606 in . In addition, active noise control is disabled, as shown by the inverted waveform 612 in FIG. 3C Moreover, the headphones 604 begin to actively pass through ambient sound from the physical environment 600 (e.g., using a microphone, such as the microphone 616, to detect the ambient sound and to provide a signal to the headphones 604 to pass through the ambient sound). FIG. 6Dthe headset 604 (e.g., to help a hard-of-hearing user). In some embodiments, the headset 604 amplifies the ambient sound from the physical environment 600 so that the user 602 hears ambient sound at a volume level that is higher than the volume level of the ambient sound in the physical environment 600 (e.g., to help a hard-of-hearing user).

[0189] FIG. 6C a transition from FIG. 6B to FIG. 6C is shown. In particular, FIG. 6D the user 602 has stopped speaking and left the physical environment 600. Thus, the ambient sound present in the physical environment 600 (as shown by the ambient sound waveform 608) and the attenuated ambient sound (as shown by the attenuated ambient sound waveform 610) return to the same volume levels as shown in FIG. 6B. Thus, the ambient sound present in the physical environment 600 again satisfies the predefined audio output criteria. In some embodiments, the ambient sound present in the physical environment 600 in FIG. 6B is deemed to no longer satisfy the second predefined audio output criteria described with respect to FIG. 6B. In response, the headset 604 resumes playback of the audio content from the device at the same volume level as in FIG. 6B, as shown by the device content waveform 606 in FIG. 6C. The headset 604 also resumes active noise control, and thus resumes generating the noise cancellation signal, as shown by the inverted waveform 612, to cancel the attenuated ambient sound. As a result of the resumed active noise control, the user 602 does not perceive the ambient sound, as indicated by the perceived ambient sound waveform 614. FIG. 6A FIG. 6D FIG. 6B through FIG. 6C FIG. 6A FIG. 6D FIG. 6E

[0190] FIG. 6E ​​​​​​An example scenario is shown in which the user 602 is not listening to audio content from the device (e.g., when using the headphones 604 only for noise reduction and not for listening to audio content). Environmental sounds present in the physical environment 600 are shown by the environmental sound waveform 608, and attenuated environmental sounds inside the headphones 604 (e.g., environmental sounds that are not blocked by the headphones 604 from the physical barrier of the user’s 602 ears) are shown by the attenuated environmental sound waveform 610. In FIG. 6A , because the user 602 is not listening to audio content from the device, as shown by the device content waveform 606, the balance between the device audio content and the environmental sounds from the physical environment 600 is more biased toward the environmental sounds and less biased toward the device audio content than in FIG. 6A . Thus, instead of generating an anti-phase signal that completely cancels out the attenuated environmental sound waveform 610 (e.g., as shown by the anti-phase waveform 612), FIG. 6E the headphones 604 in FIG. 6F through FIG. 6J generate an anti-phase signal that only partially cancels out the attenuated environmental sound waveform 610 (shown by the anti-phase waveform 612). Thus, the user 602 perceives some but not all of the attenuated environmental sounds, as shown by the perceived environmental sound waveform 614 (e.g., the user 602 perceives some environmental sounds, but at a volume level that is even lower than the volume level of the attenuated environmental sounds).

[0191] FIG. 6F An example scenario is shown in which the user 602 is not listening to audio content from the device (e.g., when using the headphones 604 only for noise reduction and not for listening to audio content). Environmental sounds present in the physical environment 600 are shown by the environmental sound waveform 608, and attenuated environmental sounds inside the headphones 604 (e.g., environmental sounds that are not blocked by the headphones 604 from the physical barrier of the user’s 602 ears) are shown by the attenuated environmental sound waveform 610. In FIG. 6A , the user 602 is seated in the physical environment 600 and wearing the headphones 604, as in FIG. 6F . The user 602 is listening to device audio content, as shown by the device content waveform 606. Environmental sounds present in the physical environment 600 are shown by the environmental sound waveform 608, and attenuated environmental sounds inside the headphones 604 are shown by the attenuated environmental sound waveform 610. The headphones 604 also provide a noise cancellation signal, shown by the anti-phase waveform 612, to cancel out the attenuated environmental sound waveform 610. As a result of the active noise control, the user 602 does not perceive the environmental sounds, as indicated by the perceived environmental sound waveform 614. In addition, FIG. 6G through FIG. 6J , the user 602 provides an input 618 (e.g., a tap gesture) via the headphones 604 at time t = 0, as indicated by the timer 620. As described in more detail herein with reference to FIG. 6G , the input 618 initiates a temporary audio output state that lasts until time t = T th , as indicated by the timer 620 (e.g., for a duration T th ).

[0192] FIG. 6F It shows from FIG. 6G The transformation. In FIG. 6G In response to the detection of input 618 via the headset 604, the headset 604 switches to a temporary audio output state. FIG. 6H In the temporary audio output state shown, the volume level of the device audio content is reduced, as shown in device content waveform 606, and active noise control is disabled, as shown in inverted waveform 612. Furthermore, the headphones 604 actively allow ambient sound from the physical environment 600 to pass through, so that the user 602 hears ambient sound at a volume level higher than the attenuated ambient sound level produced by the passive attenuation of the headphones 604 and approximately equal to the actual volume level of the ambient sound in the physical environment 600 (e.g., within 10%, 15%, or 20%), as shown in perceived ambient sound waveform 614.

[0193] FIG. 6G It shows from FIG. 6H The transformation. In FIG. 6I In the middle, another person 622 has approached user 602 and is speaking to user 602. The increase in ambient sound in the physical environment 600 caused by person 622 speaking is shown by ambient sound waveform 608 and attenuation ambient sound waveform 610. Because the headset 604 is in a temporary audio output state (e.g., because t <T th As indicated by timer 620, the headset 604 continues to output audio content from the device at a reduced volume level, as shown in the device content waveform 606. The headset 604 also continues to allow ambient sounds from the physical environment 600 to pass through without active noise control, as shown in the inverted waveform 612. Therefore, the user 602 hears people 622 speaking and other ambient sounds at a volume level higher than the attenuated ambient sound level produced by the passive attenuation of the headset 604 and approximately equal to the actual volume level of speech and other ambient sounds in the physical environment 600 (e.g., within 10%, 15%, or 20% of it), as shown in the perceived ambient sound waveform 614.

[0194] FIG. 6H It shows from FIG. 6I The transformation. Specifically, FIG. 6G Person 622 has stopped speaking and left the physical environment 600. Therefore, the ambient sounds present in the physical environment 600 (as shown in ambient sound waveform 608) and the attenuated ambient sounds (as shown in attenuated ambient sound waveform 610) return to the physical environment 600. FIG. 6J The same volume level as shown. Furthermore, because the headphones 604 are still in a temporary audio output state (e.g., because t...), <T thAs indicated by timer 620, the headphones 604 continue to output audio content from the device at a reduced volume level, as shown in device content waveform 606. The headphones 604 also continue to allow ambient sound from the physical environment 600 to pass through without active noise control, as shown in inverted waveform 612. Therefore, the user 602 hears ambient sound at a volume level higher than the attenuated ambient sound level produced by the passive attenuation of the headphones 604 and approximately equal to the actual volume level of the ambient sound in the physical environment 600 (e.g., within 10%, 15%, or 20%), as shown in perceived ambient sound waveform 614.

[0195] FIG. 6I It shows from FIG. 6J The transformation. In FIG. 6F In the middle, input 618 ( FIG. 6F The temporary audio output state initiated has expired, as shown in the figure t=T. th As indicated by timer 620, the headphones 604 resume providing audio output at the same volume level as before initiating the temporary audio output state. Specifically, the headphones 604... FIG. 6J Restore playback of audio content from the device at the same volume level, such as FIG. 6J The device content waveform 606 is shown in the image. The headset 604 also restores active noise control, and thus restores the generation of a noise cancellation signal, as shown in the inverted waveform 612, to eliminate attenuated ambient sound. As a result of the restored active noise control, the user 602 does not perceive ambient sound, as indicated by the perceived ambient sound waveform 614. From... FIG. 6A through FIG. 6C In a further exemplary transition, where another person approaches and begins speaking to user 602 after a temporary audio output state has expired (and before initiating another temporary audio output state), headset 604 responds to changes in ambient sound in physical environment 600, as referenced herein. FIG. 7A through FIG. 7B As stated above.

[0196] FIG. 1A This is a flowchart illustrating a method 700 for adaptively changing the analog spatial position of audio output in response to changes in user posture, according to some implementation schemes. Method 700 is used in electronic devices (e.g., FIG. 3A Portable multi-functional device 100 FIG. 3B Equipment 300 or FIG. 1A Performed at a wearable audio output device 301), the electronic device includes one or more posture sensors for detecting or communicating with the posture (e.g., orientation and / or position) of the user of the electronic device relative to a first physical environment, and includes one or more audio output devices (e.g., FIG. 3B Speaker 111 in, or FIG. 1Athe wearable audio output device 301) or in communication with the wearable audio output device 301. In some embodiments, the gesture sensor includes one or more cameras (e.g., FIG. 3A the optical sensor 164 of the electronic device FIG. 1A the sensor 359 of the electronic device FIG. 3A the accelerometer 168 of the electronic device FIG. 1A the sensor 359 of the electronic device In some embodiments, the electronic device detects changes in user gestures by detecting changes in the orientation and / or position of the electronic device or a component of the electronic device (e.g., a display generation component, such as FIG. 3A the touch-sensitive display system 112 in the electronic device FIG. 3B the display 340 in the electronic device, a projector, a heads-up display, a head-mounted display, etc. FIG. 3B the gesture sensor 304 of the wearable audio output device 301) that is in communication with the electronic device and held or worn by the user. Some operations in method 700 are, optionally, combined and / or the order of some operations is, optionally, changed.

[0197] As described below, method 700 provides audio output in an intuitive manner by changing output properties of the audio output (such as the spatial location of the audio output, the volume of the audio output, the content of the audio output, etc.) in response to changes in user gestures (e.g., position and / or orientation). Note that spatial location (sometimes referred to as simulated spatial location) is a perceived property of audio output. Spatial location can be controlled or changed using well-known audio synthesis techniques in order to cause the audio output to be perceived as coming from a particular spatial location in three-dimensional space that is different from the physical location of the speaker that produces the audio output. Generally, at least two speakers are needed to change the spatial location of audio output. Changing output properties of the audio output (such as by moving the simulated spatial location of the audio output) in response to changes in user gestures allows the manner in which the audio output is provided to be dynamically adjusted to better suit the user’s current activity without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device), which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently.

[0198] When the first pose of the user satisfies the first presentation criteria (702), the device provides (704) the audio content at the first simulated spatial location relative to the user. In some embodiments, the audio is provided via one or more audio output devices (e.g., wearable audio output device 301, FIG. 5A ) in communication with (and in some embodiments, separate from) the electronic device. In some embodiments, the pose of the user corresponds to and indicates that the user is in a particular position (e.g., lying down, reclining / backrested, sitting, or standing). In some embodiments, the pose of the user satisfies the first presentation criteria when the first pose of the user is in a predefined range or set of poses (e.g., corresponding to a particular position of the user, such as when the user is reclined relative to horizontal to within a predefined range of angles). For example, as described herein with reference to FIG. 5A , FIG. 5G shows audio content provided in simulated spatial region 506 while user 502 is sitting upright. In some embodiments, the pose of the user satisfies the first presentation criteria when the user is in a particular type of space (e.g., a private space). For example, as described herein with reference to FIG. 5G , FIG. 5A through FIG. 5B shows audio content provided in simulated spatial region 506 while user 502 is in private space 512.

[0199] The device detects (706) a change in the pose of the user from the first pose to a second pose. In some embodiments, the first pose of the user is (708) one of a lying down pose, a sitting pose, and a standing pose, and the second pose of the user is another one of the lying down pose, the sitting pose, and the standing pose. In some embodiments, the user changes pose when moving between standing, sitting, and lying down positions. For example, as described herein with reference to FIG. 5A through FIG. 5B , FIG. 5A shows a change in the pose of user 502 from sitting ( FIG. 5B ) to standing ( FIG. 5I through FIG. 5K ). In another example, as described herein with reference to FIG. 5I through FIG. 5K , FIG. 5I shows a change in the pose of user 502 from reclining ( FIG. 5J through FIG. 5K ) to sitting upright ( FIG. 5F through FIG. 5G ).

[0200] In some embodiments, the level of immersion of the provided audio is highest when the user is lying down, and the level of immersion of the provided audio is lowest when the user is standing up or moving around the physical environment. In some cases, the audio is provided at one or more simulated spatial locations inside or around the user’s head when the user is lying down such that the audio sounds as if it is coming from or playing inside or around the user’s head. In some other cases, the audio is provided at simulated spatial locations above the user’s head when the user is standing up such that the audio sounds as if it is coming from or playing above the user’s head, and optionally at a lower volume.

[0201] In some embodiments in which a simulated three-dimensional environment including one or more virtual objects is displayed on or using a display generation component in conjunction with providing audio at corresponding simulated spatial locations, the level of immersion of the displayed simulated three-dimensional environment is highest when the user is lying down, and the level of immersion of the displayed simulated three-dimensional environment is lowest when the user is standing up or moving around the physical environment. In some cases, the simulated three-dimensional environment includes more virtual objects and less (or no) presentation of physical objects in the physical environment when the user is lying down. In other cases, the electronic device ceases to display at least a portion of the one or more virtual objects and / or display presentation of an additional portion of the physical environment (e.g., display additional presentation of physical objects in the physical environment) when the user sits up from a lying down position. In other cases, the electronic device ceases to display even more virtual objects or portions of virtual objects and / or display even more presentation of the physical environment when the user stands up.

[0202] Providing simulated spatial locations of audio content based on changes in user posture moving between lying down, sitting, and / or standing allows the manner in which audio output is provided to be dynamically adjusted to better suit the user’s current activity (e.g., inferred based on whether the user is lying down, sitting, or standing) without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device), which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently.

[0203] In some embodiments, detecting a change in user posture includes (710) detecting a change in location of the user from a first physical environment to a second physical environment that is different from the first physical environment (e.g., the user moves from one room in a building to a different room in the building; or the user moves from a first type of physical environment to a second type of physical environment, such as between a private space and a public space, e.g., from a bedroom to a living room, or vice versa). For example, as described herein with reference to FIGS. 6A-6D, the electronic device detects a change in location of the user from a first physical environment to a second physical environment that is different from the first physical environment.FIG. 5F through FIG. 5G The, FIG. 5A through FIG. 5B It is shown that the pose of the user 502 is changed by the user 502 moving from the public space 510 to the private space 512.

[0204] Moving the simulated spatial location from which audio content is provided based on the user’s movement between the private and public spaces allows the manner in which audio output is provided to be dynamically adjusted to better suit the user’s current environment (e.g., inferred based on whether the user is in a public space where less audio immersion is typically desired or a private space where more audio immersion is typically desired), without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping to reduce user mistakes when operating / interacting with the device), which, additionally, reduces power usage and improves battery life of the device by enabling users to get what they want more quickly and efficiently.

[0205] In response to detecting the change in user pose, and in accordance with a determination that the second pose of the user does not satisfy the first presentation criteria (712), the device provides (714) the audio content at a second simulated spatial location relative to the user that is different from the first simulated spatial location (e.g., via one or more audio output devices). For example, as described herein with reference to FIG. 5F through FIG. 5G , FIG. 5I through FIG. 5K and FIG. 5B the simulated spatial location of the audio output (e.g., music) is moved in response to the detected change in user pose. In some embodiments, the audio provided corresponds to audio content from the electronic device. In some embodiments, the audio content is stored on the electronic device. In some embodiments, the audio content is obtained by the electronic device from an external source (e.g., the internet, a content streaming source, etc.). In some embodiments, the audio content from the electronic device is different from the sounds in the physical environment (e.g., sounds detected via a microphone directed at the physical environment).

[0206] In some embodiments, the second simulated spatial location is above the user (716). In some embodiments, providing the audio at the second simulated spatial location includes providing the audio such that the audio sounds as if it is coming from above the user. For example, as described herein with reference to FIG. 5F , FIG. 5K and FIG. 5C through FIG. 5DThe audio output (e.g., music) is provided at the simulated spatial location 508 above the user 502's head. Moving the simulated spatial location providing the audio content to the simulated spatial location above the user in response to the change in the user's posture decreases the level of immersion of the provided audio content, such that the user can better interact with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device, and makes the user-device interface more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device) which, Additionally, enables the user to use the device more quickly and efficiently, and reduces power usage and improves battery life of the device by enabling the user to perform the same functions more quickly and efficiently.

[0207] In some embodiments, the device detects (718) a change in the user's posture from the second posture to a third posture, and in response to detecting the change in the user's posture from the second posture to the third posture, and in accordance with a determination that the third posture of the user satisfies the first presentation criteria, the device provides the audio content at the first simulated spatial location relative to the user. For example, as described herein with reference to FIG. 6B, the user 502 resumes the seated posture after previously assuming the standing posture. FIG. 5C through FIG. 5D The user 502 is shown resuming the seated posture after previously assuming the standing posture (e.g., as shown in FIG. 6B). FIG. 5D The user 502 is shown resuming the seated posture after previously assuming the standing posture (e.g., as shown in FIG. 6B). FIG. 5B through FIG. 5C The user 502 is shown resuming the seated posture after previously assuming the standing posture (e.g., as shown in FIG. 6B). FIG. 5A The user 502 is shown resuming the seated posture after previously assuming the standing posture (e.g., as shown in FIG. 6B). FIG. 5D The user 502 is shown resuming the seated posture after previously assuming the standing posture (e.g., as shown in FIG. 6B). FIG. 5A The user 502 is shown resuming the seated posture after previously assuming the standing posture (e.g., as shown in FIG. 6B). FIG. 1A The user 502 is shown resuming the seated posture after previously assuming the standing posture (e.g., as shown in FIG. 6B).

[0208] Moving the simulated spatial location providing the audio content to the previous location in response to the user resuming the corresponding previous posture allows the manner in which the audio output is provided to be dynamically adjusted to better suit the user's current environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device, and makes the user-device interface more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device) which, Additionally, enables the user to use the device more quickly and efficiently, and reduces power usage and improves battery life of the device by enabling the user to perform the same functions more quickly and efficiently.

[0209] In some embodiments, the device detects (720) sound in the physical environment via one or more microphones (e.g., microphone 113 of FIG. 1, or microphone 302 of FIG. 3). FIG. 3B For example, as described herein with reference to FIG. 6, FIG. 6B For example, as described herein with reference to FIG. 6, FIG. 6B For example, as described herein with reference to FIG. 6, FIG. 6C For example, as described herein with reference to FIG. 6, FIG. 6C For example, as described herein with reference to FIG. 6, FIG. 6C For example, as described herein with reference to FIG. 6, FIG. 6C For example, as described herein with reference to FIG. 6, FIG. 6C For example, as described herein with reference to FIG. 6, FIG. 6C For example, as described herein with reference to FIG. 6, FIG. 5I through FIG. 5J For example, as described herein with reference to FIG. 6,

[0210] In some embodiments, the output level of the provided audio content is gradually decreased (e.g., the audio content is gradually faded out) relative to the sound detected from the physical environment. In some embodiments, the simulated spatial position of the provided audio content is gradually changed from the first simulated spatial position to the second spatial position (e.g., the audio content sounds as if it is gradually moving to a different simulated spatial position). In some embodiments, the gradual decrease of the output level of the provided audio content relative to the sound detected from the physical environment is performed in unison with the gradual change of the simulated spatial position of the provided audio content from the first simulated spatial position to the second simulated spatial position (e.g., the audio content sounds as if it is gradually moving to a different simulated spatial position while also fading out).

[0211] Reducing the output level (such as volume) of the provided audio content relative to the detected sound in the ambient physical environment reduces the level of immersion of the provided audio content, enabling the user to better interact with the ambient physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device, and makes the user-device interface more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device), which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently.

[0212] In some embodiments, the device determines (722) an event associated with a change in the user’s posture from a first posture to a second posture (e.g., the device determines an event that occurred immediately before detecting the change in the user’s posture or within a predefined time period that can be inferred to have caused the user to change posture), and in response to detecting the change in the user’s posture, and in accordance with a determination that the second posture of the user does not satisfy the first presentation criteria, the device performs an operation to assist the user in responding to the event. In some examples, the event includes another person entering the room and speaking (e.g., speaking to the user), and the operation performed to assist the user in responding to the event includes stopping at least a portion of the media content being provided (e.g., reducing the volume of audio being played or pausing audio being played, increasing the transparency of a video being presented or pausing and / or stopping display of a video being presented, etc.). In other examples, the event includes a notification, such as an alert for an upcoming calendar event, and the operation performed to assist the user in responding to the event includes operations such as displaying details of the upcoming calendar event, launching a map application with navigation directions to the location of the upcoming calendar event, and / or providing a reminder of one or more physical objects to retrieve for the upcoming calendar event (e.g., a key, a wallet, a coat, and / or an item required for the upcoming calendar event). For example, as described herein with reference to FIGS. 5A-5C, 6A-6B, and 7A-7B. FIG. 5L and FIG. 5I through FIG. 5J FIG. 5L The user 502 is shown changing posture by sitting up from a reclined posture in response to the notification 520, and FIG. 3A The display of the beacons 524-1 and 524-2 is shown to assist the user 502 in responding to the event (notification 520) that prompted the user 502 to change posture.

[0213] ​Assisting users in responding to events that cause them to change their posture (such as reminders to continue a meeting by providing a physical object, or reducing the immersive level of the provided audio and / or video content) allows the way audio and / or visual output is delivered to be dynamically adjusted to better suit the user's current activity and help the user interact better with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the amount of input required to achieve this enhances device operability and makes the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.

[0214] In some implementations, the electronic device includes a display generating component (e.g., a display such as...). FIG. 1A The display 340, touch screen, etc. FIG. 5H through FIG. 5L The device (or communicates with) a touch-sensitive display system 112, a projector, a head-up display, a head-mounted display, etc. In some embodiments, when a user's first posture meets a first presentation criterion (724), the device displays video content corresponding to the audio content via a display generation component, and in response to detecting a change in the user's posture, and based on determining that the user's second posture does not meet the first presentation criterion, the device (e.g., by pausing playback of the video content) stops displaying at least a portion of the video content. For example, as referenced herein... FIG. 5H through FIG. 5I The above, FIG. 5J The video 516 is shown playing back while user 502 is reclining, and in response to user 502... FIG. 5J through FIG. 5L He sat up, paused the playback of video 516, and... FIG. 5K As shown. In some embodiments, video content corresponding to audio content is optionally stored on the electronic device in conjunction with the audio content. In some embodiments, the video content is optionally acquired by the electronic device from an external source (e.g., the Internet, a content streaming source, etc.) in conjunction with the audio content. In some embodiments, the video content differs from a view of the physical environment (e.g., presentation) (e.g., a live view of a camera pointing at the physical environment). For example, video 516 ( FIG. 5K ) Unlike real-time view 522 ( FIG. 5H through FIG. 5L ).

[0215] When video content is provided in conjunction with corresponding audio content, stopping the display of at least a portion of the video content in response to changes in user posture reduces the immersive level of the provided video content. This helps the user readjust to and better interact with their surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the amount of input required to achieve this experience enhances device operability and makes the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0216] In some embodiments of an electronic device including a display generation component, when a user's first posture meets a first presentation criterion (726), the device displays a simulated three-dimensional environment comprising one or more virtual objects via the display generation component, and in response to detecting a change in the user's posture, and based on determining that the user's second posture does not meet the first presentation criterion, the device stops displaying at least a portion of one or more virtual objects in the simulated three-dimensional environment. For example, as referenced herein... FIG. 5H through FIG. 5I The above, FIG. 5J The video 516 (e.g., as a full-screen virtual object in a simulated 3D environment) is shown playing back while user 502 is reclining, and, in response to user 502... FIG. 5J through FIG. 5L He sat up, paused the playback of video 516, and... FIG. 5L As shown, and optionally, the transparency of video 516 is increased, as... FIG. 5J through FIG. 5L As shown.

[0217] When combined with providing corresponding audio content to offer a simulated 3D environment (e.g., augmented and / or virtual reality environment), stopping the display of at least a portion of the video content in response to changes in user posture reduces the level of immersion in the simulated 3D environment. This helps the user readjust to and better interact with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the amount of input required to achieve this experience enhances device operability and makes the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0218] In some implementations, in response to detecting a change in the user's posture (728), and based on determining that the user's second posture does not meet a first presentation criterion, the device displays a presentation of at least a portion of the first physical environment via a display generation component. For example, as referenced herein...FIG. 5J In response to the user 502 sitting up in FIG. 5K through FIG. 5L , at least a portion of the live view 522 is displayed, as ​ shown. In some embodiments, the electronic device displays a representation (e.g., a live view) of physical objects (e.g., the plant 518, the wallet 526, and the keys 528, FIGS. 5K-5L ) in the first physical environment. In some embodiments, the electronic device displays an animated transition from the representation of the simulated three-dimensional environment to at least a portion of the physical environment. In some embodiments, the electronic device increases the apparent transparency of video content and / or other virtual objects displayed in the simulated three-dimensional environment in conjunction with displaying the representation of at least a portion of the first physical environment (e.g., to create the illusion that the physical environment becomes visible “through” the video content and / or other virtual objects). For example, FIG. 5L is shown with increased transparency such that a live view 522 of the physical environment 500, including the physical objects plant 518, wallet 526, and keys 528, is at least partially visible “through” the video 516.

[0219] Displaying a representation of at least a portion of the surrounding physical environment (e.g., a live view from a camera) in response to a change in user pose decreases the level of immersion of the displayed simulated three-dimensional environment or other displayed user interface to help the user readjust to and better interact with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device), which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently.

[0220] In some implementations, in response to detecting a change in user posture (730), and based on determining that the user's second posture does not meet a first presentation criterion, the device changes the corresponding position of one or more virtual objects in the simulated 3D environment. In some implementations, one or more virtual objects are moved out of a specific area of ​​the simulated 3D environment (e.g., to clear an area for displaying / viewing a live view of the physical environment). In some implementations, when displaying video content in a simulated 3D environment, the video content is displayed at a first position (e.g., a position in the direction the user is facing) when the user's first posture meets the first presentation criterion. In some implementations, in response to detecting a change in user posture to a second posture that does not meet the first presentation criterion, the video content is moved to a second position (e.g., a position above or to the side of an area in the direction the user is facing). For example, as referenced herein... FIGS. 5J-5K As stated, in response to user 502 FIG. 5J As he sat up, video 516 was shrunk and moved to a corner of the displayed simulated 3D environment, as... FIG. 5K As shown.

[0221] Responding to changes in user posture by moving the position of one or more virtual objects displayed in a simulated 3D environment (e.g., to clear an area for displaying a live view of the surrounding physical environment) reduces the immersion level of the displayed simulated 3D environment, helping the user readjust to and better interact with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the amount of input required to achieve this enhances device operability and makes the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.

[0222] It should be understood that, FIGS. 7A-7B The specific order in which the operations described herein are presented is merely exemplary and not intended to indicate that this order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. Furthermore, it should be noted that details of other processes described herein with reference to other methods (e.g., method 800) are similarly applicable to the above references. FIGS. 7A-7B Method 700 is described above. For example, the audio output, analog spatial location, and physical environment described in method 700 may optionally have one or more of the characteristics of the pose, presentation criteria, audio content, analog spatial location, and physical environment described herein with reference to other methods (e.g., method 800). For the sake of brevity, these details will not be repeated here.

[0223] FIGS. 8A-8B FIG. 8 is a flowchart showing a method 800 of adaptively changing audio output levels in response to changes in audio properties of a surrounding physical environment, in accordance with some embodiments. The method 800 is performed at one or more wearable audio output devices (e.g., the wearable audio output devices 301 of FIG. 3) that are in respective physical environments and in communication with an electronic device (e.g., the portable multifunctional device 100 of FIG. 1 or the device 300 of FIG. 3) (e.g., via a wireless connection, via a wired connection, or integrated). FIG. 1A FIG. 3B The method 800 is performed at one or more wearable audio output devices (e.g., the wearable audio output devices 301 of FIG. 3) that are in respective physical environments and in communication with an electronic device (e.g., the portable multifunctional device 100 of FIG. 1 or the device 300 of FIG. 3) (e.g., via a wireless connection, via a wired connection, or integrated).

[0224] As described below, the method 800 provides audio output in an intuitive manner by adjusting the levels of different audio components in the audio output in response to changes in audio properties of a surrounding physical environment. In some examples, when speech (or an increase in speech) is detected in the surrounding physical environment, the provided audio output is automatically adjusted to allow the user to hear more of the ambient sounds. Changing the levels of different audio components in the audio output, as well as the balance between different audio components, allows the manner in which the audio output is provided to be dynamically adjusted to better fit the current state of the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device and makes the user-device interfaces more efficient (e.g., by helping to

[0225] ​When one or more audio properties of the respective physical environment meet a first audio criterion (802), the wearable audio output device provides audio output corresponding to the first audio criterion. In some embodiments, the one or more audio properties of the respective physical environment meet the first audio criterion when the respective physical environment includes a respective type of sound, such as white noise, or when an energy density of sound in the respective physical environment (e.g., corresponding to a respective frequency range and / or over a respective moving time window) meets a predefined threshold. The provided audio output includes: audio corresponding to audio content from the electronic device at a first device content audio level; and audio corresponding to environmental sound from the respective physical environment at a first environmental sound audio level. In some embodiments, the environmental sound audio level (e.g., of audio corresponding to environmental sound from the respective physical environment) is zero (e.g., the one or more wearable audio output devices do not actively pass-through the environmental sound, but due to imperfect passive attenuation of the wearable audio output devices, a user wearing the one or more wearable audio output devices can perceive some of the environmental sound). For example, as described herein with reference to FIG. 6A When the environmental sound present in the physical environment 600 meets a predefined audio output criterion, the headphones 604 provide device audio content at the level shown by the device content waveform 606 in FIG. 6B, and do not actively pass-through the environmental sound from the physical environment 600. In another example, as described herein with reference to FIG. 6A When the environmental sound present in the physical environment 600 meets a different (e.g., second) predefined audio output criterion, the headphones 604 provide device audio content at the level shown by the device content waveform 606 in FIG. 6C, and actively pass-through the environmental sound from the physical environment (e.g., at a non-zero level), as described with reference to the perceived environmental sound waveform 614 in FIG. 6C. FIG. 6C When the environmental sound present in the physical environment 600 meets a different (e.g., second) predefined audio output criterion, the headphones 604 provide device audio content at the level shown by the device content waveform 606 in FIG. 6C, and actively pass-through the environmental sound from the physical environment (e.g., at a non-zero level), as described with reference to the perceived environmental sound waveform 614 in FIG. 6C. FIG. 6C When the environmental sound present in the physical environment 600 meets a different (e.g., second) predefined audio output criterion, the headphones 604 provide device audio content at the level shown by the device content waveform 606 in FIG. 6C, and actively pass-through the environmental sound from the physical environment (e.g., at a non-zero level), as described with reference to the perceived environmental sound waveform 614 in FIG. 6C.

[0226] The wearable audio output device detects (804) a change in one or more audio properties of the respective physical environment. In some embodiments, detecting a change in one or more audio properties of the respective physical environment includes detecting a change in speech relative to environmental sound (e.g., detecting an increase in speech corresponding to a person speaking or starting to speak, or detecting a decrease in speech corresponding to a person finishing speaking). For example, as described herein with reference to FIG. 6B FIG. 6B An increase in speech of a person 616 in the physical environment 600 is shown. In another example, as described herein with reference to FIG. 6D FIG. 6D A decrease in speech in the physical environment 600 is shown.

[0227] ​​In response to detecting a change in one or more audio properties of the respective physical environment (806), the wearable audio output device provides audio corresponding to environmental sound from the respective physical environment at a second environmental sound audio level that is different from the first environmental sound audio level. For example, as described herein with reference to FIGS. 6B-6C As described herein with reference to FIG. 6C In response to detecting an increase in speech of the person 616, the headset 604 changes (e.g., increases) the level at which environmental sound from the physical environment 600 is provided to the user 602, in part by beginning to pass through environmental sound from the physical environment 600, as described herein with reference to FIG. 6D As described herein with reference to FIG. 6D In response to detecting a decrease in speech of the person 616, the headset 604 changes (e.g., decreases) the level at which environmental sound from the physical environment 600 is provided to the user 602, in part by stopping the passing through of environmental sound from the physical environment 600, as described herein with reference to

[0228] In some embodiments, the detection and adjustment are performed using hardware circuitry and / or software modules on the one or more wearable audio output devices to better respond to changes in the physical environment. In some examples, detecting a change in one or more audio properties of the respective physical environment includes detecting an increase in speech in the physical environment, and the second environmental sound audio level is greater than the first environmental sound audio level (e.g., increasing the amount of environmental sound passed through from the physical environment in response to detecting an increase in speech in the physical environment, as described herein with reference to FIGS. 6B-6C ).

[0229] In some embodiments, in response to detecting a change in one or more audio properties of the respective physical environment (808), the wearable audio output device provides audio corresponding to audio content from the electronic device at a second device content audio level that is different from the first device content audio level. In some embodiments in which detecting a change in one or more audio properties of the physical environment includes detecting an increase in speech (e.g., beginning), the device content audio level (e.g., of audio corresponding to audio content from the electronic device) is decreased (e.g., to a lower device content audio level that is above zero) or in some cases paused (e.g., the device content audio level is decreased to zero). For example, as described herein with reference to FIGS. 6B-6C In response to detecting an increase in speech of the person 616, the headset 604 changes (e.g., decreases) the level at which audio content from the device is provided, as described herein with reference to FIG. 6CThe device content waveform 606 is described herein. In some embodiments that detect changes in one or more audio properties of the physical environment, including detecting a decrease (e.g., an end) in speech, the device content audio level (e.g., the audio corresponding to the audio content from the electronic device) increases (e.g., increases from a lower device content audio level above zero to a higher device content audio level) or, in some cases, recovers (e.g., the device content audio level increases from zero). For example, as referenced herein... FIG. 6D As described, in response to the detection of a decrease in the voice of person 616, the headset 604 changes (e.g., increases) the level of audio content provided from the electronic device, as referenced. FIG. 6D The device content waveform 606 is described in the text.

[0230] By adapting the level of the audio content provided by the device to changes in the audio properties of the surrounding physical environment, the user's ability to hear the provided device audio content is balanced with the user's ability to interact with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the amount of input required to achieve this experience enhances device operability and makes the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.

[0231] In some implementations, one or more wearable audio output devices include one or more posture sensors (e.g., ...) for detecting the posture of one or more wearable audio output devices. FIG. 3B (The posture sensor 304). In some embodiments, the posture sensor includes one or more gyroscopes, inertial measurement units, or other sensors that enable one or more wearable audio output devices to detect changes in the orientation and / or position of one or more wearable audio output devices or components of one or more wearable audio output devices (e.g., when worn by a user) relative to the physical environment in which the one or more wearable audio output devices are located. In some embodiments, the wearable audio output device detects changes in the posture of the wearable audio output device and (e.g., in addition to providing audio in response to detecting changes in one or more audio properties of the corresponding physical environment) further provides audio corresponding to ambient sounds from the corresponding physical environment at a second ambient sound audio level in response to detecting changes in the posture of one or more wearable audio output devices. For example, this document refers to... FIGS. 5A-5L The audio output may include different types of audio (e.g., device audio content, ambient audio, or inverted audio), as referenced herein. FIGS. 6A-6JThe volume levels of different types of audio can be changed in response to changes in the audio properties of the surrounding physical environment, or in response to changes in the user's posture (e.g., as referenced herein). FIGS. 5A-5B , FIGS. 5F-5G and FIGS. 5I-5K The above).

[0232] Adjusting the levels of different audio components in the audio output in response to changes in user posture (and changes in the audio properties of the surrounding physical environment) allows the way the audio output is provided to be dynamically adjusted to better suit the user's current activity (in addition to better adapting to the current state of the surrounding physical environment) without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the amount of input required to achieve this enhances device operability and makes the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.

[0233] In some embodiments, when one or more audio properties of the corresponding physical environment meet a first audio criterion (812), the audio output also includes audio configured to cancel at least a portion of ambient sound from the corresponding physical environment at a first audio cancellation audio level. In some embodiments, the audio at the first audio cancellation audio level is greater than zero and is configured to cancel at least a portion of ambient sound from the corresponding physical environment that would otherwise be perceived by the user due to imperfect passive attenuation of the wearable audio output device, even if the ambient sound audio level of the audio actively provided by one or more wearable audio output devices is zero. For example, as referenced herein... FIG. 6A The inverted waveform 612 is provided and configured to eliminate the attenuation of the ambient sound waveform 610, even when the headset 604 does not actively allow ambient sound from the physical environment 600 to pass through. In some embodiments, in response to detecting a change in one or more audio properties of the corresponding physical environment, the wearable audio output device provides audio configured to eliminate at least a portion of the ambient sound from the corresponding physical environment at a second audio cancellation audio level different from the first audio cancellation audio level.

[0234] In some implementations that detect changes in one or more audio properties of a physical environment, including detecting an increase in speech (e.g., the onset of speech), the audio cancellation audio level (e.g., configured to cancel at least a portion of the audio from the ambient sound of the corresponding physical environment) decreases (e.g., decreases to a lower audio cancellation audio level above zero) or stops in some cases (e.g., the audio cancellation audio level decreases to zero). For example, as referenced herein...FIGS. 6B-6C In response to detecting an increase in speech of the person 616, the headphones 604 decrease (e.g., to zero) the level of the provided anti-phase audio, as described with reference to the anti-phase waveform 612. In some embodiments in which detecting a change in one or more audio properties of the physical environment includes detecting a decrease (e.g., an end) in speech, the audio-cancellation audio level increases (e.g., (e.g., from a lower audio-cancellation audio level above zero to a higher audio-cancellation audio level) or, in some cases, resumes (e.g., the audio-cancellation audio level increases from zero). For example, as described herein with reference to FIG. 6D In response to detecting an increase in speech of the person 616, the headphones 604 decrease (e.g., to zero) the level of the provided anti-phase audio, as described with reference to the anti-phase waveform 612. In some embodiments in which detecting a change in one or more audio properties of the physical environment includes detecting a decrease (e.g., an end) in speech, the audio-cancellation audio level increases (e.g., (e.g., from a lower audio-cancellation audio level above zero to a higher audio-cancellation audio level) or, in some cases, resumes (e.g., the audio-cancellation audio level increases from zero). For example, as described herein with reference to

[0235] Changing the level of noise cancellation in response to changes in audio properties of the surrounding physical environment balances the amount of noise reduction with the ability of the user to interact with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device and makes the user-device interfaces more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device), which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently.

[0236] In some embodiments, detecting a change in one or more audio properties of the respective physical environment includes (814) detecting an increase in speech in the physical environment, and the second audio-cancellation audio level is zero in response to detecting the change in one or more audio properties of the respective physical environment. In other words, in some embodiments, in response to detecting an increase in speech in the physical environment, the one or more wearable audio output devices stop providing audio configured to cancel at least a portion of ambient sound. For example, as described herein with reference to FIGS. 6B-6CIn response to detecting the increase in speech of the person 616, the headphones 604 reduce the level of the provided anti-phase audio to zero, as described with reference to the anti-phase waveform 612. Turning off noise cancellation (e.g., reducing the noise cancellation level to zero) in response to detecting an increase in speech in the surrounding physical environment helps the user to better interact with the surrounding physical environment without requiring additional input from the user. Providing an adaptive and more intuitive user experience while reducing the number of inputs needed to implement such an experience enhances the operability of the device, and makes the user-device interface more efficient (e.g., by helping the user to achieve an intended result and reducing user mistakes when operating / interacting with the device) which, Additionally, enables the user to use the device more quickly and efficiently, and thereby conserves power and improves the battery life of the device.

[0237] In some embodiments, the audio configured to cancel at least a portion of the ambient sound from the respective physical environment is based on (816) an amount by which the ambient sound from the respective physical environment is reduced by the one or more wearable audio output devices when worn by the user. In some embodiments, the one or more wearable audio output devices determine an amount by which the ambient sound from the respective physical environment is reduced (sometimes referred to as “passive attenuation”) by the one or more wearable audio output devices when worn by the user, and use the determined amount to generate the audio configured to cancel the ambient sound (sometimes referred to as “anti-phase” audio). In some embodiments, the one or more wearable audio output devices include one or more sensors to determine the passive attenuation of the one or more wearable audio output devices. For example, as described herein with reference to FIG. 3, the one or more wearable audio output devices include one or more first sensors configured to measure the ambient sound in the respective physical environment and one or more second sensors configured to measure the ambient sound perceptible to the user when wearing the one or more wearable audio output devices. FIG. 3C In some embodiments, the one or more wearable audio output devices include one or more first sensors configured to measure the ambient sound in the respective physical environment and one or more second sensors configured to measure the ambient sound perceptible to the user when wearing the one or more wearable audio output devices. In some embodiments, the one or more first sensors include one or more microphones (e.g., microphones 302-1 of FIG. 3) that measure the ambient sound outside of the earcup of the headphones (e.g., as represented by the waveform 322 of FIG. 3). FIG. 3C In some embodiments, the one or more second sensors include one or more microphones (e.g., microphones 302-2 of FIG. 3) that measure the ambient sound inside the earcup of the headphones (e.g., due to imperfect passive attenuation of the headphones and earcup when worn by the user, as represented by the waveform 324 of FIG. 3) that is different from the audio generated by the headphones and provided by the headphones (e.g., different from the device audio content, the anti-phase audio, and the pass-through audio). FIG. 3C In some embodiments, the one or more second sensors include one or more microphones (e.g., microphones 302-2 of FIG. 3) that measure the ambient sound inside the earcup of the headphones (e.g., due to imperfect passive attenuation of the headphones and earcup when worn by the user, as represented by the waveform 324 of FIG. 3) that is different from the audio generated by the headphones and provided by the headphones (e.g., different from the device audio content, the anti-phase audio, and the pass-through audio). FIG. 3C FIG. 3C

[0238] ​​Considering the level of passive attenuation (e.g., provided by headphones acting as a physical barrier on the user's ears) when providing noise-cancelled audio improves noise cancellation effectiveness and reduces overcompensation for ambient sounds, which in turn reduces overuse of the audio circuitry. Providing an improved user experience and protecting the audio circuitry enhances device operability and makes the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device), thereby further reducing power consumption and extending device battery life by enabling users to use the device more quickly and efficiently.

[0239] In some implementations, when providing audio corresponding to audio content from an electronic device (818), audio corresponding to ambient sounds from the corresponding physical environment is provided at an ambient sound audio level lower than the ambient sound audio level at which audio corresponding to ambient sounds from the corresponding physical environment is provided but audio corresponding to audio content from the electronic device is not provided. For example, as referenced herein... FIG. 6A The above, with FIG. 6E In contrast, when providing device audio content (e.g., by...) FIG. 6A When the device content waveform 606 is shown, the amount of ambient sound perceived (e.g., by...) FIG. 6A The perceived ambient sound waveform (shown in 614) is smaller than that when no device audio content is provided (e.g., by...). FIG. 6E The amount of ambient sound perceived when the device content waveform 606 is shown (e.g., by...). FIG. 6E The perceived ambient sound waveform is shown in Figure 614. Those skilled in the art will recognize that altering the ambient sound audio level can be achieved by reducing the amount of noise cancellation and / or increasing the audio passthrough.

[0240] Providing less ambient sound when a user is listening to audio content balances the user's ability to hear the provided audio content with their ability to interact with their surrounding physical environment without requiring additional input from the user. This provides an adaptive and more intuitive user experience while reducing the amount of input required to achieve this experience, enhancing device operability and making the user-device interface more efficient (e.g., by helping users achieve expected results and reducing user errors when operating / interacting with the device). This further reduces power consumption and extends device battery life by enabling users to use the device more quickly and efficiently.

[0241] In some implementations, before detecting a second change in one or more audio attributes of the corresponding physical environment (820), the wearable audio output device detects user input via one or more wearable audio output devices (e.g., gestures performed on one or more wearable audio output devices, such as taps or swipes). For example, as referenced herein... FIG. 6F As mentioned above, input 618 is detected via headphones 604.

[0242] In some implementations, in response to the detection of user input, one or more wearable audio output devices change one or more corresponding audio levels of the corresponding audio to, for example, one or more corresponding preset audio levels. For example, if user input is detected when device content audio and ambient sound audio are provided at corresponding levels determined based on audio properties of the corresponding physical environment (e.g., automatically by one or more wearable audio output devices), as referenced herein... FIG. 6F As described above, one or more wearable audio output devices temporarily alter the audio level of the device's content audio (e.g., as referenced herein). FIG. 6G The device sets the audio levels of the device content audio and ambient sound audio to a predefined level (e.g., set by the user using an audio settings user interface on the electronic device). In some implementations, after a predefined period of time since the user input was detected, the device restores the corresponding audio levels of the device content audio and ambient sound audio from the predefined (e.g., user-set) levels to levels determined based on audio attributes of the corresponding physical environment (e.g., automatically) (e.g., as referenced herein). FIG. 6J (as described herein). In some embodiments, generally speaking, regardless of whether one or more wearable audio output devices initially change the corresponding audio level of the provided audio to a predefined level in response to the detection of input, the detection of user input causes one or more wearable audio output devices to abandon changing (e.g., automatically changing) the corresponding audio level of the provided audio for a predefined period of time since the detection of input (e.g., as referenced herein). FIGS. 6G-6I The above).

[0243] In some implementations, the wearable audio output device detects a second change in one or more audio attributes of the corresponding physical environment. In some implementations, based on determining that a second change in one or more audio attributes of the corresponding physical environment has been detected since a predefined time period (e.g., a time period starting when the input is detected) has been detected, the wearable audio output device changes the corresponding audio level of the corresponding audio (e.g., by changing the device content audio level of audio corresponding to audio content from an electronic device, changing the ambient sound audio level of audio corresponding to ambient sounds from the corresponding physical environment, and / or changing the audio cancellation audio level of audio configured to cancel at least a portion of the ambient sounds from the corresponding physical environment). For example, as referenced herein... FIG. 6J The method described above involves detecting subsequent speech after a temporary audio output state has expired (and before initiating another temporary audio output state) causes a change in the corresponding audio level provided by the headset 604, as referenced herein. FIGS. 6A-6C As stated above.

[0244] In some implementations, based on determining a predefined time period since the user input was detected (e.g., within...), FIG. 6H Time T th Previously, a second change in one or more audio properties of the corresponding physical environment was detected (e.g., FIG. 6H (The person 622 is speaking), wearable audio output devices abandon changing the corresponding audio level of the corresponding audio (e.g., as referenced in this article). FIG. 6H For example, in response to the detection of input, one or more wearable audio output devices are temporarily placed in a state where automatic adjustment of the audio level of the corresponding audio (e.g., device content audio level corresponding to audio content from an electronic device, ambient sound audio level corresponding to ambient sound from the corresponding physical environment, and / or audio cancellation audio level configured to cancel at least a portion of the ambient sound from the corresponding physical environment) is performed without response to the detection of one or more changes in one or more audio attributes of the physical environment (e.g., as referenced herein). FIGS. 6G-6I (as described above). In some embodiments, the wearable audio output device detects multiple changes in one or more audio attributes of the corresponding physical environment, including a first corresponding change detected within a predefined time period since the detection of user input for initiating a temporary audio output state and a second corresponding change detected outside the predefined time period (e.g., before or after) since the detection of user input for initiating a temporary audio output state.

[0245] Allowing a user to initiate a temporary audio output state in which the levels of different audio components in the audio output are not automatically adjusted in response to changes in audio properties of the ambient physical environment provides the user additional control over the audio output using a single input via the headphones, rather than requiring the user to interact with a displayed user interface for adjusting audio settings. Reducing the number of inputs required to control the audio output and providing additional control options without requiring the display to be turned on enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping to reduce user mistakes when operating / interacting with the device), which, additionally, reduces power usage and improves battery life of the device by enabling users to use the device more quickly and efficiently.

[0246] It should be understood that the specific order or hierarchy of operations in FIGS. 8A-8B described is merely exemplary and is not intended to indicate that a specific order or hierarchy to the operations is the only order or hierarchy that can be executed to implement the operations described herein. One of ordinary skill in the art will recognize that many ways to re-order operations in the methods described herein are possible. Additionally, it should be noted that other processes described herein with reference to other methods described herein (e.g., method 700) are equally applicable to the method 800 described above with reference to method 800. For example, the audio output, simulated spatial locations, and physical environments described above with reference to method 800 optionally have one or more of the features of the audio output, simulated spatial locations, and physical environments described herein with reference to other methods described herein (e.g., method 700). For the sake of brevity, these details are not repeated here. FIGS. 8A-8B

[0247] The preceding description is presented to enable any person skilled in the art to practice the applications as described in the preceding specific details. The description is not intended to limit the applications to the form or application described. Various modifications can be made by those skilled in the art without departing from the spirit or scope of the application as described and defined by the following claims. The scope of the application should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.​

Claims

1. A computer program product storing one or more programs, said one or more programs including instructions that, when executed by a wearable audio output device in a corresponding physical environment and in communication with an electronic device, cause said wearable audio output device to: When one or more audio attributes of the corresponding physical environment meet a first audio standard, the wearable audio output device provides an audio output corresponding to the first audio standard, the audio output including: Audio corresponding to audio content from the electronic device, the audio corresponding to the audio content from the electronic device provided at a first device content audio level; and A first ambient audio corresponding to ambient sound from the corresponding physical environment, wherein the first ambient audio is actively provided by the wearable audio output device at a first ambient sound audio level, and the first ambient sound audio level is higher than the ambient sound level attenuated by the passive attenuation of the wearable audio output device; When providing audio output corresponding to the first audio standard, changes in one or more audio attributes of the corresponding physical environment are detected, including increases in sound within the corresponding physical environment; and In response to detecting a change in one or more audio attributes of the corresponding physical environment, a second ambient audio corresponding to the ambient sound from the corresponding physical environment is provided by the wearable audio output device at a second ambient sound audio level higher than the first ambient sound audio level.

2. The computer program product according to claim 1, wherein, The one or more programs include instructions that, when executed by the wearable audio output device, cause the wearable audio output device to: In response to detecting a change in one or more audio attributes of the corresponding physical environment, audio corresponding to the audio content from the electronic device is provided via the wearable audio output device at a second device content audio level different from the first device content audio level.

3. The computer program product according to claim 1, wherein: When the one or more audio properties of the corresponding physical environment satisfy the first audio standard, the audio output further includes audio configured to cancel at least a portion of ambient sound from the corresponding physical environment at a first audio cancellation audio level; and The one or more programs include instructions that, when executed by the wearable audio output device, cause the wearable audio output device to: In response to detecting a change in one or more audio attributes of the corresponding physical environment, audio is provided that is configured to eliminate at least a portion of ambient sound from the corresponding physical environment at a second audio cancellation level different from the first audio cancellation level.

4. The computer program product according to claim 3, wherein: Detecting changes in one or more audio attributes of the corresponding physical environment includes detecting speech amplification in the corresponding physical environment; and In response to detecting a change in one or more audio attributes of the corresponding physical environment, the second audio cancellation audio level is set to zero.

5. The computer program product according to any one of claims 1-4, wherein the audio output includes audio configured to eliminate at least a portion of ambient sound from the respective physical environment, and the audio configured to eliminate at least a portion of ambient sound from the respective physical environment is based on the amount by which ambient sound from the respective physical environment is reduced by the wearable audio output device when worn by a user.

6. The computer program product according to any one of claims 1-4, wherein, When providing audio corresponding to audio content from the electronic device, the audio corresponding to the ambient sound from the corresponding physical environment is provided at an ambient sound audio level lower than the ambient sound audio level provided when providing audio corresponding to ambient sound from the corresponding physical environment without providing audio corresponding to audio content from the electronic device.

7. The computer program product according to any one of claims 1-4, wherein the one or more programs include instructions that, when executed by the wearable audio output device, cause the wearable audio output device to: Before detecting a second change in one or more audio attributes of the corresponding physical environment, user input via the wearable audio output device is detected; Detect the second change in one or more audio attributes of the corresponding physical environment; Based on determining that a second change in one or more audio attributes of the corresponding physical environment is detected after a predefined time period since the user input was detected, the corresponding audio level is changed, the corresponding audio including audio corresponding to audio content from the electronic device, audio corresponding to ambient sound from the corresponding physical environment, and / or audio configured to eliminate at least a portion of ambient sound from the corresponding physical environment; as well as Based on the determination that a second change in one or more audio attributes of the corresponding physical environment is detected within the predefined time period since the user input was detected, the change in the corresponding audio level is abandoned.

8. The computer program product according to any one of claims 1-4, wherein the wearable audio output device includes one or more posture sensors for detecting the posture of the wearable audio output device, and the one or more programs include instructions that, when executed by the wearable audio output device, cause the wearable audio output device to: Detect changes in the posture of the wearable audio output device; Further, in response to detecting a change in the posture of the wearable audio output device, a second ambient audio corresponding to the ambient sound from the corresponding physical environment is provided at a second ambient sound audio level.

9. A wearable audio output device, said wearable audio output device communicating with an electronic device in a corresponding physical environment, said wearable audio output device comprising: A means for providing audio output corresponding to the first audio standard via the wearable audio output device, activated when one or more audio properties of the corresponding physical environment satisfy a first audio standard, the audio output including: Audio corresponding to audio content from the electronic device, the audio corresponding to audio content from the electronic device provided at a first device content audio level; and A first ambient audio corresponding to ambient sound from the corresponding physical environment, wherein the first ambient audio is actively provided by the wearable audio output device at a first ambient sound audio level, and the first ambient sound audio level is higher than the ambient sound level attenuated by the passive attenuation of the wearable audio output device; Means for detecting changes in one or more audio properties of the corresponding physical environment when providing audio output corresponding to the first audio standard, the changes including an increase in sound in the corresponding physical environment; and A means for providing a second ambient audio corresponding to an ambient sound from the corresponding physical environment, activated in response to detecting a change in one or more audio attributes of the corresponding physical environment, the second ambient audio being actively provided by the wearable audio output device at a second ambient sound audio level higher than the first ambient sound audio level.

10. The wearable audio output device according to claim 9, comprising: A means for providing audio corresponding to audio content from the electronic device at a second device content audio level different from the first device content audio level, activated in response to detecting a change in one or more audio attributes of the corresponding physical environment.

11. The wearable audio output device according to claim 9, wherein: When the one or more audio properties of the corresponding physical environment satisfy the first audio standard, the audio output further includes audio configured to cancel at least a portion of the ambient sound from the corresponding physical environment at a first audio cancellation audio level; and The wearable audio output device includes: A means for providing audio configured to eliminate at least a portion of ambient sound from the corresponding physical environment at a second audio cancellation level different from the first audio cancellation audio level, activated in response to detecting a change in one or more audio properties of the corresponding physical environment.

12. The wearable audio output device according to claim 11, wherein: Detecting changes in one or more audio attributes of the corresponding physical environment includes detecting speech amplification in the corresponding physical environment; as well as In response to detecting a change in one or more audio attributes of the corresponding physical environment, the second audio cancellation audio level is set to zero.

13. The wearable audio output device according to any one of claims 9-12, wherein the audio output includes audio configured to eliminate at least a portion of ambient sound from the respective physical environment, and the audio configured to eliminate at least a portion of ambient sound from the respective physical environment is based on the amount by which ambient sound from the respective physical environment is reduced by the wearable audio output device when worn by a user.

14. The wearable audio output device according to any one of claims 9-12, wherein, When providing audio corresponding to audio content from the electronic device, the audio corresponding to the ambient sound from the corresponding physical environment is provided at an ambient sound audio level lower than the ambient sound audio level provided when providing audio corresponding to ambient sound from the corresponding physical environment without providing audio corresponding to audio content from the electronic device.

15. The wearable audio output device according to any one of claims 9-12, comprising: A means for detecting user input via the wearable audio output device, activated prior to detecting a second change in one or more audio attributes of the corresponding physical environment; A means for detecting the second change in one or more audio properties of the corresponding physical environment; According to means for changing the corresponding audio level of a corresponding audio, which is enabled after determining that a second change in one or more audio attributes of the corresponding physical environment is detected after a predefined time period since the user input was detected, the corresponding audio includes audio corresponding to audio content from the electronic device, audio corresponding to ambient sound from the corresponding physical environment, and / or audio configured to eliminate at least a portion of ambient sound from the corresponding physical environment; and The means for abandoning changes to the corresponding audio level is activated based on determining that a second change in one or more audio attributes of the corresponding physical environment has been detected within the predefined time period since the user input was detected.

16. The wearable audio output device according to any one of claims 9-12, wherein the wearable audio output device includes one or more posture sensors for detecting the posture of the wearable audio output device, and: A device for detecting changes in the posture of the wearable audio output device; Further, in response to detecting a change in the posture of the wearable audio output device, a second ambient audio corresponding to the ambient sound from the corresponding physical environment is provided at a second ambient sound audio level.

17. A method for providing audio, the method comprising: At one or more wearable audio output devices that are in the corresponding physical environment and communicating with electronic devices: When one or more audio attributes of the corresponding physical environment meet a first audio standard, the wearable audio output device provides an audio output corresponding to the first audio standard, the audio output including: Audio corresponding to audio content from the electronic device, the audio corresponding to audio content from the electronic device provided at a first device content audio level; and A first ambient audio corresponding to ambient sound from the corresponding physical environment, wherein the first ambient audio is actively provided by the wearable audio output device at a first ambient sound audio level, and the first ambient sound audio level is higher than the ambient sound level attenuated by the passive attenuation of the wearable audio output device; When providing audio output corresponding to the first audio standard, changes in one or more audio attributes of the corresponding physical environment are detected, including increases in sound within the corresponding physical environment; and In response to detecting a change in one or more audio attributes of the corresponding physical environment, a second ambient audio corresponding to the ambient sound from the corresponding physical environment is provided by the wearable audio output device at a second ambient sound audio level higher than the first ambient sound audio level.

18. The method of claim 17, wherein the method comprises: In response to detecting a change in one or more audio attributes of the corresponding physical environment, audio corresponding to the audio content from the electronic device is provided via the one or more wearable audio output devices at a second device content audio level different from the first device content audio level.

19. The method of claim 17, wherein: When the one or more audio properties of the corresponding physical environment satisfy the first audio standard, the audio output further includes audio configured to cancel at least a portion of the ambient sound from the corresponding physical environment at a first audio cancellation audio level; and The method includes: In response to detecting a change in one or more audio attributes of the corresponding physical environment, audio is provided that is configured to eliminate at least a portion of ambient sound from the corresponding physical environment at a second audio cancellation level different from the first audio cancellation level.

20. The method of claim 19, wherein: Detecting changes in one or more audio attributes of the corresponding physical environment includes detecting speech amplification in the corresponding physical environment; and In response to detecting a change in one or more audio attributes of the corresponding physical environment, the second audio cancellation audio level is set to zero.

21. The method of any one of claims 17 to 20, wherein the audio output includes audio configured to eliminate at least a portion of ambient sound from the respective physical environment, and the audio configured to eliminate at least a portion of ambient sound from the respective physical environment is based on the amount by which ambient sound from the respective physical environment is reduced by the one or more wearable audio output devices when worn by a user.

22. The method according to any one of claims 17 to 20, wherein, When providing audio corresponding to audio content from the electronic device, the audio corresponding to the ambient sound from the corresponding physical environment is provided at an ambient sound audio level lower than the ambient sound audio level provided when providing audio corresponding to ambient sound from the corresponding physical environment without providing audio corresponding to audio content from the electronic device.

23. The method according to any one of claims 17 to 20, the method comprising: Before detecting a second change in one or more audio attributes of the corresponding physical environment, user input via one or more wearable audio output devices is detected; Detect the second change in one or more audio attributes of the corresponding physical environment; Based on determining that a second change in one or more audio attributes of the corresponding physical environment is detected after a predefined time period since the user input was detected, the corresponding audio level is changed, the corresponding audio including audio corresponding to audio content from the electronic device, audio corresponding to ambient sound from the corresponding physical environment, and / or audio configured to eliminate at least a portion of ambient sound from the corresponding physical environment; as well as Based on the determination that a second change in one or more audio attributes of the corresponding physical environment is detected within the predefined time period since the user input was detected, the change in the corresponding audio level is abandoned.

24. The method of any one of claims 17 to 20, wherein the one or more wearable audio output devices include one or more posture sensors for detecting the posture of the one or more wearable audio output devices, and the method comprises: Detect changes in the posture of the one or more wearable audio output devices; Further in response to detecting a change in the posture of the one or more wearable audio output devices, a second ambient audio corresponding to the ambient sound from the corresponding physical environment is provided at a second ambient sound audio level.

Images