Devices, methods, and graphical user interfaces for interacting with user interface objects and providing feedback

Abstract

The present disclosure relates to devices, methods, and graphical user interfaces for interacting with user interface objects and providing feedback. In response to detecting an input on an object in the first user interface: if the input does not satisfy a first input threshold and does not include a substantial movement, the device replaces display of the first user interface with a second user interface corresponding to the object after detecting an end of the input; if the input satisfies a first input threshold, does not include the substantive movement, and does not satisfy a second input threshold greater than the first input threshold, the device displays a first respective user interface area including information corresponding to the object after detecting an end of the first input; and if the input satisfies a second input threshold and does not include the substantial movement, the device displays a first respective user interface area including information corresponding to the object before detecting an end of the first input.

Description

[0001] This application is a divisional application based on the Chinese invention patent application with the application date of September 26, 2019, application number 201980073199.1, and invention name “Device, method and graphical user interface for interacting with user interface objects and providing feedback”. Technical Field

[0002] The present disclosure generally relates to electronic devices with touch-sensitive surfaces, including but not limited to electronic devices with touch-sensitive surfaces that display user interface objects within a user interface and generate non-visual output to provide non-visual feedback to the user during user interaction with the user interface objects. Background Art

[0003] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has grown significantly in recent years. Exemplary touch-sensitive surfaces include trackpads and touch screen displays. Such surfaces are widely used to manipulate user interfaces on displays and objects therein.

[0004] Exemplary user interface objects include digital images, videos, text, icons, and control elements (such as buttons), as well as other graphics. Exemplary manipulations include adjusting the position and / or size of one or more user interface objects, activating a button or opening a file / application represented by a user interface object, as well as associating metadata with one or more user interface objects or otherwise manipulating the user interface. In some cases, a user will need to perform such manipulations on user interface objects in: a file management program (e.g., Finder from Apple Inc. in Cupertino, California); a system-level user interface (e.g., the Home screen or Springboard user interface, the Control Center user interface, the Application Switcher user interface, etc., from iOS from Apple Inc. in Cupertino, California); an image management application (e.g., Aperture, iPhoto, Photos from Apple Inc. in Cupertino, California); a digital content (e.g., video and music) management application (e.g., iTunes from Apple Inc. in Cupertino, California); a drawing application; a presentation application (e.g., Keynote from Apple Inc. in Cupertino, California); a word processing application (e.g., Pages from Apple Inc. in Cupertino, California); or a spreadsheet application (e.g., Numbers from Apple Inc. in Cupertino, California).

[0005] However, methods for performing these manipulations are cumbersome and inefficient. For example, using a sequence of mouse-based inputs to select one or more user interface objects and perform one or more actions on the selected user interface objects is tedious and places a significant cognitive burden on the user. Furthermore, these methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-powered devices.

[0006] Haptic feedback is often used (often in combination with visual and / or audio feedback) in an attempt to make the manipulation of user interfaces and user interface objects more efficient and intuitive for users, thereby improving the operability of electronic devices. However, conventional methods of providing haptic feedback have not been as helpful as they could be.

[0007] Some devices have touch-sensitive surfaces that are sensitive to a range of contact intensities. On such devices, multiple contact intensity thresholds are monitored by the device, and different responses are mapped to different contact intensity thresholds. However, this input and response mapping is not applicable to devices that do not have the ability to sense a range of contact intensities, which limits the general applicability of interaction models that utilize multiple contact intensity thresholds. Summary of the Invention

[0008] Therefore, electronic devices need to have faster, more efficient and more flexible methods and interfaces for manipulating user interface objects and providing feedback. Such methods and interfaces optionally supplement or replace conventional methods for manipulating user interface objects and providing feedback. Such methods and interfaces reduce the amount, extent and / or nature of inputs from the user by helping the user understand the connection between the inputs provided and the device's response to these inputs, thereby forming a more effective human-computer interface. For battery-powered devices, such methods and interfaces can save electricity and increase the time between battery charges. Such methods and interfaces are also more easily adapted to devices with different capabilities with respect to detecting and monitoring contact intensity relative to multiple intensity thresholds, thereby creating a more universally applicable user interaction model for operating systems installed on various devices. This unified interaction model also reduces confusion and user errors when a user switches between devices with different intensity sensing capabilities.

[0009] The disclosed computer system reduces or eliminates the above-mentioned defects and other problems associated with conventional methods and interfaces for manipulating user interface objects. In some embodiments, the computer system includes a desktop computer. In some embodiments, the computer system is portable (e.g., a laptop, tablet computer, or handheld device). In some embodiments, the computer system includes a personal electronic device (e.g., a wearable electronic device, such as a watch). In some embodiments, the computer system has a display generation component and one or more input devices (and / or communicates with the display generation component and one or more input devices). In some embodiments, the computer system has a touchpad (and / or communicates with the touchpad). In some embodiments, the computer system has a touch-sensitive display (also referred to as a "touch screen" or "touch screen display") (and / or communicates with a touch-sensitive display). In some embodiments, the computer system has a graphical user interface (GUI), one or more processors, memory, and one or more modules, a program or instruction set stored in the memory for performing multiple functions. In some embodiments, the user interacts with the GUI in part through stylus and / or finger contact and gestures on the touch-sensitive surface. In some embodiments, these functions optionally include playing games, image editing, drawing, presentations, word processing, spreadsheet creation, making and receiving calls, video conferencing, sending and receiving emails, instant messaging, fitness support, digital photography, digital video recording, web browsing, digital music playback, note taking and / or digital video playback. Executable instructions for performing these functions are optionally included in a non-transitory computer-readable storage medium or other computer program product configured for execution by one or more processors.

[0010] According to some embodiments, a method is performed at an electronic device having a display and an input element. The method includes displaying a user interface object corresponding to a first application on the display. The method includes detecting a first input directed to the user interface object via the input element while displaying the user interface object. The method includes, in response to detecting the first input directed to the user interface object: based on determining that the first input meets a first input threshold and includes movement less than a threshold amount, generating a non-visual output corresponding to the first input before detecting the end of the first input, and after detecting the end of the first input, displaying a system user interface including information about the first application without displaying the first application; and based on determining that the first input does not meet the first input threshold and includes movement less than a threshold amount, abandoning generating a non-visual output corresponding to the first input, and displaying the first application on the display after detecting the end of the first input.

[0011] According to some embodiments, a method is performed at an electronic device having a display and an input element. The method includes displaying a user interface object corresponding to a first application on the display. The method includes detecting a first input directed to the user interface object via the input element while displaying the user interface object. The method includes, in response to detecting the first input directed to the user interface object: generating a first tactile output in response to detecting the end of the first input based on determining that the first input satisfies an input threshold and includes movement exceeding a threshold amount after satisfying the input threshold; generating a second tactile output in response to detecting the end of the first input based on determining that the first input satisfies the input threshold and includes movement less than a threshold amount after satisfying the input threshold, wherein the second tactile output is different from the first tactile output; and performing an operation without generating the first tactile output and without generating the second tactile output in response to detecting the end of the first input based on determining that the first input does not satisfy the input threshold.

[0012] According to some embodiments, a method is performed at an electronic device having a display, an input element, and multiple audio output elements at different locations along the periphery of the electronic device. The method includes displaying a user interface object and, while displaying the user interface object, detecting a first input directed to the user interface object via the input element. In response to detecting the first input directed to the user interface object, and in response to determining that the first input meets predefined criteria and the user interface object is positioned closer to a first audio output element among the multiple audio output elements than to a second audio output element among the multiple audio output elements, the device generates a first audio output corresponding to the first input, wherein the first audio output has a greater amplitude at the first audio output element than at the second audio output element. In addition, in response to detecting the first input directed to the user interface object, and in response to determining that the first input meets predefined criteria and the user interface object is positioned closer to the second audio output element than to the first audio output element, a second audio output corresponding to the first input is generated, wherein the second audio output has a greater amplitude at the second audio output element than at the first audio output element.

[0013] According to some embodiments, a method is performed at an electronic device having a display and an input element. The method includes: displaying a first user interface including a user interface object on a display; while displaying the user interface object, detecting a first input directed to the user interface object via an input element; and in response to detecting the first input directed to the user interface object: based on a determination that the first input does not satisfy a first input threshold and includes movement less than a threshold amount, after detecting an end of the first input, replacing display of the first user interface with a second user interface corresponding to the user interface object; based on a determination that the first input satisfies the first input threshold and includes movement less than the threshold amount and the first input does not satisfy a second input threshold that is greater than the first input threshold, after detecting an end of the first input, displaying a first corresponding user interface area including information corresponding to the user interface object, wherein the first corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of the at least portion of the first user interface at a location outside of the first corresponding user interface area; and based on a determination that the first input satisfies a second input threshold and includes movement less than the threshold amount, before detecting an end of the first input, displaying the first corresponding user interface area including information corresponding to the user interface object, wherein the user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of the at least portion of the first user interface at a location outside of the first corresponding user interface area.

[0014] In some embodiments, a method is performed at an electronic device having a display and an input element. The method includes: displaying a first user interface object corresponding to first content on the display; while displaying the first user interface object, detecting a first input directed to the first user interface object via the input element; and in response to detecting the first input: based on determining that the first input includes movement in a first direction exceeding a first threshold amount after satisfying a first criterion, displaying a sharing user interface for the first content, wherein displaying the sharing user interface includes simultaneously displaying multiple representations of different sharing options, including a first sharing option associated with a first sharing protocol and a second sharing option associated with a second sharing protocol different from the first sharing protocol; and based on determining that the first input includes movement less than the first threshold amount after satisfying the first criterion, displaying a representation of the first content without displaying the multiple representations of the different sharing options.

[0015] According to some embodiments, an electronic device includes a display generating component and one or more input devices, optionally one or more sensors for detecting intensity of contact with a touch-sensitive surface, optionally one or more tactile output generators, optionally one or more device orientation sensors, one or more processors, and a memory storing one or more programs; the one or more programs are configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the performance of operations of any of the methods described herein. According to some embodiments, a computer-readable storage medium has instructions stored therein that, when executed by an electronic device having a display generating component, one or more input devices, optionally one or more sensors for detecting intensity of contact with a touch-sensitive surface, optionally one or more tactile output generators, and optionally one or more device orientation sensors, cause the device to perform or cause the performance of operations of any of the methods described herein. According to some embodiments, a graphical user interface on an electronic device having a display generation component, one or more input devices, one or more optional sensors for detecting the intensity of contact with a touch-sensitive surface, one or more optional tactile output generators, one or more optional device orientation sensors, memory, and one or more processors for executing one or more programs stored in the memory includes one or more elements displayed in any of the methods described herein, which are updated in response to the input, as described in any of the methods described herein. According to some embodiments, the electronic device includes: a display generation component, one or more input devices, one or more optional sensors for detecting the intensity of contact with a touch-sensitive surface, one or more optional tactile output generators, and one or more optional device orientation sensors; and means for performing or causing the performance of the operations of any of the methods described herein. According to some embodiments, an information processing device in an electronic device having a display generation component, one or more input devices, one or more optional sensors for detecting the intensity of contact with a touch-sensitive surface, one or more optional tactile output generators, and one or more optional device orientation sensors includes means for performing or causing the performance of the operations of any of the methods described herein.

[0016] Thus, an electronic device having a display generating component, one or more input devices, optionally one or more sensors for detecting intensity of contact with a touch-sensitive surface, optionally one or more tactile output generators, and optionally one or more device orientation sensors is provided with improved methods and interfaces for interacting with user interface objects, thereby increasing the effectiveness, efficiency, and user satisfaction of utilizing such devices. Such methods and interfaces can supplement or replace conventional methods for interacting with user interface objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, wherein like reference numerals designate corresponding parts throughout the several views.

[0018] Figure 1A is a block diagram illustrating a portable multifunction device with a touch-sensitive display according to some embodiments.

[0019] Figure 1B is a block diagram illustrating example components for event processing according to some embodiments.

[0020] Figure 1C is a block diagram illustrating a tactile output module according to some embodiments.

[0021] Figure 2 A portable multifunction device with a touch screen according to some embodiments is shown.

[0022] Figure 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments.

[0023] Figure 4A An exemplary user interface for an application menu on a portable multifunction device according to some embodiments is shown.

[0024] Figure 4B An exemplary user interface for a multifunction device having a touch-sensitive surface separate from the display is shown according to some embodiments.

[0025] Figures 4C to 4E Examples of dynamic intensity thresholds are shown according to some embodiments.

[0026] Figures 4F to 4G A set of sample tactile output patterns according to some embodiments are shown.

[0027] Figures 4H to 4J An example tactile audio output pattern that varies over time is shown in conjunction with tactile output for simulating a button click according to some embodiments.

[0028] Figure 4K Example combinations of time-varying tactile output patterns and tactile audio output patterns are shown according to some embodiments.

[0029] Figures 4L to 4Q Zoomed in Figure 4K The combinations shown are for clarity.

[0030] Figures 5A1 to 5A67 An exemplary user interface for interacting with user interface objects is shown according to some embodiments.

[0031] Figures 5B1 to 5B6 An exemplary user interface for generating audio output in response to input is shown according to some embodiments.

[0032] Figures 5C1 to 5C19 An exemplary user interface for interacting with user interface objects is shown according to some embodiments.

[0033] Figures 5D1 to 5D30 An exemplary user interface is shown for interacting with user interface objects and for interacting with previews corresponding to content and displaying a sharing user interface for sharing content using various sharing mechanisms according to some embodiments.

[0034] Figures 5E1 to 5E4 Various exemplary user interaction models for interacting with user interface objects are shown according to some embodiments.

[0035] 6A to 6D is a flowchart of a process for interacting with a user interface object according to some embodiments.

[0036] 7A to 7B is a flowchart of a process for generating tactile output to provide tactile feedback for user interaction with a user interface object according to some embodiments.

[0037] Figures 8A to 8C is a flowchart of a process for generating audio output in an electronic device in response to input, according to some embodiments, the electronic device having multiple audio output elements at different locations along the perimeter of the electronic device.

[0038] Figures 9A to 9G is a flowchart of a process for interacting with a user interface object (e.g., integrating the display of a preview or menu corresponding to the object by dragging the object or other interactions with the object, and providing multiple interaction paths to trigger the display of the preview) according to some embodiments.

[0039] Figures 10A to 10E is a flowchart of a process for interacting with a preview corresponding to content and displaying a sharing user interface for sharing the content using various sharing protocols, according to some embodiments. DETAILED DESCRIPTION

[0040] The methods, devices, and GUIs described herein provide visual and / or tactile feedback (eg, audio and haptic feedback) that makes manipulation of user interface objects more efficient and intuitive for the user.

[0041] In some embodiments, the user interface provides responses (e.g., visual, audio, and / or tactile cues) that indicate: (1) whether the input satisfies an input threshold (e.g., a threshold based on contact intensity and the amount of time the contact remains on the touch-sensitive surface after the contact is detected (or after a predefined intensity threshold is reached) (e.g., with less than a threshold amount of movement)), (2) whether the input has ended, (3) if the input has ended, whether it ended before or after the input threshold was met, (4) whether the input included movement, and / or (5) whether the movement, if any, occurred before or after the input threshold was met. For example, visual output, audio output (e.g., an audio alert and a separate audio output), and / or tactile output (e.g., tactile feedback and optionally accompanying audio output) provided in conjunction with displaying a preview associated with a user interface object and moving the user interface object, and / or those generated when the input satisfies a particular threshold (e.g., an input threshold based on contact intensity, duration, and / or a combination of both), alerts the user to changes that have occurred or are about to occur in the internal state of the device, giving the user an opportunity to continue or correct the current input to achieve the desired result.

[0042] In some embodiments, for a device with a touch-sensitive surface that is sensitive to various parameters of input (e.g., a range including contact intensity, position, duration, direction, rate of change, etc.), the device monitors multiple input thresholds (e.g., thresholds based on intensity, duration, position, movement, speed, rate of change of one or more of the foregoing, and / or a combination of two or more of the foregoing). Different responses are mapped to different input thresholds of a given input threshold type (e.g., a duration threshold type, an intensity threshold type, a speed threshold type, a position threshold type, or a combination of multiple thresholds of different types) and are triggered when the corresponding input threshold of the input threshold type is reached.

[0043] In some embodiments, for a device with a touch-sensitive surface that is sensitive to a range of contact intensities, the device provides additional functionality by allowing a user to perform complex operations with a single continuous contact, for example, by satisfying various input thresholds based on the input at different points during the input (e.g., when different intensity thresholds and / or time thresholds are satisfied).

[0044] In some embodiments, when input directed to a user interface object is detected, the current value of the input relative to multiple input thresholds (e.g., multiple different time thresholds or multiple intensity thresholds), as well as the movement of the input (e.g., movement of the contact providing the input) are monitored. The presence of movement in the input and the timing of the movement relative to the timing of satisfying one or more of the input thresholds are used in various criteria to determine which of multiple operations to perform in response to the input (e.g., when the threshold is satisfied, or after the end of the input is detected). In some embodiments, some operations are performed after the end of the input is detected if the criteria for performing those operations are met before the end of the input; and other operations are performed before the end of the input is detected (e.g., once the criteria for performing those operations are met). In some embodiments, some operations are performed only if movement is not detected before the input thresholds for those operations are met; and some operations are performed only if the end of the input or movement of the input is detected before zero or more of the input thresholds are met. In some embodiments, the same or similar operations, such as displaying a content preview or a quick action menu, can be triggered in response to the end of an input that has met a smaller input threshold and in response to an input that has met a larger input threshold but has not yet ended. Allowing an operation to be performed in response to detecting the end of an input after the smaller input threshold has been met shortens the amount of time the user must wait to trigger the operation, provides an opportunity to cancel the operation after the input threshold is met (e.g., by subsequent movement of the input), and allows the user to directly control when the operation is actually performed (e.g., within a small time window before the larger input threshold is met). At the same time, allowing the same or similar operations to be performed when the larger input threshold is met without detecting the end of the input enables the user to perform additional operations using the same continuous contact. In some embodiments, when the input continues after a preview or a quick action menu has been displayed in response to an initial portion of the input, subsequent movements in the input are evaluated against various criteria to determine the additional operations to be performed (e.g., initiating a drag operation on an object corresponding to the preview, activating a displayed selectable option, displaying a sharing interface, displaying a different preview, etc.). Integrating (1) multiple input thresholds and (2) the timing of input movement relative to the timing of satisfying one or more of these input thresholds into a multi-branch user interaction model improves the efficiency of the user interface, provides more direct control to the user, and reduces user error (e.g., with improved feedback provided in response to the evaluation of input for various input thresholds).

[0045] In some embodiments, interacting with the preview through an input (e.g., a new input or a later portion of an input that triggers display of the preview) causes the preview to shift on the user interface to reveal a plurality of selectable sharing options. This allows a user to quickly and conveniently view and select sharing operations for an object while viewing a preview of the object without having to navigate to an entirely different user interface, thereby reducing user errors and reducing the number of inputs required to perform a task.

[0046] This article describes a plurality of different methods for manipulating user interface objects. Using one or more of these methods (optionally in combination with each other) helps provide a user interface that intuitively provides additional information and functionality to the user. Using one or more of these methods (optionally in combination with each other) reduces the quantity, degree, and / or nature of input from the user and provides a more efficient human-computer interface. This enables users to use devices with touch-sensitive surfaces faster and more efficiently. For battery-powered devices, these improvements save power and increase the time between battery charges.

[0047] under, Figures 1A to 1C 、 Figure 2 and Figure 3 A description of an exemplary apparatus is provided. Figures 4A to 4B 、 Figures 5A1 to 5A67 、 Figures 5B1 to 5B6 、 Figures 5C1 to 5C19 、 Figures 5D1 to 5D30 as well as Figures 5E1 to 5E4 An exemplary user interface and interaction model for manipulating user interface objects and providing feedback is shown. Specifically, Figures 5A1 to 5A67 、 Figures 5B1 to 5B6 、 Figures 5C1 to 5C19 as well as Figures 5D1 to 5D30 An exemplary user interface for interacting with and providing feedback on user interface objects within a user interface is shown according to some embodiments. Figures 5E1 to 5E4 Various exemplary user interaction models for interacting with user interface objects are shown according to some embodiments. 6A to 6D is a flowchart of a process for interacting with a user interface object according to some embodiments. 7A to 7B is a flowchart of a process for generating tactile output to provide tactile feedback for user interaction with a user interface object according to some embodiments. Figures 8A to 8B is a flowchart of a process for generating audio output in an electronic device in response to input, according to some embodiments, the electronic device having multiple audio output elements at different locations along the perimeter of the electronic device. Figures 9A to 9G is a flowchart of a process for interacting with a user interface object according to some embodiments. Figures 10A to 10Eis a flowchart of a process for interacting with a preview corresponding to content and displaying a sharing user interface for sharing the content using various sharing mechanisms, according to some embodiments. Figures 5A1 to 5A67 、 Figures 5B1 to 5B6 、 Figures 5C1 to 5C19 and Figures 5D1 to 5D30 The user interface in 6A to 6D 、 7A to 7B 、 Figures 8A to 8C 、 Figures 9A to 9G and Figures 10A to 10E The process and Figures 5E1 to 5E4 The interaction model shown in .

[0048] Exemplary devices

[0049] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other cases, well-known methods, processes, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure various aspects of the embodiments.

[0050] It will also be understood that, although in some cases, the terms "first," "second," etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. For example, a first contact can be named a second contact, and similarly, a second contact can be named a first contact without departing from the scope of the various described embodiments. Both a first contact and a second contact are contacts, but they are not the same contact unless the context clearly indicates otherwise.

[0051] The terms used in the description of the various embodiments described herein are only for the purpose of describing specific embodiments and are not intended to be limiting. As used in the description of the various embodiments described and in the appended claims, the singular forms "a" and "the" are intended to also include plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "and / or" used herein refer to and encompass any and all possible combinations of one or more items in the associated listed items. It will also be understood that the terms "includes," "including," "comprises," and / or "comprising" when used in this specification specify the presence of stated features, integers, steps, operations, elements, and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.

[0052] As used herein, the term "if" is optionally interpreted to mean "when" followed by "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [stated condition or event] is detected" are optionally interpreted to mean "upon determining" or "in response to determining" or "upon detecting [stated condition or event]" or "in response to detecting [stated condition or event]," depending on the context.

[0053] Embodiments of electronic devices, user interfaces for such devices, and processes associated with using such devices are described herein. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions, such as a PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, but are not limited to, the Apple Watch from Apple Inc. (Cupertino, California). iPod and Device. Other portable electronic devices, such as laptops or tablets with touch-sensitive surfaces (e.g., touch screen displays and / or trackpads), are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or trackpads).

[0054] In the following discussion, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and / or joystick.

[0055] The device typically supports a variety of applications, such as one or more of the following: a note-taking application, a drawing application, a presentation application, a word processing application, a website creation application, a disk editing application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness support application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and / or a digital video player application.

[0056] Various applications executed on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or varied for different applications and / or adjusted and / or varied within the respective applications. In this way, the common physical architecture of the device (such as the touch-sensitive surface) optionally supports the various applications with a user interface that is intuitive and clear to the user.

[0057] Attention is now turned to embodiments of portable devices having touch-sensitive displays. Figure 1A 1 is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 is sometimes referred to as a "touch screen" for convenience, and sometimes simply as a touch-sensitive display. The device 100 includes memory 102 (which optionally includes one or more computer-readable storage media), a memory controller 122, one or more processing units (CPUs) 120, a peripheral device interface 118, RF circuitry 108, audio circuitry 110, a speaker 111, a microphone 113, an input / output (I / O) subsystem 106, other input or control devices 116, and external ports 124. The device 100 optionally includes one or more optical sensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting intensity of contact on the device 100 (e.g., a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100). Device 100 optionally includes one or more tactile output generators 167 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface such as touch-sensitive display system 112 of device 100 or touch pad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0058] As used in this specification and claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of a device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device that will be detected by a user using the user's sense of touch. For example, when a device or a component of the device is in contact with a surface that is touch-sensitive to a user (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical characteristic of the device or component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "press click" or "release click" on a physical actuation button. In some cases, the user will feel a tactile sensation, such as a "press click" or "release click," even when the physical actuation button associated with the touch-sensitive surface that was physically pressed (e.g., displaced) by the user's movement does not move. As another example, even when the smoothness of the touch-sensitive surface does not change, movement of the touch-sensitive surface may optionally be interpreted or sensed by the user as "roughness" of the touch-sensitive surface. Although such interpretation of touch by the user will be limited by the user's individualized sensory perception, many sensory perceptions of touch are common to most users. Therefore, when a tactile output is described as corresponding to a specific sensory perception of a user (e.g., "press click," "release click," "roughness"), unless otherwise stated, the tactile output generated corresponds to a physical displacement of the device or a component thereof that would generate the sensory perception of a typical (or average) user. Providing tactile feedback to the user using tactile output enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user provide appropriate input and reducing user errors when operating / interacting with the device), thereby further reducing power usage and extending the battery life of the device by enabling the user to use the device more quickly and efficiently.

[0059] In some embodiments, the tactile output pattern specifies characteristics of the tactile output, such as the amplitude of the tactile output, the shape of the motion waveform of the tactile output, the frequency of the tactile output, and / or the duration of the tactile output.

[0060] When a device (e.g., one or more tactile output generators that generate tactile output via movement of a movable mass) generates tactile output with different tactile output patterns, the tactile output can produce different tactile sensations in a user holding or touching the device. While a user's senses are based on their perception of the tactile output, most users will be able to discern changes in the waveform, frequency, and amplitude of the tactile output generated by the device. Therefore, the waveform, frequency, and amplitude can be adjusted to indicate to the user that different operations have been performed. Thus, tactile output with tactile output patterns designed, selected, and / or arranged to simulate the characteristics (e.g., size, material, weight, stiffness, smoothness, etc.); behavior (e.g., oscillation, displacement, acceleration, rotation, extension, etc.); and / or interaction (e.g., collision, adhesion, repulsion, attraction, friction, etc.) of objects in a given environment (e.g., a user interface including graphical features and objects, a simulated physical environment with virtual boundaries and virtual objects, a real physical environment with physical boundaries and physical objects, and / or any combination of the foregoing) can, in some cases, provide helpful feedback to the user, reducing input errors and improving the user's efficiency in operating the device. In addition, tactile output is optionally generated to correspond to feedback that is unrelated to the simulated physical properties (such as input threshold or object selection). Such tactile output will provide helpful feedback to the user in some cases, which reduces input errors and improves the efficiency of the user's operation of the device.

[0061] In some embodiments, a tactile output with a suitable tactile output pattern serves as a prompt for an event of interest in the user interface or behind the screen in the device. Examples of events of interest include activation of an affordance (e.g., a real or virtual button or toggle switch) provided on the device or in the user interface, the success or failure of a requested operation, reaching or crossing a boundary in the user interface, entering a new state, switching input focus between objects, activating a new mode, reaching or crossing an input threshold, detecting or recognizing a type of input or gesture, and the like. In some embodiments, a tactile output is provided to serve as a warning or prompt for an upcoming event or result that would otherwise occur unless a change in direction or interrupt input is detected in a timely manner. Tactile output is also used in other contexts to enrich the user experience, improve the accessibility of the device for users with visual or motor difficulties or other accessibility needs, and / or improve the efficiency and functionality of the user interface and / or device. Optionally, the tactile output is compared with audio input and / or visual user interface changes, which further enhances the user's experience when the user interacts with the user interface and / or device, and facilitates better transmission of information about the state of the user interface and / or device, and reduces input errors and improves the efficiency of the user's operation of the device.

[0062] Figure 4FA set of sample tactile output patterns are provided that can be used, individually or in combination, as is or through one or more transformations (e.g., modulation, amplification, truncation, etc.) to generate suitable tactile feedback in various scenarios for various purposes (such as those described above and those described for the user interfaces and methods discussed herein). This example of a control panel for tactile output shows how a set of three waveforms and eight frequencies can be used to generate an array of tactile output patterns. In addition to the tactile output patterns shown in the figure, each of these tactile output patterns is optionally adjusted in amplitude by changing the gain value of the tactile output pattern, as shown, for example, for Figure 4G 80Hz, FullTap 200Hz, MiniTap 80Hz, MiniTap 200Hz, MicroTap 80Hz, and MicroTap 200Hz, are each illustrated with variations having gains of 1.0, 0.75, 0.5, and 0.25. Figure 4G As shown, changing the gain of a tactile output pattern changes the amplitude of the pattern without changing the frequency of the pattern or changing the shape of the waveform. In some embodiments, changing the frequency of the tactile output pattern also results in a lower amplitude because some tactile output generators are limited by how much force can be applied to the movable mass, so the higher frequency movement of the mass is constrained to a lower amplitude to ensure that the acceleration required to form the waveform does not require forces outside the operating force range of the tactile output generator (e.g., the peak amplitude of FullTap at 230 Hz, 270 Hz, and 300 Hz is lower than the amplitude of FullTap at 80 Hz, 100 Hz, 125 Hz, and 200 Hz).

[0063] exist Figure 4F In FIG, each column shows a tactile output pattern with a specific waveform. The waveform of the tactile output pattern represents the position of the tactile output relative to the neutral position (e.g., x zero ) versus time, through which the movable mass passes to generate a tactile output having the tactile output pattern. For example, Figure 4F The first set of tactile output patterns shown in the left column (eg, the “FullTap” tactile output pattern) each have a waveform including oscillations having two complete cycles (eg, oscillations that start and end at a neutral position and pass through the neutral position three times). Figure 4F The second set of tactile output patterns shown in the middle column (e.g., the tactile output pattern of "MiniTap") each have a waveform that includes an oscillation having one complete cycle (e.g., an oscillation that starts and ends at a neutral position and passes through the neutral position once). Figure 4FThe third set of tactile output patterns shown in the right column (e.g., the "MicroTap" tactile output pattern) each has a waveform that includes an oscillation with half a complete cycle (e.g., an oscillation that begins and ends at a neutral position and does not pass through the neutral position). The waveform of the tactile output pattern also includes a starting buffer and an ending buffer that represent the gradual acceleration and deceleration of the movable mass at the beginning and end of the tactile output. Figures 4F to 4G The exemplary waveforms shown include x and y representing the maximum and minimum extents of movement of the movable mass. min and x max For larger electronic devices with larger movable masses, the minimum and maximum degrees of movement of the mass may be larger or smaller. Figures 4F to 4G The examples shown describe the movement of a mass in 1 dimension, but similar principles can also be applied to the movement of a movable mass in two or three dimensions.

[0064] like Figure 4F As shown, each tactile output pattern also has a corresponding characteristic frequency, which affects the "pitch" of the tactile sensation felt by the user from the tactile output having that characteristic frequency. For continuous tactile output, the characteristic frequency represents the number of cycles (e.g., cycles per second) that the movable mass of the tactile output generator completes in a given time period. For discrete tactile output, a discrete output signal is generated (e.g., having 0.5, 1, or 2 cycles), and the characteristic frequency value specifies how fast the movable mass needs to move to generate a tactile output having that characteristic frequency. Figure 4F As shown, for each type of tactile output (e.g., defined by a corresponding waveform, such as FullTap, MiniTap, or MicroTap), higher frequency values ​​correspond to faster movement of the movable mass and, therefore, generally, to a shorter tactile output completion time (e.g., including the time required to complete the number of cycles of the discrete tactile output plus the start and end buffer times). For example, a FullTap with a characteristic frequency of 80 Hz takes longer to complete (e.g., in seconds) than a FullTap with a characteristic frequency of 100 Hz. Figure 4F, 35.4ms vs. 28.3ms). In addition, for a given frequency, a tactile output with more cycles in its waveform at the corresponding frequency takes longer to complete than a tactile output with fewer cycles in its waveform at the same corresponding frequency. For example, a 150Hz FullTap takes longer to complete than a 150Hz MiniTap (e.g., 19.4ms vs. 12.8ms), and a 150Hz MiniTap takes longer to complete than a 150Hz MicroTap (e.g., 12.8ms vs. 9.4ms). However, for tactile output modes with different frequencies, this rule may not apply (e.g., a tactile output with more cycles but a higher frequency may take a shorter amount of time to complete than a tactile output with fewer cycles but a lower frequency, and vice versa). For example, at 300Hz, FullTap takes the same amount of time as MiniTap (e.g., 9.9ms).

[0065] like Figure 4F As shown, the tactile output pattern also has a characteristic amplitude, which affects the amount of energy contained in the tactile signal, or the "intensity" of the tactile sensation that the user can feel through the tactile output having the characteristic amplitude. In some embodiments, the characteristic amplitude of the tactile output pattern refers to the absolute value or normalized value representing the maximum displacement of the movable mass relative to the neutral position when the tactile output is generated. In some embodiments, the characteristic amplitude of the tactile output pattern can be adjusted according to various conditions (e.g., customized based on user interface context and behavior) and / or preconfigured metrics (e.g., input-based metrics, and / or user interface-based metrics), for example, by a fixed or dynamically determined gain factor (e.g., a value between 0 and 1). In some embodiments, an input-based metric (e.g., an intensity change metric or an input speed metric) measures a characteristic of the input during the period that triggers the generation of the tactile output (e.g., the rate of change of the characteristic intensity of the contact in a press input or the rate of movement of the contact on the touch-sensitive surface). In some embodiments, a user interface-based metric (e.g., a boundary-crossing velocity metric) measures a characteristic of a user interface element (e.g., the speed at which the element moves across a hidden or visible boundary in the user interface) during a user interface change that triggers generation of a tactile output. In some embodiments, a characteristic amplitude of the tactile output pattern can be "envelope" modulated, and peaks of adjacent cycles can have different amplitudes, where one of the waveforms shown above is further modified by multiplying by an envelope parameter that changes over time (e.g., from 0 to 1) to gradually adjust the amplitude of a portion of the tactile output over time as the tactile output is generated.

[0066] Although Figure 4FSpecific frequencies, amplitudes, and waveforms are shown in the sample tactile output patterns for illustrative purposes, but tactile output patterns with other frequencies, amplitudes, and waveforms can also be used for similar purposes. For example, a waveform with between 0.5 and 4 cycles can be used. Other frequencies in the range of 60 Hz-400 Hz can also be used. According to some embodiments, Figures 5A1 to 5A67 The tactile output shown is based on Figure 4F Various tactile output patterns are generated as shown.

[0067] Figures 4H to 4J An example tactile audio output pattern that varies over time is shown in conjunction with tactile output for simulating a button click according to some embodiments. Figures 4H to 4J The principles shown are applicable to non-visual outputs generated in conjunction with other user interactions with user interface objects and are not limited to button interactions. An audio output that accompanies a tactile output to induce a tactile sensation in a user is referred to as a tactile audio output. In some embodiments, the corresponding non-visual output is a pure audio output (e.g., an audible vibration configured to induce a tactile sensation in a user) without an accompanying tactile output generated substantially simultaneously or with a preset time offset. In some embodiments, the corresponding non-visual output is a pure tactile output without an accompanying tactile audio output generated substantially simultaneously or with a preset time offset. In some embodiments, the corresponding non-visual output is a tactile output accompanied by a tactile audio output generated substantially simultaneously or with a preset time offset. According to some embodiments, when different non-visual outputs are generated to provide different types of alerts, the device changes (e.g., changes a base value or gain value, performs a transformation, etc.) the tactile output mode (e.g., amplitude, frequency, number of cycles, waveform) of the tactile output, the tactile audio output mode (e.g., amplitude, frequency, number of cycles, waveform, time offset with the tactile output) of the tactile audio output, or both (when both are present in the non-visual output).

[0068] Figure 4K An example combination of a time-varying tactile output pattern and a tactile audio output pattern is shown according to some embodiments. For clarity, Figures 4L to 4Q Zoomed in Figure 4K Combination shown.

[0069] exist Figure 4H In the top haptic audio mode "Click A1 Audio" is the audio output played along with the "Click A" normal MiniTap (230Hz) to simulate the first press click in the "normal" first click, such as Figure 4K (first row in first click column) and Figure 4LAs shown in the upper portion of , where the rate of change of intensity of the contact at the control activation threshold is higher than the threshold rate of change (e.g., the contact is undergoing a "normal" hard / fast press). In this example, the "Click A1 Audio" is offset by 2ms from the start of the "Click A" normal MiniTap (230Hz) tactile output. In some cases, the same "Click A1 Audio" and "Click A" normal MiniTap (230Hz) are played to simulate the first release click after the first press click. In some cases, the gain of the "Click A1 Audio" and / or the "Click A" normal MiniTap (230Hz) is reduced in the release click relative to the previous press click (e.g., by 50%).

[0070] The top haptic audio mode "Click A1 Audio" is also played along with the "Click A" soft MiniTap (230Hz) to simulate the first press click in a "soft" first click, as Figure 4K (second row in first click column) and Figure 4L As shown in the lower portion of , where the rate of change of the intensity of the contact at the control activation threshold is lower than the threshold rate of change (e.g., the contact is performing a "soft" and / or slow press). To simulate a "soft" press click, the gain of the "Click A1 Audio" and the "Click A" Soft MiniTap (230Hz) is reduced in the "soft" press click relative to the "normal" press click (e.g., by 50%). In this example, the "Click A1 Audio" is offset by 2ms from the start of the "Click A" Soft MiniTap (230Hz) tactile output. In some cases, the same "Click A1 Audio" and "Click A" Soft MiniTap (230Hz) are played to simulate the first release click after the first press click. In some cases, the gain of the "Click A1 Audio" and / or the "Click A" Soft MiniTap (230Hz) is reduced in the release click relative to the previous press click (e.g., by 50%).

[0071] exist Figure 4H In the bottom haptic audio mode "Click A2 Audio" is the audio output played along with the "Click A" normal MiniTap (230Hz) to simulate a second press click in a "normal" second click (e.g., as in a double-click input) after the first click within a predetermined time period, as in Figure 4K (first row in second click column) and Figure 4MAs shown in the upper portion of , where the rate of change of intensity of the contact at the control activation threshold is higher than the threshold rate of change (e.g., the contact in the second click is performing a "normal" hard / fast press). In this example, the "Click A2 Audio" is offset by 2ms from the start of the "Click A" normal MiniTap (230Hz) tactile output. In some cases, the same "Click A2 Audio" and "Click A" normal MiniTap (230Hz) are played to simulate a second release click after the second press click. In some cases, the gain of the "Click A2 Audio" and / or the "Click A" normal MiniTap (230Hz) is reduced in the second release click relative to the previous second press click (e.g., by 50%).

[0072] The bottom haptic audio mode "Click A2 Audio" is also played along with the "Click A" soft MiniTap (230Hz) to simulate a second down click in a "soft" second click (e.g., like the second click in a double-click input) following the first click within a predetermined time period. Figure 4K (second row in second click column) and Figure 4M As shown in the lower portion of , where the rate of change of the intensity of the contact at the control activation threshold is lower than the threshold rate of change (e.g., the contact is performing a "soft" / slow press). To simulate a "soft" press click, the gain of the "Click A2 Audio" and the "Click A" Soft MiniTap (230Hz) is reduced in the "soft" press click relative to the "normal" press click (e.g., by 50%). In this example, the "Click A2 Audio" is offset by 2ms from the start of the "Click A" Soft MiniTap (230Hz) tactile output. In some cases, the same "Click A2 Audio" and "Click A" Soft MiniTap (230Hz) are played to simulate a second release click after the second press click. In some cases, the gain of the "Click A2 Audio" and / or the "Click A" Soft MiniTap (230Hz) is reduced in the second release click relative to the previous second press click (e.g., by 50%).

[0073] exist Figure 4I In the top haptic audio mode "Click B1 Audio" is the audio output played together with the "Click B" normal MiniTap (270Hz) to simulate the first press click in the "normal" first click, such as Figure 4K (third row in the first click column) and Figure 4NAs shown in the upper portion of , where the rate of change of intensity of the contact at the control activation threshold is higher than the threshold rate of change (e.g., the contact is undergoing a "normal" hard / fast press). In this example, the "Click B1 Audio" is offset by 2.8ms from the start of the "Click B" normal MiniTap (270Hz) tactile output. In some cases, the same "Click B1 Audio" and "Click B" normal MiniTap (270Hz) are played to simulate the first release click after the first press click. In some cases, the gain of the "Click B1 Audio" and / or the "Click B" normal MiniTap (270Hz) is reduced in the release click relative to the previous press click (e.g., by 50%).

[0074] The top haptic audio mode "Click B1 Audio" is also played along with the "Click B" soft MiniTap (270Hz) to simulate the first press click in a "soft" first click, as Figure 4K (fourth row in the first click column) and Figure 4N As shown in the lower portion of , where the rate of change of the intensity of the contact at the control activation threshold is lower than the threshold rate of change (e.g., the contact is performing a "soft" and / or slow press). To simulate a "soft" press click, the gain of the "Click B1 Audio" and the "Click B" Soft MiniTap (270 Hz) is reduced in the "soft" press click relative to the "normal" press click (e.g., by 50%). In this example, the "Click B1 Audio" is offset by 2.8ms from the start of the "Click B" Soft MiniTap (270 Hz) tactile output. In some cases, the same "Click B1 Audio" and "Click B" Soft MiniTap (270 Hz) are played to simulate the first release click after the first press click. In some cases, the gain of the "Click B1 Audio" and / or the "Click B" Soft MiniTap (230 Hz) is reduced in the release click relative to the previous press click (e.g., by 50%).

[0075] exist Figure 4I In the bottom haptic audio mode "Click B2 Audio" is the audio output played along with the "Click B" normal MiniTap (270Hz) to simulate a second press click in a "normal" second click (e.g., as in a double-click input) after the first click within a predetermined time period, as in Figure 4K (third row in the second click column) and Figure 4OAs shown in the upper portion of , where the rate of change of intensity of the contact at the control activation threshold is higher than the threshold rate of change (e.g., the contact in the second click is performing a "normal" hard / fast press). In this example, the "Click B2 Audio" is offset by 2.8ms from the start of the "Click B" normal MiniTap (270Hz) tactile output. In some cases, the same "Click B2 Audio" and "Click B" normal MiniTap (230Hz) are played to simulate a second release click after the second press click. In some cases, the gain of the "Click B2 Audio" and / or the "Click B" normal MiniTap (270Hz) is reduced in the second release click relative to the previous second press click (e.g., by 50%).

[0076] The bottom haptic audio mode "Click B2 Audio" is also played along with the "Click B" soft MiniTap (270Hz) to simulate a second press click in a "soft" second click (e.g., as in a double-click input) following the first click within a predetermined period of time. Figure 4K (fourth row in the second click column) and Figure 4O As shown in the lower portion of , where the rate of change of the intensity of the contact at the control activation threshold is lower than the threshold rate of change (e.g., the contact is performing a "soft" / slow press). To simulate a "soft" press click, the gain of the "Click B2 Audio" and the "Click B" Soft MiniTap (270Hz) is reduced in the "soft" press click relative to the "normal" press click (e.g., by 50%). In this example, the "Click B2 Audio" is offset by 2.8ms from the start of the "Click B" Soft MiniTap (270Hz) tactile output. In some cases, the same "Click B2 Audio" and "Click B" Soft MiniTap (270Hz) are played to simulate a second release click following the second press click. In some cases, the gain of the "Click B2 Audio" and / or the "Click B" Soft MiniTap (270Hz) is reduced in the second release click relative to the previous second press click (e.g., by 50%).

[0077] exist Figure 4J In the top haptic audio mode "Click C1 Audio" is the audio output played together with the "Click C" normal MiniTap (300Hz) to simulate the first press click in the "normal" first click, such as Figure 4K (fifth row in the first click column) and Figure 4PAs shown in the upper portion of , where the rate of change of intensity of the contact at the control activation threshold is higher than the threshold rate of change (e.g., the contact is undergoing a "normal" hard / fast press). In this example, the "Click C1 Audio" is offset by 1.9ms from the start of the "Click C" normal MiniTap (300Hz) tactile output. In some cases, the same "Click C1 Audio" and "Click C" normal MiniTap (300Hz) are played to simulate the first release click after the first press click. In some cases, the gain of the "Click C1 Audio" and / or the "Click C" normal MiniTap (300Hz) is reduced in the release click relative to the previous press click (e.g., by 50%).

[0078] The top haptic audio mode "Click C1 Audio" is also played along with the "Click C" soft MiniTap (300Hz) to simulate the first press click in a "soft" first click, such as Figure 4K (sixth row in the first click column) and Figure 4P As shown in the lower portion of , where the rate of change of the intensity of the contact at the control activation threshold is lower than the threshold rate of change (e.g., the contact is performing a "soft" and / or slow press). To simulate a "soft" press click, the gain of the "Click C1 Audio" and the "Click C" Soft MiniTap (300Hz) is reduced in the "soft" press click relative to the "normal" press click (e.g., by 50%). In this example, the "Click C1 Audio" is offset by 1.9ms from the start of the "Click C" Soft MiniTap (300Hz) tactile output. In some cases, the same "Click C1 Audio" and "Click C" Soft MiniTap (270Hz) are played to simulate the first release click after the first press click. In some cases, the gain of the "Click C1 Audio" and / or the "Click C" Soft MiniTap (300Hz) is reduced in the release click relative to the previous press click (e.g., by 50%).

[0079] exist Figure 4J In the bottom tactile audio mode "Click C2 Audio" is the audio output played together with the "Click C" normal MiniTap (300Hz) to simulate a second press click in a "normal" second click (e.g., as in a double-click input) after the first click within a predetermined time period, as shown in Figure 4K (fifth row in the second click column) and Figure 4QAs shown in the upper portion of , where the rate of change of intensity of the contact at the control activation threshold is higher than the threshold rate of change (e.g., the contact in the second click is performing a "normal" hard / fast press). In this example, the "Click C2 Audio" is offset by 1.9ms from the start of the "Click C" normal MiniTap (300Hz) tactile output. In some cases, the same "Click C2 Audio" and "Click C" normal MiniTap (300Hz) are played to simulate a second release click after the second press click. In some cases, the gain of the "Click C2 Audio" and / or the "Click C" normal MiniTap (300Hz) is reduced in the second release click relative to the previous second press click (e.g., by 50%).

[0080] The bottom haptic audio pattern “Click C2 Audio” is also played along with the “Click C” soft MiniTap (300Hz) to simulate a second press click in a “soft” second click (e.g., like the second click in a double-click input) following the first click within a predetermined time period. Figure 4K (sixth row in the second click column) and Figure 4Q As shown in the lower portion of , where the rate of change of the intensity of the contact at the control activation threshold is lower than the threshold rate of change (e.g., the contact is performing a "soft" / slow press). To simulate a "soft" press click, the gain of the "Click C2 Audio" and the "Click C" Soft MiniTap (300Hz) is reduced (e.g., by 50%) in the "soft" press click relative to the "normal" press click. In this example, the "Click C2 Audio" is offset by 1.9ms from the start of the "Click C" Soft MiniTap (300Hz) tactile output. In some cases, the same "Click C2 Audio" and "Click C" Soft MiniTap (300Hz) are played to simulate a second release click after the second press click. In some cases, the gain of the "Click C2 Audio" and / or the "Click C" Soft MiniTap (300Hz) is reduced (e.g., by 50%) in the second release click relative to the previous second press click.

[0081] It should be understood that device 100 is merely one example of a portable multifunction device and that device 100 optionally has more or fewer components than shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. Figure 1A The various components shown in the drawings are implemented in hardware, software, firmware, or any combination thereof, including one or more signal processing circuits and / or application specific integrated circuits.

[0082] Memory 102 optionally includes high-speed random access memory and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100 (such as CPU 120 and peripheral device interface 118) is optionally controlled by memory controller 122.

[0083] Peripherals interface 118 may be used to couple the device's input and output peripherals to CPU 120 and memory 102. One or more processors 120 run or execute various software programs and / or instruction sets stored in memory 102 to perform various functions of device 100 and process data.

[0084] In some embodiments, peripherals interface 118, CPU 120, and memory controller 122 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0085] RF (radio frequency) circuitry 108 receives and transmits RF signals, also known as electromagnetic signals. RF circuitry 108 converts electrical signals into / from electromagnetic signals and communicates with a communication network and other communication devices via the electromagnetic signals. RF circuitry 108 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec chipset, a subscriber identity module (SIM) card, memory, and the like. RF circuitry 108 optionally communicates with networks and other devices via wireless communications, such as the Internet (also known as the World Wide Web (WWW)), an intranet, and / or a wireless network (such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN)). The wireless communication optionally uses any of a variety of communication standards, protocols and technologies, including, but not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolution Data Only (EV-DO), HSPA, HSPA+, Dual Cell HSPA (DC-HSDPA), Long Term Evolution (LTE), Near Field Communication (NFC), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Wi-MAX, email protocols (e.g., Internet Message Access Protocol (IMAP) and / or Post Office Protocol (POP)), instant messaging (e.g., Extensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Utilizing Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and / or Short Message Service (SMS), or any other suitable communication protocol including communication protocols not yet developed on the date of filing of this document.

[0086] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between the user and the device 100. The audio circuit 110 receives audio data from the peripheral device interface 118, converts the audio data into electrical signals, and transmits the electrical signals to the speaker 111. The speaker 111 converts the electrical signals into sound waves audible to humans. The audio circuit 110 also receives electrical signals converted from sound waves by the microphone 113. The audio circuit 110 converts the electrical signals into audio data and transmits the audio data to the peripheral device interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripheral device interface 118. In some embodiments, the audio circuit 110 also includes a headset jack (e.g., Figure 2 The headset jack provides an interface between the audio circuit 110 and a removable audio input / output peripheral device, such as an output-only headset or a headset with both output (e.g., a single or dual-ear headset) and input (e.g., a microphone).

[0087] The I / O subsystem 106 couples input / output peripherals on the device 100, such as a touch-sensitive display system 112 and other input or control devices 116, to a peripherals interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a tactile feedback controller 161, and one or more input controllers 160 for other input or control devices. The one or more input controllers 160 receive / send electrical signals from / to other input or control devices 116. The other input control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slide switches, joysticks, click wheels, etc. In some alternative embodiments, the one or more input controllers 160 are optionally coupled to any (or none) of the following: a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. The one or more buttons (e.g., Figure 2 208) optionally includes up / down buttons for volume control of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button (e.g., Figure 2 206 in the ).

[0088] The touch-sensitive display system 112 provides an input interface and an output interface between the device and the user. The display controller 156 receives electrical signals from the touch-sensitive display system 112 and / or sends electrical signals to the touch-sensitive display system 112. The touch-sensitive display system 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some visual outputs or all of the visual outputs correspond to user interface objects. As used herein, the term "indicator" refers to a user-interactive graphical user interface object (e.g., a graphical user interface object that is configured to respond to input directed to a graphical user interface object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

[0089] The touch-sensitive display system 112 has a touch-sensitive surface, sensor, or sensor group that accepts input from the user based on tactile and / or haptic contact. The touch-sensitive display system 112 and the display controller 156 (together with any associated modules and / or instruction sets in the memory 102) detect contact (and any movement or interruption of that contact) on the touch-sensitive display system 112 and convert the detected contact into interaction with a user interface object (e.g., one or more soft keys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In some embodiments, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0090] The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) technology, LPD (light emitting polymer display) technology, or LED (light emitting diode) technology, although other display technologies are used in other embodiments. The touch-sensitive display system 112 and display controller 156 optionally use any of a variety of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive display system 112 to detect contact and any movement or interruption thereof. In some embodiments, projected mutual capacitance sensing technology is used, such as the ATmega 2000 from Apple Inc. (Cupertino, California). iPod and Technology found in.

[0091] The touch-sensitive display system 112 optionally has a video resolution exceeding 100 dpi. In some embodiments, the touch screen video resolution exceeds 400 dpi (e.g., 500 dpi, 800 dpi, or greater). The user optionally uses any suitable object or appendage, such as a stylus, finger, or the like, to contact the touch-sensitive display system 112. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may not be as precise as stylus-based input due to the larger contact area of ​​a finger on the touch screen. In some embodiments, the device converts rough finger-based input into precise pointer / cursor positions or commands for performing the action desired by the user.

[0092] In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad for activating or deactivating specific functions. In some embodiments, the touchpad is a touch-sensitive area of ​​the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface that is separate from the touch-sensitive display system 112 or an extension of the touch-sensitive surface formed by the touch screen.

[0093] Device 100 also includes a power system 162 for powering the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, power fault detection circuitry, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with the generation, management, and distribution of power in a portable device.

[0094] Device 100 optionally also includes one or more optical sensors 164 . Figure 1A An optical sensor coupled to an optical sensor controller 158 in the I / O subsystem 106 is shown. One or more optical sensors 164 optionally include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The one or more optical sensors 164 receive light projected from the environment through one or more lenses and convert the light into data representing an image. In conjunction with the imaging module 143 (also called a camera module), the one or more optical sensors 164 optionally capture still images and / or video. In some embodiments, the optical sensor is located on the rear portion of the device 100 opposite the touch-sensitive display system 112 on the front of the device, so that the touch screen can be used as a viewfinder for still image and / or video image acquisition. In some embodiments, another optical sensor is located on the front of the device to obtain an image of the user (e.g., for selfies, for video conferencing when the user views other video conference participants on the touch screen, etc.).

[0095] Device 100 optionally also includes one or more contact intensity sensors 165 . Figure 1A A contact force sensor is shown coupled to a force sensor controller 159 in the I / O subsystem 106. One or more contact force sensors 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other force sensors (e.g., sensors for measuring the force (or pressure) of a contact on a touch-sensitive surface). One or more contact force sensors 165 receive contact force information (e.g., pressure information or a surrogate for pressure information) from the environment. In some embodiments, at least one contact force sensor is juxtaposed with or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact force sensor is located on the back of the device 100 opposite the touch-sensitive display system 112 located on the front of the device 100.

[0096] Device 100 optionally also includes one or more proximity sensors 166 . Figure 1A A proximity sensor 166 is shown coupled to the peripherals interface 118. Alternatively, the proximity sensor 166 is coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, when the multifunction device is placed near the user's ear (e.g., when the user is on a phone call), the proximity sensor turns off and disables the touch-sensitive display system 112.

[0097] Device 100 optionally also includes one or more tactile output generators 167 . Figure 1A A tactile output generator is shown coupled to the tactile feedback controller 161 in the I / O subsystem 106. In some embodiments, the tactile output generator 167 includes one or more electroacoustic devices such as speakers or other audio components; and / or electromechanical devices for converting energy into linear motion such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generation components (e.g., components for converting electrical signals into tactile outputs on the device). The tactile output generator 167 receives tactile feedback generation instructions from the tactile feedback module 133 and generates tactile outputs on the device 100 that can be felt by the user of the device 100. In some embodiments, at least one tactile output generator is juxtaposed or adjacent to a touch-sensitive surface (e.g., touch-sensitive display system 112) and optionally generates tactile outputs by moving the touch-sensitive surface vertically (e.g., inward / outward toward the surface of the device 100) or laterally (e.g., back and forth in the same plane as the surface of the device 100). In some embodiments, at least one tactile output generator sensor is located on the back of the device 100 opposite the touch-sensitive display system 112 located on the front of the device 100.

[0098] Device 100 optionally also includes one or more accelerometers 168 . Figure 1A An accelerometer 168 is shown coupled to the peripherals interface 118. Alternatively, the accelerometer 168 is optionally coupled to the input controller 160 in the I / O subsystem 106. In some embodiments, information is displayed on the touch screen display in a portrait view or a landscape view based on analysis of data received from one or more accelerometers. The device 100 optionally includes a magnetometer and a GPS (or GLONASS or other global navigation system) receiver in addition to the accelerometer 168 for obtaining information about the position and orientation (e.g., portrait or landscape) of the device 100.

[0099] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a graphics module (or instruction set) 132, a tactile feedback module (or instruction set) 133, a text input module (or instruction set) 134, a global positioning system (GPS) module (or instruction set) 135, and an application (or instruction set) 136. In addition, in some embodiments, memory 102 stores device / global internal state 157, as shown in Figures 1A and 1B. Figure 3 . The device / global internal state 157 includes one or more of the following: an active application state, which indicates which applications, if any, are currently active; a display state, which indicates what applications, views, or other information occupy various areas of the touch-sensitive display system 112; a sensor state, which includes information obtained from the device's various sensors and other input or control devices 116; and position and / or orientation information regarding the device's position and / or posture.

[0100] The operating system 126 (e.g., iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system such as VxWorks) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitates communication between various hardware and software components.

[0101] The communication module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FireWire, etc.) are suitable for coupling directly to other devices or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external ports are compatible with some of the Apple Inc. (Cupertino, California) iPod and In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is the same as or similar to and / or compatible with the 30-pin connector used in some Apple Inc. (Cupertino, California) iPod and A Lightning connector that is the same as, or similar to, and / or compatible with, the Lightning connector used in the device.

[0102] The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in conjunction with the display controller 156) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact / motion module 130 includes various software components for performing various operations related to contact detection (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger press event), determining the strength of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there has been movement of the contact and tracking movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has ceased (e.g., detecting a finger lift event or contact break). The contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of a contact point optionally includes determining a rate (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point, the movement of which is represented by a series of contact data. These operations are optionally applied to a single point of contact (e.g., a single finger contact or stylus contact) or multiple points of simultaneous contact (e.g., "multi-touch" / multi-finger contact). In some embodiments, the contact / motion module 130 and display controller 156 detect contact on the touchpad.

[0103] The contact / motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and / or intensities of the detected contacts). Thus, gestures are optionally detected by detecting specific contact patterns. For example, detecting a finger tap gesture includes detecting a finger press event and then detecting a finger lift (lift-off) event at the same location (or substantially the same location) as the finger press event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger press event, then detecting one or more finger drag events, and then detecting a finger lift (lift-off) event. Similarly, taps, swipes, drags, and other gestures of the stylus are optionally detected by detecting a specific contact pattern of the stylus.

[0104] In some embodiments, detecting a finger tap gesture depends on detecting the length of time between a finger press event and a finger lift event, but is independent of detecting the intensity of the finger contact between the finger press event and the finger lift event. In some embodiments, a tap gesture is detected based on determining that the length of time between the finger press event and the finger lift event is less than a predetermined value (e.g., less than 0.1, 0.2, 0.3, 0.4, or 0.5 seconds), regardless of whether the intensity of the finger contact during the tap reaches a given intensity threshold (greater than a nominal contact detection intensity threshold), such as a light press or deep press intensity threshold. Thus, a finger tap gesture can meet a specific input criterion that does not require the characteristic intensity of the contact to meet a given intensity threshold to meet the specific input criterion. For clarity, the finger contact in a tap gesture is generally required to meet a nominal contact detection intensity threshold to detect a finger press event, below which the contact is not detected. A similar analysis applies to detecting a tap gesture by a stylus or other contact. In cases where the device is capable of detecting contact by a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold optionally does not correspond to physical contact between the finger or stylus and the touch-sensitive surface.

[0105] The same concepts apply in a similar manner to other types of gestures. For example, a swipe gesture, a pinch gesture, an expand gesture, and / or a long press gesture may be optionally detected based on satisfying criteria that are independent of the intensity of the contacts included in the gesture or that do not require one or more contacts performing the gesture to reach an intensity threshold in order to be recognized. For example, a swipe gesture is detected based on the amount of movement of one or more contacts; a zoom gesture is detected based on the movement of two or more contacts toward each other; a zoom gesture is detected based on the movement of two or more contacts away from each other; and a long press gesture is detected based on the duration of a contact on the touch-sensitive surface having less than a threshold amount of movement. Thus, the statement that a particular gesture recognition criterion does not require the intensity of the contacts to meet the corresponding intensity threshold in order to satisfy the particular gesture recognition criterion means that the particular gesture recognition criterion can be satisfied when the contacts in the gesture do not reach the corresponding intensity threshold, and can also be satisfied when one or more contacts in the gesture reaches or exceeds the corresponding intensity threshold. In some embodiments, a tap gesture is detected based on determining that a finger down event and a finger up event are detected within a predefined time period, regardless of whether the contact is above or below a corresponding intensity threshold during the predefined time period, and a swipe gesture is detected based on determining that the contact moves by more than a predefined amount, even if the contact is above the corresponding intensity threshold at the end of the contact movement. Even in specific implementations where detection of gestures is affected by the intensity of the contact performing the gesture (e.g., the device detects a long press more quickly when the intensity of the contact is above an intensity threshold, or the device delays detection of a tap input when the intensity of the contact is higher), detection of these gestures does not require the contact to reach a particular intensity threshold (e.g., even if the amount of time required to recognize the gesture varies), as long as the criteria for recognizing the gesture can be met without the contact reaching the particular intensity threshold.

[0106] In some cases, contact intensity thresholds, duration thresholds, and movement thresholds are combined in various combinations to create heuristic algorithms that distinguish between two or more different gestures directed to the same input element or area, enabling multiple different interactions with the same input element to provide a richer set of user interactions and responses. A statement that a particular set of gesture recognition criteria does not require the intensity of one or more contacts to meet the corresponding intensity threshold in order to meet the particular gesture recognition criteria does not preclude the simultaneous evaluation of other intensity-related gesture recognition criteria to identify other gestures whose criteria are met when the gesture includes a contact with an intensity above the corresponding intensity threshold. For example, in some cases, a first gesture recognition criterion for a first gesture (which does not require the intensity of the contact to meet the corresponding intensity threshold to meet the first gesture recognition criterion) competes with a second gesture recognition criterion for a second gesture (which depends on the contact meeting the corresponding intensity threshold). In such a competition, if the second gesture recognition criterion for the second gesture is met first, the gesture is optionally not recognized as meeting the first gesture recognition criterion for the first gesture. For example, if the contact reaches the corresponding intensity threshold before the contact moves a predefined amount, a deep press gesture is detected instead of a swipe gesture. Conversely, if the contact moves a predefined amount before the contact reaches the corresponding intensity threshold, a swipe gesture is detected instead of a deep press gesture. Even in such cases, the first gesture recognition criterion for the first gesture still does not require the intensity of the contact to meet the corresponding intensity threshold in order to satisfy the first gesture recognition criterion because, if the contact remains below the corresponding intensity threshold until the gesture ends (e.g., a swipe gesture with an intensity that does not increase above the corresponding intensity threshold), the gesture will be recognized as a swipe gesture by the first gesture recognition criterion. Thus, a particular gesture recognition criterion that does not require the intensity of the contact to meet the corresponding intensity threshold in order to satisfy the particular gesture recognition criterion may (A) in some cases ignore the intensity of the contact relative to the intensity threshold (e.g., for a tap gesture) and / or (B) in some cases fail to satisfy the particular gesture recognition criterion (e.g., for a long press gesture) if a set of competing intensity-related gesture recognition criteria (e.g., for a deep press gesture) recognizes the input as corresponding to an intensity-related gesture before the particular gesture recognition criterion recognizes the gesture corresponding to the input, and in this sense still depends on the intensity of the contact relative to the intensity threshold (e.g., for a long press gesture competing with a deep press gesture for recognition).

[0107] The graphics module 132 includes various known software components for rendering and displaying graphics on the touch-sensitive display system 112 or other display, including components for changing the visual impact (e.g., brightness, transparency, saturation, contrast, or other visual attributes) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, etc.

[0108] In some embodiments, the graphics module 132 stores data representing graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module 132 receives one or more codes specifying the graphics to be displayed from an application program or the like, along with coordinate data and other graphic attribute data, if necessary, and then generates screen image data for output to the display controller 156.

[0109] Haptic feedback module 133 includes various software components for generating instructions used by haptic feedback controller 161 and tactile output generator 167 to produce tactile output at one or more locations on device 100 in response to user interaction with device 100 .

[0110] Text input module 134, which is optionally a component of graphics module 132, provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0111] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for location-based dialing; to the camera 143 as picture / video metadata; and to applications that provide location-based services such as the weather widget, the local yellow pages widget, and the map / navigation widget).

[0112] Application 136 optionally includes the following modules (or instruction sets), or a subset or superset thereof:

[0113] Contacts module 137 (sometimes called address book or contact list);

[0114] Telephone module 138;

[0115] Video conferencing module 139;

[0116] Email client module 140;

[0117] Instant messaging (IM) module 141;

[0118] Fitness support module 142;

[0119] A camera module 143 for still and / or video images;

[0120] Image management module 144;

[0121] Browser module 147;

[0122] Calendar module 148;

[0123] Widget module 149, which optionally includes one or more of the following: weather widget 149-1, stock widget 149-2, calculator widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets obtained by the user, and user-created widgets 149-6;

[0124] A widget creator module 150 for forming user-created widgets 149-6;

[0125] Search module 151;

[0126] Video and music player module 152, optionally consisting of a video player module and a music player module;

[0127] Memo module 153;

[0128] Map module 154; and / or

[0129] Online video module 155.

[0130] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, rendering applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

[0131] In combination with the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the contacts module 137 includes executable instructions for managing an address book or contact list (e.g., stored in the application internal state 192 of the contacts module 137 in memory 102 or memory 370), including: adding names to the address book; deleting names from the address book; associating phone numbers, email addresses, physical addresses, or other information with names; associating images with names; categorizing and classifying names; providing phone numbers and / or email addresses to initiate and / or facilitate communications via telephone 138, video conferencing 139, email 140, or instant messaging 141; and so on.

[0132] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, phone module 138 includes executable instructions for entering a character sequence corresponding to a phone number, accessing one or more phone numbers in address book 137, modifying an entered phone number, dialing the corresponding phone number, conducting a conversation, and disconnecting or hanging up when the conversation is complete. As described above, wireless communication optionally uses any of a variety of communication standards, protocols, and technologies.

[0133] In combination with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and phone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants in accordance with user instructions.

[0134] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 includes executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still images or video images captured by camera module 143.

[0135] In conjunction with the RF circuitry 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the instant messaging module 141 includes executable instructions for entering a character sequence corresponding to an instant message, modifying previously entered characters, transmitting the corresponding instant message (e.g., using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for phone-based instant messaging or using XMPP, SIMPLE, Apple Push Notification Service (APNs), or IMPS for Internet-based instant messaging), receiving instant messages, and viewing received instant messages. In some embodiments, the transmitted and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments supported in MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0136] In combination with the RF circuitry 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the video and music player module 152, the fitness support module 142 includes executable instructions for creating a workout (e.g., with time, distance, and / or calorie burn goals); communicating with fitness sensors (in sports equipment and smart watches); receiving fitness sensor data; calibrating sensors for monitoring a workout; selecting and playing music for a workout; and displaying, storing, and transmitting fitness data.

[0137] In conjunction with touch-sensitive display system 112, display controller 156, one or more optical sensors 164, optical sensor controller 158, contact module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of still images or videos, and / or deleting still images or videos from memory 102.

[0138] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 includes executable instructions for arranging, modifying (e.g., editing), or otherwise manipulating, labeling, deleting, presenting (e.g., in a digital slideshow or album), and storing still images and / or video images.

[0139] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the browser module 147 includes executable instructions for browsing the Internet (including searching for, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages) in accordance with user instructions.

[0140] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, the email client module 140 and the browser module 147, the calendar module 148 includes executable instructions for creating, displaying, modifying and storing calendars and data associated with the calendar (e.g., calendar entries, to-do items, etc.) in accordance with user instructions.

[0141] In conjunction with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget module 149 is a mini-application that is optionally downloaded and used by the user (e.g., the weather desktop widget 149-1, the stock desktop widget 149-2, the calculator desktop widget 149-3, the alarm desktop widget 149-4, and the dictionary desktop widget 149-5), or a mini-application created by the user (e.g., the user-created desktop widget 149-6). In some embodiments, the desktop widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the desktop widget includes XML (Extensible Markup Language) files and JavaScript files (e.g., the Yahoo! desktop widget).

[0142] In combination with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, the text input module 134, and the browser module 147, the desktop widget creator module 150 includes executable instructions for creating a desktop widget (e.g., transferring a user-specified portion of a web page into a desktop widget).

[0143] In conjunction with the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphics module 132, and the text input module 134, the search module 151 includes executable instructions for searching the memory 102 for text, music, sound, images, videos, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

[0144] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 includes executable instructions that allow a user to download and play back recorded music and other sound files stored in one or more file formats (such as MP3 or AAC files), as well as executable instructions for displaying, presenting, or otherwise playing back video (e.g., on touch-sensitive display system 112 or on an external display wirelessly connected via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player such as an iPod (trademark of Apple Inc.).

[0145] In conjunction with the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, and the text input module 134, the memo module 153 includes executable instructions for creating and managing memos, to-do lists, etc. according to user instructions.

[0146] In combination with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, map module 154 includes executable instructions for receiving, displaying, modifying, and storing maps and data associated with the maps (e.g., driving directions; data about stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

[0147] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, text input module 134, email client module 140, and browser module 147, online video module 155 includes executable instructions that allow a user to access, browse, receive (e.g., by streaming and / or downloading), play back (e.g., on touch screen 112 or on an external display connected wirelessly or via external port 124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module 141 is used instead of email client module 140 to send a link to a particular online video.

[0148] Each module and application identified above corresponds to a set of executable instructions for performing one or more functions and methods described in this application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., instruction sets) do not have to be implemented as independent software programs, processes, or modules, so various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above-mentioned modules and data structures. In addition, memory 102 optionally stores other modules and data structures not described above.

[0149] In some embodiments, device 100 is a device in which operation of a predefined set of functions on the device is performed exclusively through a touch screen and / or a touchpad. By using a touch screen and / or a touchpad as the primary input control device for operating device 100, the number of physical input control devices (e.g., push buttons, dials, etc.) on device 100 is optionally reduced.

[0150] A predefined set of functions that are exclusively performed through the touch screen and / or trackpad optionally includes navigation between user interfaces. In some embodiments, the trackpad, when touched by the user, navigates the device 100 from any user interface displayed on the device 100 to a main menu, home menu, or root menu. In such embodiments, the trackpad is used to implement a "menu button." In some other embodiments, the menu button is a physical push button or other physical input control device, rather than a trackpad.

[0151] Figure 1B is a block diagram illustrating exemplary components for event processing according to some embodiments. In some embodiments, memory 102 ( Figure 1A ) or memory 370 ( Figure 3 ) includes an event classifier 170 (e.g., in the operating system 126) and a corresponding application 136-1 (e.g., any one of the aforementioned applications 136, 137 to 155, 380 to 390).

[0152] The event classifier 170 receives event information and determines the application 136-1 and the application view 191 of the application 136-1 to which the event information is to be delivered. The event classifier 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, the application 136-1 includes an application internal state 192 that indicates one or more current application views displayed on the touch-sensitive display system 112 when the application is active or executing. In some embodiments, the device / global internal state 157 is used by the event classifier 170 to determine which application(s) is currently active, and the application internal state 192 is used by the event classifier 170 to determine the application view 191 to which the event information is to be delivered.

[0153] In some embodiments, the application internal state 192 includes additional information, such as one or more of the following: resumption information to be used when the application 136-1 resumes execution, user interface state information indicating that information is being displayed or is ready to be displayed by the application 136-1, a state queue for enabling the user to return to a previous state or view of the application 136-1, and a redo / undo queue of previous actions taken by the user.

[0154] Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., a user touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, one or more accelerometers 168, and / or microphone 113 (through audio circuit 110). The information received by peripherals interface 118 from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

[0155] In some embodiments, event monitor 171 sends requests to peripheral device interface 118 at predetermined intervals. In response, peripheral device interface 118 transmits event information. In other embodiments, peripheral device interface 118 transmits event information only when there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or receiving an input for more than a predetermined duration).

[0156] In some embodiments, the event classifier 170 also includes a hit view determination module 172 and / or an active event identifier determination module 173.

[0157] When the touch-sensitive display system 112 displays more than one view, the hit view determination module 172 provides software procedures for determining where within one or more views a sub-event has occurred. A view consists of controls and other elements that a user can see on the display.

[0158] Another aspect of the user interface associated with an application is a set of views, sometimes also referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of the respective application) in which a touch is detected optionally correspond to programmatic levels within the application's programmatic or view hierarchy. For example, the lowest-level view in which a touch is detected is optionally referred to as a hit view, and the set of events recognized as correct input is optionally determined based at least in part on the hit view of the initial touch that started the touch-based gesture.

[0159] Hit view determination module 172 receives information related to sub-events of touch-based gestures. When an application has multiple views organized in a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy where the sub-events should be processed. In most cases, the hit view is the lowest-level view in which the initiating sub-event (i.e., the first sub-event in a sequence of sub-events that form an event or potential event) occurs. Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0160] Active event recognizer determination module 173 determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that include the physical location of the sub-event are actively participating views, and therefore determines that all actively participating views should receive a particular sequence of sub-events. In other embodiments, even if a touch sub-event is completely confined to an area associated with one particular view, views higher in the hierarchy will still remain actively participating views.

[0161] Event dispatcher module 174 dispatches event information to event recognizers (e.g., event recognizer 180). In embodiments that include active event recognizer determination module 173, event dispatcher module 174 delivers the event information to the event recognizer determined by active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores the event information in an event queue, which is retrieved by corresponding event receiver module 182.

[0162] In some embodiments, operating system 126 includes event classifier 170. Alternatively, application 136-1 includes event classifier 170. In yet another embodiment, event classifier 170 is a standalone module or part of another module stored in memory 102, such as contact / motion module 130.

[0163] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each of which includes instructions for handling touch events that occur within a corresponding view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, a corresponding application view 191 includes multiple event recognizers 180. In other embodiments, one or more of event recognizers 180 is part of a separate module, such as a user interface toolkit or a higher-level object from which application 136-1 inherits methods and other properties. In some embodiments, a corresponding event handler 190 includes one or more of the following: a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from an event classifier 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update the application's internal state 192. Alternatively, one or more of the application views in application view 191 include one or more corresponding event handlers 190. Additionally, in some embodiments, one or more of data updater 176, object updater 177, and GUI updater 178 are included in the corresponding application view 191.

[0164] A corresponding event identifier 180 receives event information (e.g., event data 179) from event classifier 170 and identifies an event from the event information. Event identifier 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event identifier 180 also includes metadata 183 and at least a subset of event delivery instructions 188 (which optionally include sub-event delivery instructions).

[0165] The event receiver 182 receives event information from the event classifier 170. The event information includes information about sub-events such as touches or touch movements. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves the movement of a touch, the event information optionally also includes the rate and direction of the sub-event. In some embodiments, the event includes the device rotating from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device posture).

[0166] Event comparator 184 compares event information with predefined event or sub-event definitions and determines an event or sub-event based on the comparison, or determines or updates the state of an event or sub-event. In some embodiments, event comparator 184 includes event definition 186. Event definition 186 includes the definition of an event (e.g., a predefined sequence of sub-events), such as event 1 (187-1), event 2 (187-2), and others. In some embodiments, the sub-events in event 187 include, for example, touch start, touch end, touch move, touch cancel, and multi-touch. In one example, the definition of event 1 (187-1) is a double-click on a displayed object. For example, a double-click includes a first touch (touch start) of a predetermined duration on a displayed object, a first lift-off (touch end) of a predetermined duration, a second touch (touch start) of a predetermined duration on a displayed object, and a second lift-off (touch end) of a predetermined duration. In another example, the definition of event 2 (187-2) is a drag on a displayed object. For example, dragging includes a touch (or contact) of a predetermined duration on a displayed object, movement of the touch on the touch-sensitive display system 112, and lifting of the touch (touch end). In some embodiments, the event also includes information for one or more associated event handlers 190.

[0167] In some embodiments, event definition 187 includes definitions of events for corresponding user interface objects. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with a sub-event. For example, in an application view displaying three user interface objects on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, the event comparator uses the result of the hit test to determine which event handler 190 should be activated. For example, event comparator 184 selects an event handler that is associated with the sub-event and the object that triggered the hit test.

[0168] In some embodiments, the definition of the corresponding event 187 also includes delay actions that delay the delivery of the event information until it has been determined that the sub-event sequence does or does not correspond to the event type of the event identifier.

[0169] When a corresponding event recognizer 180 determines that a sequence of sub-events does not match any event in event definitions 186, the corresponding event recognizer 180 enters the event impossible, event failed, or event ended state, after which subsequent sub-events of the touch-based gesture are ignored. In this case, other event recognizers (if any) that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0170] In some embodiments, corresponding event recognizers 180 include metadata 183 with configurable properties, flags, and / or lists that indicate how the event delivery system should perform sub-event delivery for actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or can interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to different levels in a view or programmatic hierarchy.

[0171] In some embodiments, when one or more specific sub-events of an event are identified, the corresponding event recognizer 180 activates an event handler 190 associated with the event. In some embodiments, the corresponding event recognizer 180 delivers event information associated with the event to the event handler 190. Activating an event handler 190 is different from sending (and deferred sending) sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the identified event, and the event handler 190 associated with the flag obtains the flag and performs a predefined process.

[0172] In some embodiments, event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with the sub-event sequence or to an actively participating view. The event handler associated with the sub-event sequence or with the actively participating view receives the event information and performs a predetermined process.

[0173] In some embodiments, data updater 176 creates and updates data used in application 136-1. For example, data updater 176 updates phone numbers used in contact module 137 or stores video files used in video or music player module 152. In some embodiments, object updater 177 creates and updates objects used in application 136-1. For example, object updater 177 creates new user interface objects or updates the location of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends the display information to graphics module 132 for display on a touch-sensitive display.

[0174] In some embodiments, event handler 190 includes or has access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of the corresponding application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0175] It should be understood that the above discussion of event handling for user touches on a touch-sensitive display also applies to other forms of user input utilizing input devices to operate the multifunction device 100, and not all user input is initiated on a touch screen. For example, mouse movement and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holddowns; contact movement on a trackpad, such as taps, drags, scrolls, etc.; stylus input; movement of the device; spoken commands; detected eye movement; biometric input; and / or any combination thereof, are optionally used as input corresponding to sub-events defining the event to be recognized.

[0176] Figure 1C is a block diagram illustrating a tactile output module according to some embodiments. In some embodiments, the I / O subsystem 106 (e.g., the tactile feedback controller 161 ( Figure 1A ) and / or one or more other input controllers 160 ( Figure 1A ))include Figure 1C At least some of the example components shown. In some embodiments, the peripheral device interface 118 includes Figure 1C At least some of the example components shown.

[0177] In some embodiments, the tactile output module includes a tactile feedback module 133. In some embodiments, the tactile feedback module 133 aggregates and combines tactile outputs from software applications on the electronic device for user interface feedback (e.g., feedback in response to user input corresponding to the displayed user interface and prompts and other notifications indicating the execution of operations or the occurrence of events in the user interface of the electronic device). The tactile feedback module 133 includes one or more of a waveform module 123 (for providing waveforms for generating tactile outputs), a mixer 125 (for mixing waveforms, such as waveforms in different channels), a compressor 127 (for reducing or compressing the dynamic range of the waveform), a low-pass filter 129 (for filtering out high-frequency signal components in the waveform), and a thermal controller 131 (for adjusting the waveform according to thermal conditions). In some embodiments, the tactile feedback module 133 is included in a tactile feedback controller 161 ( Figure 1A In some embodiments, a separate unit of the tactile feedback module 133 (or a separate specific implementation of the tactile feedback module 133) is also included in the audio controller (e.g., audio circuit 110, Figure 1A ) and used to generate an audio signal. In some embodiments, a single tactile feedback module 133 is used to generate an audio signal and to generate a waveform for a tactile output.

[0178] In some embodiments, the haptic feedback module 133 also includes a trigger module 121 (e.g., a software application, operating system, or other software module that determines that a tactile output is to be generated and initiates a process for generating the corresponding tactile output). In some embodiments, the trigger module 121 generates a trigger signal for initiating (e.g., by the waveform module 123) the generation of a waveform. For example, the trigger module 121 generates the trigger signal based on a preset timing standard. In some embodiments, the trigger module 121 receives the trigger signal from outside the haptic feedback module 133 (e.g., in some embodiments, the haptic feedback module 133 receives the trigger signal from a hardware input processing module 146 located outside the haptic feedback module 133) and relays the trigger signal to other components within the haptic feedback module 133 (e.g., the waveform module 123) or a software application that triggers an operation (e.g., utilizing the trigger module 121) based on activation of a user interface element (e.g., an application icon or enable representation within an application) or a hardware input device (e.g., a primary button or an intensity-sensitive input interface, such as an intensity-sensitive touch screen). In some embodiments, the trigger module 121 also (e.g., from the haptic feedback module 133, Figure 1A and Figure 3 ) receives the tactile feedback generation instruction. In some embodiments, the trigger module 121 responds to the tactile feedback module 133 (or the trigger module 121 in the tactile feedback module 133) (e.g., from the tactile feedback module 133, Figure 1Aand Figure 3 ) receives tactile feedback instructions and generates a trigger signal.

[0179] Waveform module 123 receives as input a trigger signal (e.g., from trigger module 121) and, in response to receiving the trigger signal, provides a waveform for generating one or more tactile outputs (e.g., a waveform selected from a predefined set of waveforms designated for use by waveform module 123, such as those described below with reference to Figures 4F to 4G waveforms described in more detail).

[0180] The mixer 125 receives waveforms as input (e.g., from the waveform module 123) and mixes the waveforms together. For example, when the mixer 125 receives two or more waveforms (e.g., a first waveform in a first channel and a second waveform in a second channel that at least partially overlaps with the first waveform), the mixer 125 outputs a combined waveform corresponding to the sum of the two or more waveforms. In some embodiments, the mixer 125 also modifies one or more of the two or more waveforms to emphasize a particular waveform relative to the rest of the two or more waveforms (e.g., by increasing the scale of the particular waveform and / or reducing the scale of the other waveforms in the waveforms). In some cases, the mixer 125 selects one or more waveforms to remove from the combined waveform (e.g., when waveforms from more than three sources have been requested to be output simultaneously by the tactile output generator 167, the waveform from the oldest source is discarded).

[0181] Mixer 127 receives waveforms (e.g., combined waveforms from mixer 125) as input and modifies these waveforms. In some embodiments, compressor 127 reduces these waveforms (e.g., based on the tactile output generator 167 ( Figure 1A ) or 357( Figure 3 ) such that the tactile output corresponding to these waveforms is reduced. In some embodiments, compressor 127 limits the waveforms, such as by imposing a predefined maximum amplitude on the waveforms. For example, compressor 127 reduces the amplitude of the waveform portions that exceed a predefined amplitude threshold, while maintaining the amplitude of the waveform portions that do not exceed the predefined amplitude threshold. In some embodiments, compressor 127 reduces the dynamic range of the waveforms. In some embodiments, compressor 127 dynamically reduces the dynamic range of the waveforms such that the combined waveform remains within the performance specifications of tactile output generator 167 (e.g., force and / or movable mass displacement limits).

[0182] Low-pass filter 129 receives a waveform (e.g., a compressed waveform from compressor 127) as input and filters (e.g., smoothes) the waveform (e.g., removes or reduces high-frequency signal components in the waveform). For example, in some cases, compressor 127 may include extraneous signals (e.g., high-frequency signal components) in the compressed waveform that interfere with generating tactile output and / or exceed the performance specifications of tactile output generator 167 when generating tactile output based on the compressed waveform. Low-pass filter 129 reduces or removes such extraneous signals in the waveform.

[0183] Thermal controller 131 receives a waveform (e.g., a filtered waveform from low-pass filter 129) as input and adjusts the waveform based on the thermal condition of device 100 (e.g., based on an internal temperature detected within device 100, such as the temperature of haptic feedback controller 161, and / or an external temperature detected by device 100). For example, in some cases, the output of haptic feedback controller 161 varies based on temperature (e.g., in response to receiving the same waveform, haptic feedback controller 161 generates a first tactile output when haptic feedback controller 161 is at a first temperature and generates a second tactile output when haptic feedback controller 161 is at a second temperature different from the first temperature). For example, the magnitude (or amplitude) of the tactile output may vary based on temperature. To reduce the effects of temperature changes, the waveform is modified (e.g., the amplitude of the waveform is increased or decreased based on temperature).

[0184] In some embodiments, the tactile feedback module 133 (e.g., trigger module 121) is coupled to the hardware input processing module 146. In some embodiments, Figure 1A Other input controllers 160 in the embodiment include a hardware input processing module 146. In some embodiments, the hardware input processing module 146 receives input from a hardware input device 145 (e.g., Figure 1A In some embodiments, the hardware input device 145 is any input device described herein, such as a touch-sensitive display system 112 ( Figure 1A )、Keyboard / Mouse 350( Figure 3 )、Touchpad 355( Figure 3 ), one of the other input or control devices 116 ( Figure 1A ) or intensity-sensitive primary buttons. In some embodiments, the hardware input device 145 consists of intensity-sensitive primary buttons rather than touch-sensitive display system 112 ( Figure 1A )、Keyboard / Mouse 350( Figure 3 ) or touchpad 355( Figure 3). In some embodiments, in response to input from a hardware input device 145 (e.g., an intensity-sensitive home button or a touch screen), the hardware input processing module 146 provides one or more trigger signals to the haptic feedback module 133 to indicate that user input that meets predefined input criteria has been detected, such as input corresponding to a home button "click" (e.g., "press click" or "release click"). In some embodiments, the haptic feedback module 133 provides a waveform corresponding to a home button "click" in response to input corresponding to a home button "click", thereby simulating the haptic feedback of pressing a physical home button.

[0185] In some embodiments, the tactile output module includes a tactile feedback controller 161 (e.g. Figure 1A 145 ). In some embodiments, the tactile feedback controller 161 is coupled to a plurality of tactile output generators and selects one or more tactile output generators from the plurality of tactile output generators and sends a waveform to the selected one or more tactile output generators for generating a tactile output. In some embodiments, the tactile feedback controller 161 coordinates a tactile output request corresponding to activation of the hardware input device 145 and a tactile output request corresponding to a software event (e.g., a tactile output request from the tactile feedback module 133), and modifies one or more of the two or more waveforms to emphasize a particular waveform relative to the remaining waveforms of the two or more waveforms (e.g., by increasing the scale of the particular waveform and / or reducing the scale of the remaining waveforms of the waveforms to prioritize the tactile output corresponding to the activation of the hardware input device 145 over the tactile output corresponding to the software event).

[0186] In some embodiments, as Figure 1C As shown, the output of the tactile feedback controller 161 is coupled to the audio circuit of the device 100 (eg, the audio circuit 110, Figure 1A ) and provides the audio signal to the audio circuit of device 100. In some embodiments, haptic feedback controller 161 provides both a waveform for generating a tactile output and an audio signal for providing an audio output along with generating the tactile output. In some embodiments, haptic feedback controller 161 modifies the audio signal and / or the waveform (for generating the tactile output) so that the audio output and the tactile output are synchronized (e.g., by delaying the audio signal and / or the waveform). In some embodiments, haptic feedback controller 161 includes a digital-to-analog converter for converting the digital waveform into an analog signal, which is received by amplifier 163 and / or tactile output generator 167.

[0187] In some embodiments, the tactile output module includes an amplifier 163. In some embodiments, the amplifier 163 receives a waveform (e.g., from the tactile feedback controller 161), amplifies the waveform, and then sends the amplified waveform to the tactile output generator 167 (e.g., the tactile output generator 167 ( Figure 1A ) or 357( Figure 3 )). For example, amplifier 163 amplifies the received waveform to a signal level that meets the physical specifications of tactile output generator 167 (e.g., to a voltage and / or current required by tactile output generator 167 to generate a tactile output such that the signal sent to tactile output generator 167 generates a tactile output corresponding to the waveform received from haptic feedback controller 161) and sends the amplified waveform to tactile output generator 167. In response, tactile output generator 167 generates a tactile output (e.g., by displacing the movable mass back and forth in one or more dimensions relative to the neutral position of the movable mass).

[0188] In some embodiments, the tactile output module includes a sensor 169 coupled to the tactile output generator 167. The sensor 169 detects the state or state change (e.g., mechanical position, physical displacement, and / or movement) of the tactile output generator 167 or one or more components of the tactile output generator 167 (e.g., one or more moving components for generating a tactile output, such as a membrane). In some embodiments, the sensor 169 is a magnetic field sensor (e.g., a Hall effect sensor) or other displacement and / or motion sensor. In some embodiments, the sensor 169 provides information (e.g., the position, displacement, and / or movement of one or more components in the tactile output generator 167) to the tactile feedback controller 161, and based on the information about the state of the tactile output generator 167 provided by the sensor 169, the tactile feedback controller 161 adjusts the waveform output from the tactile feedback controller 161 (e.g., the waveform optionally sent to the tactile output generator 167 via the amplifier 163).

[0189] Figure 2 A device with a touch screen (e.g., Figure 1AIn some embodiments, the portable multifunction device 100 includes a touch-sensitive display system 112 (e.g., a touch screen). The touch screen optionally displays one or more graphics within a user interface (UI) 200. In these embodiments, and in other embodiments described below, a user can select one or more of the graphics by, for example, making a gesture on the graphics using one or more fingers 202 (not drawn to scale in the figures) or one or more styluses 203 (not drawn to scale in the figures). In some embodiments, selection of the one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (from left to right, from right to left, up and / or down), and / or rolling of a finger that has made contact with the device 100 (from right to left, from left to right, up and / or down). In some implementations or in some cases, inadvertent contact with a graphic does not select the graphic. For example, a swipe gesture that sweeps over an application icon optionally does not select the corresponding application when the gesture corresponding to selection is a tap.

[0190] The device 100 optionally also includes one or more physical buttons, such as a "home" or menu button 204. As previously described, the menu button 204 is optionally used to navigate to any application 136 in a set of applications that are optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen display.

[0191] In some embodiments, the device 100 includes a touch screen display, a menu button 204 (sometimes referred to as a home button 204), a push button 206 for powering the device on / off and locking the device, a volume adjustment button 208, a subscriber identity module (SIM) card slot 210, a headset jack 212, and a docking / charging external port 124. The push button 206 is optionally used to turn the device on / off by pressing the button and holding the button in the pressed state for a predefined time interval; to lock the device by pressing the button and releasing the button before the predefined time interval has passed; and / or to unlock the device or initiate an unlocking process. In some embodiments, the device 100 also accepts voice input for activating or deactivating certain functions through the microphone 113. The device 100 also optionally includes one or more contact force sensors 165 for detecting the intensity of contact on the touch-sensitive display system 112, and / or one or more tactile output generators 167 for generating tactile output for the user of the device 100.

[0192] Figure 3is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface according to some embodiments. The device 300 does not have to be portable. In some embodiments, the device 300 is a laptop, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a gaming system, or a control device (e.g., a home controller or an industrial controller). The device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, a memory 370, and one or more communication buses 320 for interconnecting these components. The communication bus 320 optionally includes circuits (sometimes referred to as a chipset) that interconnect system components and control communications between system components. The device 300 includes an input / output (I / O) interface 330 having a display 340, which is typically a touch screen display. The I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 for generating tactile output on the device 300 (e.g., similar to the above referenced devices). Figure 1A The one or more tactile output generators 167 described above), sensors 359 (e.g., optical sensors, acceleration sensors, proximity sensors, touch sensors, and / or sensors similar to those described above) Figure 1A Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices; and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from CPU 310. In some embodiments, memory 370 stores data related to portable multifunction device 100 ( Figure 1A ), or a subset thereof. In addition, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores a drawing module 380, a presentation module 382, ​​a word processing module 384, a website creation module 386, a disk editing module 388, and / or a spreadsheet module 390, while portable multifunction device 100( Figure 1A )'s memory 102 optionally does not store these modules.

[0193] Figure 3Each element in the above-mentioned identified element is optionally stored in one or more memory devices in the previously mentioned memory device.Each module in the above-mentioned identified module corresponds to the instruction set for performing the above-mentioned functions.The above-mentioned identified module or program (that is, instruction set) need not be implemented as independent software program, process or module, so the various subsets of these modules are optionally combined or otherwise rearranged in various embodiments.In some embodiments, memory 370 optionally stores the subset of above-mentioned modules and data structure.In addition, memory 370 optionally stores additional modules and data structure not described above.

[0194] Attention is now turned to an embodiment of a user interface (“UI”) that is optionally implemented on portable multifunction device 100 .

[0195] Figure 4A An example user interface 400 of an application menu on portable multifunction device 100 is shown according to some embodiments. A similar user interface is optionally implemented on device 300. In some embodiments, user interface 400 includes the following elements, or a subset or superset thereof:

[0196] Signal strength indicators for wireless communications, such as cellular and Wi-Fi signals;

[0197] ·time;

[0198] Bluetooth indicator;

[0199] Battery status indicator;

[0200] A tray 408 with common application icons, such as:

[0201] o An icon 416 labeled “Phone” of the phone module 138 , which optionally includes an indicator 414 of the number of missed calls or voice messages;

[0202] o An icon 418 of the email client module 140 labeled “Mail,” which optionally includes an indicator 410 of the number of unread emails;

[0203] o An icon 420 labeled "Browser" of the browser module 147; and

[0204] o An icon 422 labeled "Music" for the video and music player module 152; and

[0205] Icons for other applications, such as:

[0206] o Icon 424 labeled “Messages” of IM module 141;

[0207] o Icon 426 labeled “Calendar” of calendar module 148;

[0208] o Icon 428 labeled “Photos” of the image management module 144;

[0209] o An icon 430 labeled “Camera” of the camera module 143;

[0210] o Icon 432 labeled “Online Video” of the online video module 155;

[0211] o Icon 434 labeled “Stock Market” of the Stock Market widget 149-2;

[0212] o Icon 436 labeled “Map” of the map module 154;

[0213] o Icon 438 labeled “Weather” of the weather widget 149-1;

[0214] o Icon 440 labeled “Clock” of the alarm clock widget 149-4;

[0215] o An icon 442 labeled “Fitness Support” of the fitness support module 142 ;

[0216] o An icon 444 labeled “Memo” of the memo module 153; and

[0217] o A settings application or module icon 446 that provides access to settings for the device 100 and its various applications 136 .

[0218] It should be noted that Figure 4A The icon labels shown in the are merely exemplary. For example, other labels are optionally used for various application icons. In some embodiments, the label of a respective application icon includes the name of the application corresponding to the respective application icon. In some embodiments, the label of a particular application icon is different from the name of the application corresponding to the particular application icon.

[0219] Figure 4B A touch-sensitive surface 451 (e.g., Figure 3 a device (e.g., a tablet or touchpad 355) Figure 3 Although many of the examples that follow will be given with reference to input on the touch screen display 112 (where the touch-sensitive surface and the display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, such as Figure 4B In some embodiments, the touch-sensitive surface (e.g., Figure 4B 451) has a main axis (e.g., Figure 4B 453) corresponding to the main axis (for example, Figure 4B According to these embodiments, the device detects contact with the touch-sensitive surface 451 at a location that corresponds to a corresponding location on the display (e.g., Figure 4B 460 and 462 in (e.g., in Figure 4B , 460 corresponds to 468 and 462 corresponds to 470). Thus, on a touch-sensitive surface (e.g., Figure 4B 451) and a display of a multi-function device (e.g., Figure 4B When the user interface 450 in FIG. 4 is separated, the user input detected by the device on the touch-sensitive surface (e.g., contacts 460 and 462 and their movement) is used by the device to manipulate the user interface on the display. It should be understood that similar methods are optionally used for other user interfaces described herein.

[0220] In addition, although the following examples are primarily given with reference to finger inputs (e.g., finger contacts, single-finger tap gestures, finger swipe gestures, etc.), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). For another example, a tap gesture is optionally replaced by a mouse click when the cursor is over the location of the tap gesture (e.g., instead of detecting the contact, followed by ceasing to detect the contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or mice and finger contacts are optionally used simultaneously.

[0221] As used herein, the term "focus selector" refers to an input element used to indicate the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other position marker, the cursor acts as a "focus selector" such that when the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element), a focus selector is displayed on a touch-sensitive surface (e.g., Figure 3 Touchpad 355 or Figure 4B In the event that an input (e.g., a press input) is detected on the touch-sensitive surface 451 in the display, the particular user interface element is adjusted according to the detected input. In the case that the touch-sensitive surface 451 in the display is included, the touch-sensitive surface 451 can be used to directly interact with the user interface elements on the touch-screen display. Figure 1A The touch-sensitive display 112 or Figure 4AIn some implementations (such as a touch screen in a display), a contact detected on the touch screen acts as a "focus selector" such that when input (e.g., a press input by a contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touch screen display, the particular user interface element is adjusted according to the detected input. In some implementations, focus moves from one area of ​​the user interface to another area of ​​the user interface without corresponding movement of a cursor or movement of a contact on the touch screen display (e.g., by using a tab key or arrow keys to move focus from one button to another); in these implementations, the focus selector moves according to the movement of focus between different areas of the user interface. Regardless of the specific form the focus selector takes, the focus selector is generally a user interface element (or contact on the touch screen display) that is controlled by the user to deliver the user's intended interaction with the user interface (e.g., by indicating to the device the element of the user interface with which the user desires to interact). For example, when a press input is detected on a touch-sensitive surface (e.g., a trackpad or touch screen), the position of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button will indicate that the user intends to activate the corresponding button (rather than other user interface elements shown on the device display).

[0222] As used in this specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or to a surrogate (surrogate) for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​that includes at least four different values ​​and more typically includes hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch-sensitive surface are optionally used to measure the force at different points on the touch-sensitive surface. In some implementations, the force measurements from multiple force sensors are combined (e.g., weighted averaged or summed) to determine an estimated contact force. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and / or its change, the capacitance of the touch-sensitive surface near the contact and / or its change, and / or the resistance of the touch-sensitive surface near the contact and / or its change are optionally used as a substitute for the force or pressure of the contact on the touch-sensitive surface. In some embodiments, the substitute measurement of the contact force or pressure is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the substitute measurement). In some embodiments, the substitute measurement of the contact force or pressure is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether the intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in pressure units). Using the intensity of the contact as an attribute of the user input allows the user to access additional device functions that the user cannot easily access on a smaller device with limited real estate for displaying indications (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical controls / mechanical controls such as knobs or buttons).

[0223] In some embodiments, the contact / motion module 130 uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether a user has "clicked" an icon). In some embodiments, at least a subset of the intensity thresholds are determined based on software parameters (e.g., the intensity thresholds are not determined by the activation threshold of a particular physical actuator and can be adjusted without changing the physical hardware of the device 100). For example, without changing the touchpad or touchscreen display hardware, the mouse "click" threshold of a touchpad or touchscreen display can be set to any one of a large range of predefined thresholds. In addition, in some embodiments, the user of the device is provided with a software setting for adjusting one or more intensity thresholds in a set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by utilizing a system-level click on an "intensity" parameter to adjust multiple intensity thresholds at once).

[0224] As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of the contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected during a predetermined time period (e.g., 0.05 seconds, 0.1 seconds, 0.2 seconds, 0.5 seconds, 1 second, 2 seconds, 5 seconds, 10 seconds) relative to a predefined event (e.g., after contact is detected, before contact is detected to be lifted off, before or after contact begins to move, before contact ends, before or after contact is detected to increase in intensity, and / or before or after contact is detected to decrease in intensity). The characteristic intensity of a contact is optionally based on one or more of the following: the maximum value of the contact intensity, the mean value of the contact intensity, the average value of the contact intensity, the value at the top 10% of the contact intensity, the half-maximum value of the contact intensity, the 90% maximum value of the contact intensity, a value generated by low-pass filtering the contact intensity over a predefined time period or starting from a predefined time, etc. In some embodiments, the duration of the contact is used in determining the feature strength (e.g., when the feature strength is the average of the strength of the contact over time). In some embodiments, the feature strength is compared to a set of one or more strength thresholds to determine whether the user has performed an operation. For example, the set of one or more strength thresholds may include a first strength threshold and a second strength threshold. In this example, a contact whose feature strength does not exceed the first strength threshold results in a first operation, a contact whose feature strength exceeds the first strength threshold but does not exceed the second strength threshold results in a second operation, and a contact whose feature strength exceeds the second strength threshold results in a third operation. In some embodiments, a comparison between the feature strength and one or more strength thresholds is used to determine whether to perform one or more operations (e.g., whether to perform the corresponding option or give up performing the corresponding operation), rather than to determine whether to perform the first operation or the second operation.

[0225] In some embodiments, a portion of a gesture is identified for use in determining the characteristic strength. For example, a touch-sensitive surface may receive a continuous swipe contact that transitions from a starting position and reaches an ending position (e.g., a drag gesture) where the strength of the contact increases. In this example, the characteristic strength of the contact at the ending position may be based on only a portion of the continuous swipe contact, rather than the entire swipe contact (e.g., only a portion of the swipe contact at the ending position). In some embodiments, a smoothing algorithm may be applied to the strength of the swipe gesture before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of: an unweighted sliding average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some cases, these smoothing algorithms eliminate narrow peaks or dips in the intensity of the swipe contact to achieve the purpose of determining the characteristic strength.

[0226] The user interface diagrams described herein optionally include various intensity diagrams that illustrate the intensity of contacts on the touch-sensitive surface relative to one or more intensity thresholds (e.g., contact detection intensity threshold IT0, light press intensity threshold IT1, and contact detection intensity threshold IT2). L , deep press intensity threshold IT D (For example, at least initially above IT L ) and / or one or more other intensity thresholds (e.g., L Low intensity threshold IT H )). This intensity graph is not typically part of the displayed user interface, but is provided to help explain the graph. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device will perform an operation typically associated with clicking a button of a physical mouse or trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device will perform an operation different from the operation typically associated with clicking a button of a physical mouse or trackpad. In some embodiments, when a contact is detected having a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold IT0, contacts below the nominal contact detection intensity threshold are no longer detected), the device will move the focus selector according to the movement of the contact on the touch-sensitive surface without performing the operation associated with the light press intensity threshold or the deep press intensity threshold. Generally speaking, unless otherwise stated, these intensity thresholds are consistent between different groups of user interface figures.

[0227] In some embodiments, the device's response to an input detected by the device depends on criteria based on the intensity of the contact during the input. For example, for some "light press" inputs, the intensity of the contact exceeding a first intensity threshold during the input triggers a first response. In some embodiments, the device's response to an input detected by the device depends on criteria that include both the intensity of the contact during the input and a time-based criterion. For example, for some "deep press" inputs, the intensity of the contact exceeding a second intensity threshold, which is greater than the first intensity threshold for a light press, during the input triggers a second response, as long as a delay time elapses between the first intensity threshold being met and the second intensity threshold being met. The duration of this delay time is typically less than 200 ms (milliseconds) (e.g., 40 ms, 100 ms, or 120 ms, depending on the magnitude of the second intensity threshold, where the delay time increases as the second intensity threshold increases). This delay time helps avoid accidentally identifying a deep press input. As another example, for some "deep press" inputs, a period of reduced sensitivity occurs after the first intensity threshold is reached. During this period of reduced sensitivity, the second intensity threshold increases. This temporary increase in the second intensity threshold also helps avoid accidental deep press inputs.For other deep press inputs, the response to detecting the deep press input does not depend on time-based criteria.

[0228] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact motion, input timing, application running, rate at which intensity is applied, number of simultaneous inputs, user history, environmental factors (e.g., ambient noise), focus selector position, etc. Example factors are described in U.S. patent applications 14 / 399,606 and 14 / 624,296, which are incorporated herein by reference in their entireties.

[0229] For example, Figure 4C A dynamic intensity threshold 480 is shown that varies over time based in part on the intensity of the touch input 476 over time. The dynamic intensity threshold 480 is the sum of two components: a first component 474 that decays over time after a predefined delay time p1 from when the touch input 476 is initially detected, and a second component 478 that tracks the intensity of the touch input 476 over time. The initial high intensity threshold of the first component 474 reduces accidental triggering of a "deep press" response while still allowing an immediate "deep press" response if the touch input 476 provides sufficient intensity. The second component 478 reduces the unintentional triggering of a "deep press" response through gradual intensity fluctuations of the touch input. In some embodiments, when the touch input 476 satisfies the dynamic intensity threshold 480 (e.g., at Figure 4C ), triggering a “deep press” response.

[0230] Figure 4D Another dynamic intensity threshold 486 (eg, intensity threshold I D ). Figure 4D Two other intensity thresholds are also shown: a first intensity threshold I H and the second intensity threshold I L .exist Figure 4D Although the touch input 484 meets the first intensity threshold I before time p2, H and the second intensity threshold I L , but no response is provided until a delay time p2 has elapsed at time 482. Similarly, Figure 4D In the example, the dynamic intensity threshold 486 decays over time, wherein the decay is from time 482 (triggering the second intensity threshold I L This type of dynamic intensity threshold reduction begins at time 488 after the trigger is triggered with a lower threshold intensity (such as the first intensity threshold I H Or the second intensity threshold I L ) is triggered after or simultaneously with the response associated with the dynamic intensity threshold I D The associated response.

[0231] Figure 4E Another dynamic intensity threshold 492 (eg, intensity threshold I D ).exist Figure 4E In the example, after a delay time p2 has passed since the touch input 490 was initially detected, the trigger is triggered with the intensity threshold I L At the same time, the dynamic intensity threshold 492 decays after a predefined delay time p1 has passed since the touch input 490 was initially detected. L The associated response then reduces the intensity of the touch input 490, and then increases the intensity of the touch input 490 without releasing the touch input 490, which may trigger an action that is consistent with the intensity threshold I. D The associated response (e.g., at time 494) is performed even when the intensity of touch input 490 is below another intensity threshold (e.g., intensity threshold I L ) is also the case.

[0232] The contact characteristic strength is lower than the light press strength threshold IT L The intensity increases to between the light press intensity threshold IT L and deep press intensity threshold IT D The intensity between the deep press intensity threshold IT is sometimes referred to as a "light press" input. D The intensity of the deep press increases to above the deep press intensity threshold IT DThe intensity of the contact feature increases from an intensity below the contact detection intensity threshold IT0 to an intensity between the contact detection intensity threshold IT0 and the light press intensity threshold IT0. L A decrease in the characteristic intensity of a contact from an intensity above the contact detection intensity threshold IT0 to an intensity below the contact detection intensity threshold IT0 is sometimes referred to as detecting liftoff of the contact from the touch surface. In some embodiments, IT0 is zero. In some embodiments, IT0 is greater than zero. In some examples, shaded circles or ellipses are used to represent the intensity of a contact on the touch-sensitive surface. In some examples, unshaded circles or ellipses are used to represent corresponding contacts on the touch-sensitive surface without specifying the intensity of the corresponding contacts.

[0233] In some embodiments described herein, one or more operations are performed in response to detecting a gesture that includes a corresponding press input or in response to detecting a corresponding press input performed using a corresponding contact (or multiple contacts), wherein the corresponding press input is detected at least in part based on detecting an increase in the strength of the contact (or multiple contacts) to above a press input strength threshold. In some embodiments, the corresponding operation is performed in response to detecting an increase in the strength of the corresponding contact to above the press input strength threshold (e.g., the corresponding operation is performed on a "down stroke" of the corresponding press input). In some embodiments, the press input includes an increase in the strength of the corresponding contact to above the press input strength threshold and a subsequent decrease in the strength of the contact to below the press input strength threshold, and the corresponding operation is performed in response to detecting that the strength of the corresponding contact subsequently decreases to below the press input threshold (e.g., the corresponding operation is performed on an "up stroke" of the corresponding press input).

[0234] In some embodiments, the device employs intensity hysteresis to avoid unintended input, sometimes referred to as "jitter," where the device defines or selects a hysteresis intensity threshold that has a predefined relationship to a press input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable proportion of the press input intensity threshold). Thus, in some embodiments, a press input includes an increase in the intensity of the corresponding contact to above the press input intensity threshold and a subsequent decrease in the intensity of the contact to below a hysteresis intensity threshold corresponding to the press input intensity threshold, and a corresponding operation is performed in response to detecting that the intensity of the corresponding contact subsequently decreases below the hysteresis intensity threshold (e.g., performing the corresponding operation on an "up stroke" of the corresponding press input). Similarly, in some embodiments, a press input is detected only when the device detects that the contact strength increases from an strength equal to or less than a hysteresis strength threshold to an strength equal to or greater than a press input strength threshold and, optionally, the contact strength subsequently decreases to an strength equal to or less than the hysteresis strength, and a corresponding operation is performed in response to detecting the press input (e.g., an increase in contact strength or a decrease in contact strength, depending on the circumstances).

[0235] For ease of explanation, descriptions of operations performed in response to a press input associated with a press input intensity threshold, or in response to a gesture including a press input, are optionally triggered in response to detecting: the intensity of the contact increasing above the press input intensity threshold, the intensity of the contact increasing from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, the intensity of the contact decreasing below the press input intensity threshold, or the intensity of the contact decreasing below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where operations are described as being performed in response to detecting a decrease in the intensity of the contact below the press input intensity threshold, the operations are optionally performed in response to detecting a decrease in the intensity of the contact below a hysteresis intensity threshold that corresponds to and is less than the press input intensity threshold. As described above, in some embodiments, triggering of these operations is also dependent on satisfying a time-based criterion (e.g., a delay time having elapsed between satisfying the first intensity threshold and satisfying the second intensity threshold).

[0236] User interface and associated processes

[0237] Attention now turns to embodiments of a user interface (“UI”) and associated processes that may be implemented on an electronic device such as portable multifunction device 100 or device 300 having a display, a touch-sensitive surface, (optionally) one or more tactile output generators for generating tactile output, and (optionally) one or more sensors for detecting intensity of contact with the touch-sensitive surface.

[0238] Figures 5A1 to 5A67An exemplary user interface for interacting with a user interface object and providing non-visual feedback according to some embodiments is shown. The user interfaces in these figures are used to illustrate the processes described below, including 6A to 6D and 7A to 7B According to some embodiments, Figures 9A to 9G and Figures 10A to 10E The process shown and Figures 5E1 to 5E4 The various aspects of the interaction model shown are also Figures 5A1 to 5A67 and the accompanying description. For ease of explanation, some of the embodiments will be discussed with reference to operations performed on a device having a touch-sensitive display system 112. In such embodiments, the focus selector is optionally: a corresponding finger or stylus contact, a representative point corresponding to a finger or stylus contact (e.g., the center of gravity of the corresponding contact or a point associated with the corresponding contact), or the center of gravity of two or more contacts detected on the touch-sensitive display system 112. However, in response to detecting a contact on the touch-sensitive surface 451 when the user interface shown in the figures is displayed on the display 450 together with the focus selector, similar operations are optionally performed on a device having a display 450 and a separate touch-sensitive surface 451.

[0239] Figures 5A1 to 5A4 Activation of a display (e.g., touch-sensitive display 112) in response to changing the orientation of the device 100 is shown. When the display is activated, notifications received while the device was in a screen-off state are displayed. In some embodiments, the device uses one or more sensors (e.g., accelerometers, gyroscopes, audio sensors, thermal sensors, and / or light sensors) to determine whether the orientation of the device has changed. For example, the device determines whether the device has been rotated beyond a threshold angle (e.g., rotated along an axis of the device, such as tilting the device from a substantially horizontal position to a substantially vertical position of the device). Figure 5A1 , the device is in the screen-off state when the device is held flat in the user's hand 5002 such that the device display is substantially horizontal. Figure 5A2 Tilt the device so that the display is larger than Figure 5A1 More vertical. Because the tilt angle of the device does not increase above Figure 5A2 The threshold tilt angle is set so the display is not activated. Figure 5A3 Tilt the device so that the display is larger than Figure 5A2 More vertical. As the device's tilt angle increases above Figure 5A3 The threshold tilt angle in , so the display transitions from the screen-off state to the screen-on state and displays a wake-up screen user interface 5004 on the touch screen 112 (e.g., the initial user interface displayed when the device transitions from the screen-off state to the screen-on state). Figure 5A4 , the touch screen 112 is substantially vertical.

[0240] exist Figure 5A4 In , a lock icon 5005 is shown in the wake screen user interface 5004 to indicate that the current device is locked. Figure 5A5 , lock icon 5005 is replaced by unlock icon 5007 to indicate that the device has been unlocked (e.g., in response to receiving user authentication information, such as biometric information of the device user or a passcode for the device). The wake screen user interface 5004 in locked mode optionally displays a larger image than the wake screen user interface 5004 in unlocked mode (e.g., displaying an unedited version of the notification, such as Figure 5A5 ) less information (e.g., showing an edited version of the notification, such as Figure 5A4 shown).

[0241] exist Figure 5A5 , the wake screen user interface 5004 displays notifications 5006, 5008, 5010, and 5012 corresponding to events (e.g., events that occurred when the device 100 was in the screen-off state), which correspond to different applications. For example, notification 5006 corresponds to an event generated by an application whose application title is "Social Media" (as indicated by the application identification information in notification 5006). Notification 5006 also includes an icon corresponding to the social media application (e.g., a graphic for the application launch icon) and a reception time indication (e.g., "now" or "x minutes ago"). In some embodiments, these notifications are displayed on the wake screen user interface 5004 in the order in which they were received at the device. In some embodiments, related notifications (e.g., by sender, by source, by conversation thread, by application, etc.) are optionally aggregated into an overlay, with the most recently received notification in the overlay displayed at the top.

[0242] exist Figure 5A6 , a contact (e.g., contact 5024) is detected at time T0 on touch screen 112 at a location corresponding to notification 5008. According to some embodiments, detection of the contact is triggered when the intensity of the contact exceeds a nominal contact detection intensity threshold IT0. In some embodiments, the device is capable of detecting a range of intensities above the nominal contact detection intensity threshold IT0, and optionally comparing the intensity of the contact with one or more additional intensity thresholds (e.g., IT0) above the nominal contact detection intensity threshold IT0. L and IT D). In some embodiments, the device is unable to distinguish contact intensities above the nominal contact detection intensity threshold IT0 and does not compare the intensity of the contact to additional intensity thresholds above the nominal contact detection intensity threshold IT0. In some embodiments, detection of a contact triggers monitoring movement of the contact after detection of the contact and the duration that the contact remains substantially stationary (e.g., has less than a threshold amount of movement within a given time unit) on the touch screen. Figure 5A5 As shown, in response to detecting contact 5024, visual feedback is provided to indicate the target of the input on the wake screen user interface 5004 (e.g., notification 5008 is slightly highlighted while other notifications displayed on the wake screen user interface 5004 remain unchanged). In some embodiments, other visual changes (e.g., enlarging or animating notification 5008) are optionally used to call out the target of the input for other parts or components of the wake screen user interface 5004. In some embodiments, until the input made through contact 5024 satisfies the input threshold, the device forgoes providing any tactile output or audio alert at this time (e.g., for detecting the contact or selection of the target of the input).

[0243] Figure 5A6 to Figure 5A7 An exemplary scenario is shown in which the input through contact 5024 is a tap input that includes less than a threshold amount of movement before contact 5024 is lifted and does not meet a predefined input threshold (e.g., a time-based threshold, an intensity-based threshold, or an input threshold based on the time and intensity of the input). In some embodiments, for devices that detect multiple contact intensity levels, the predefined input threshold is a duration threshold T(I) that has a first value when the intensity of the input is above the nominal contact detection intensity threshold IT0, but does not exceed a first intensity threshold (e.g., a light press intensity threshold IT0) that is above the nominal contact detection intensity threshold IT0. L ). When the intensity of the input exceeds the first intensity threshold, the duration threshold T(I) has a second value that is less than the first value. According to some embodiments, when the intensity of the contact exceeds the first intensity threshold for a first time after the contact is detected, a switch from the first value to the second value within the threshold duration T(I) is triggered. In some embodiments, for devices that do not distinguish between contact intensities above the nominal contact detection intensity threshold, the predefined input threshold is a first threshold duration (e.g., the first value) that does not change during the input. Figure 5A7In response to detecting the termination of an input (e.g., the lift-off of contact 5024) and based on determining that the input is substantially stationary and the input ends without meeting a predefined input threshold (lift-off detected at T < T0 + T(I)) (e.g., by lift-off of contact or a decrease in contact strength to below a predefined strength threshold (e.g., IT0)), the device launches an application corresponding to notification 5008 (e.g., a messaging application), and replaces the wake screen user interface 5004 with the user interface of the application on the touch screen 112 (e.g., the session user interface 5014 of the messaging application). In this example, the session user interface 5014 of the messaging application includes extended content corresponding to notification 5008 (e.g., message 5018 includes all the text content shown in notification 5008 and a media object icon 5022 of an animated media object corresponding to the text content accompanying message 5018). The user interface 5014 of the messaging application also includes a message presentation area 5016 that displays previous messages exchanged in the same session as well as message 5018. The user interface 5014 also includes user interface objects for navigating within the messaging application (e.g., a back button for navigating to the session list user interface within the messaging application) and providing input to the messaging application (e.g., a message input box and virtual keyboard 5020 for writing a new message in the session).

[0244] Figure 5A6 Subsequent FIG. 5A8 shows an alternative exemplary scenario different from that Figure 5A6 to Figure 5A7 shown. In Figure 5A6In the exemplary scenario shown in Figure 5A8, before the end of the input is detected, the input has met a predefined input threshold (e.g., contact 5024 remains on the touch screen and remains substantially stationary for more than a threshold duration T(I)). As shown in Figure 5A8, when the contact has remained at the same position above notification 5008 for at least a threshold amount of time T(I) after contact 5024 is detected, the device provides a visual indication that the input threshold has been met (e.g., the size of notification 5008 is reduced, while other notifications in the wake-up screen user interface remain unchanged). In conjunction with the visual feedback, the device generates a non-visual output including a tactile output (e.g., tactile output 5028) to indicate that the input has met the input threshold. In some embodiments, the tactile output 5028 has a first tactile output representation, including a tactile output having a first tactile output mode (e.g., a first amplitude, frequency, and waveform pattern), and is optionally accompanied by a tactile audio output having a first tactile audio output mode (e.g., a first amplitude, frequency, waveform pattern, and an offset from the tactile output). In some embodiments, the appearance of the user interface is not responsive to continuous changes in contact intensity until the input threshold is met, and the user interface is capable of detecting a first intensity threshold IT above the nominal contact intensity detection threshold IT0. L The device determines that the intensity of the contact has exceeded a first intensity threshold (eg, IT L ), adjusting the input threshold from a first value to a second value (eg, a value smaller than the first value).

[0245] Figure 5A8 and after Figure 5A9 5A8 (e.g., the state entered when the input first meets the input threshold) until the input ends (e.g., when the contact remains continuously on touch screen 112 with at least a threshold amount of intensity (e.g., contact detection intensity IT0) and until movement of the contact exceeding the threshold amount is detected (e.g., when contact 5024 remains substantially stationary). This is a stable state in which the user has time and opportunity to decide how to proceed (e.g., by how the input continues and / or ends) to achieve a desired outcome. In this state, notification 5008 is displayed at a reduced size, while all other notifications and other components of wake screen user interface 5004 remain unchanged and fully visible.

[0246] Figure 5A10 from Figure 5A8 or Figure 5A9 Continuing, an exemplary scenario is shown where input ends without movement after the input threshold has been met. As shown in FIG5A10 , in response to detecting lift-off of contact 5024 after the input threshold has been met (e.g., as shown in FIG5A20 ), Figure 5A8 or Figure 5A9), the device displays a system user interface or a system user interface element that includes an expanded version of notification 5008 (e.g., expanded notification 5032), and optionally, one or more user interface objects corresponding to different actions that can be performed within and / or outside of the application with respect to notification 5008 (e.g., virtual keyboard 5020, a message input field for replying, a "close" button 5034, etc.). In conjunction with displaying the system user interface (e.g., a user interface that occupies the entire touch screen, or a system user interface element that occupies less than the entire touch screen and is overlaid on a background user interface (e.g., a blurred and dimmed version of the previous user interface)), the device generates non-visual output, including tactile output (e.g., tactile output 5031), to indicate that the end of the input has been detected, and displays the expanded notification in response to the end of the input. In some embodiments, the tactile output 5031 has a second tactile output manifestation, including a tactile output having a second tactile output mode (e.g., a second amplitude, frequency, and waveform pattern), and is optionally accompanied by a tactile audio output having a second tactile audio output mode (e.g., a second amplitude, frequency, waveform pattern, and an offset from the tactile output). In some embodiments, the tactile output 5031 provided at the end of the input has a tactile output manifestation that is different from the tactile output 5025 provided when the input threshold is met (e.g., having a different tactile output mode for the tactile output, having a different tactile output mode for the tactile output, and having a different tactile audio output mode for the accompanying tactile audio output, or having the same tactile output mode for the tactile output but having a different tactile audio output mode for the accompanying tactile audio output). The different tactile output manifestations for the end of the input and for the tactile output that meets the input threshold alert the user to different changes that have occurred in the internal state of the device. In some embodiments, the tactile output for the end of input and for the satisfaction of an input threshold appears similar or identical, in contrast to the tactile output provided when the input includes movement of a user interface object under contact away from its position.

[0247] In this example, expanded notification 5032 is displayed over background 5030 (e.g., a blurred and darkened version of the original wake screen user interface 5004). Virtual keyboard 5020 is displayed below expanded notification 5032. Expanded notification 5032 includes all text content shown in notification 5008 and the corresponding accompanying Figure 5A75018. The expanded notification 5032 differs from the user interface 5014 of the messaging application in that the expanded notification 5032 does not include previous messages exchanged in the same conversation as the message 5018 and does not include user interface objects for navigating within the messaging application (e.g., a back button for navigating to a conversation list user interface within the messaging application). In some embodiments, input typed in the text input box of the virtual keyboard 5020 is treated as input for a reply to the message 5018. In some embodiments, a tap input on the expanded notification 5032 causes the device to dismiss the expanded notification 5032, launch the messaging application, and display the message. Figure 5A6 , and a user interface 5014 of a messaging application shown in . In some embodiments, when text input is provided in the text input box of the virtual keyboard 5020 and a send input is provided (e.g., using a send button next to the message input box), the expanded notification 5032 is cleared and a reply message is sent to the sender of the message 5018. In some embodiments, if a tap input is detected on a control 5034 (e.g., a "close" button) displayed concurrently with the expanded notification 5032, the currently displayed system user interface or system user interface element (e.g., including the expanded notification 5032, the control 5034, and the virtual keyboard 5020) is dismissed, and the wake screen user interface 5004 is restored on the touch screen with the notification 5008 removed from the wake screen user interface 5004. In some embodiments, when the wake screen user interface 5004 is restored after the notification 5008 is cleared, the device does not generate non-visual output (e.g., tactile output) to accompany the visual change in the user interface.

[0248] Figures 5A11 to 5A12 Following Figure 5A10, and illustrating that when a tap input (e.g., a tap input by contact 5036) is detected (e.g., at time T1) outside the system user interface (e.g., in an area outside of expanded notification 5032 and any accompanying user interface objects (e.g., virtual keyboard 5020 and controls 5034) on background 5030), the device stops displaying the system user interface and resumes wake screen user interface 5004 with all notifications (e.g., including notification 5008) unchanged (e.g., Figure 5A125A10 ). In some embodiments, when the wake screen user interface 5004 is redisplayed at its original position with the notification 5008, the device generates a non-visual output (e.g., tactile output 5038), for example, to indicate that the notification 5008 has stabilized to its original position in the wake screen user interface 5004. In some embodiments, the tactile output 5038 provided in conjunction with the stabilization of the notification 5008 (e.g., as shown in 5A12) has a third tactile output representation that is different from the tactile output 5031 provided in conjunction with the presentation of the expanded notification 5032 (e.g., as shown in FIG. 5A10). In some embodiments, the tactile output 5038 provided in conjunction with the stabilization of the notification 5008 (e.g., as shown in 5A12) has the same tactile output representation as the tactile output 5031 provided in conjunction with the presentation of the expanded notification 5032 (e.g., as shown in FIG. 5A10) or a similar tactile output, as contrasted with the tactile output representation of the tactile output provided in conjunction with the movement of the user interface object in response to the drag input.

[0249] Figure 5A8 or Figure 5A9 Figure 5A13 below shows an exemplary scenario in which movement of contact 5024 exceeding a threshold amount (e.g., movement toward the right side of the display) is detected after the input threshold has been met. In some embodiments, as shown in Figure 5A13, in response to the movement of contact 5024, the input threshold is reset (e.g., input through contact 5024 no longer meets the input threshold), and the appearance of notification 5008 is restored (e.g., back to the display). Figure 5A5 or Figure 5A6 In some embodiments, in response to movement of contact 5024 and restoration of the appearance of notification 5008, the device does not provide non-visual output (e.g., tactile output) in conjunction with resetting the input threshold.

[0250] Figure 5A8 or Figure 5A9 Figures 5A14 to 5A14 Figure 5A16 An exemplary scenario is shown in which movement of contact 5024 exceeding a threshold amount (e.g., movement toward the left side of the display) is detected after the input threshold has been met. In some embodiments, as shown in Figures 5A14 to 5A15, Figure 5A15 As shown, in response to movement of contact 5024, notification 5008 is dragged on wake screen user interface 5004 according to the movement of contact 5024. When notification 5008 is dragged off the touch screen, one or more user interface controls corresponding to different operations that can be performed with respect to notification 5008 are displayed next to notification 5008 (e.g., control 5040 for displaying a configuration user interface, control 5042 for displaying expanded notification 5032, and control 5044 for clearing notification 5008). Figure 5A165A14 to 5A15 , the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5048)) in conjunction with contact 5024 crossing the position threshold, removing notification 5008 from the wake screen user interface 5004, and / or repositioning lower notifications 5010 and 5012 in the wake screen user interface 5004. In some embodiments, a left swipe input detected by contact on notification 5008 clears notification 5008 from the wake screen user interface 5004 even if the input threshold had not been met prior to movement of the contact (e.g., in a manner similar to that shown in FIG. 5A14 to 5A15 ). Figure 5A16 in the same manner as shown).

[0251] Figure 5A6 Figures 5A17 to 5A17 Figure 5A18 An exemplary scenario is shown in which movement of contact 5024 exceeding a threshold amount is detected after an input threshold has been met. In some embodiments, as shown in FIG. Figure 5A18 As shown, in response to movement of contact 5024, notifications on the wake screen user interface 5004 (e.g., including notifications 5006, 5008, 5010, and 5012) scroll upward in unison. When the notifications present on the wake screen user interface 5004 scroll as contact 5024 moves, in response to determining that the movement of contact 5024 exceeds a position threshold or a distance threshold, when contact 5024 is first detected, a notification history including previously received notifications not shown in the wake screen user interface 5004 (e.g., notifications 5049 and 5050) is displayed and scrolled along with the existing notifications on the wake screen user interface 5004. In some embodiments, a non-visual output (e.g., a tactile output) is generated by the device in conjunction with the contact and / or display of the notification history that crosses the position or distance threshold. In some embodiments, an upward swipe input detected by contact on notification 5008 scrolls the notifications on the wake screen user interface and causes the notification history to be displayed (e.g., in a manner consistent with Figures 5A17 to 5A18 ), even if the input threshold has not been met prior to the movement of the contact. Figure 5A18 in the same manner as shown).

[0252] Figure 5A19 Start a new example scenario, which optionally starts with Figure 5A7 Continue, or continue from a different process in which the input triggers display of the messaging application's user interface 5014 from a different starting state. Figure 5A19As shown, a contact (e.g., contact 5052) is detected (e.g., at time T3) on touch screen 112 at a location corresponding to media object icon 5022, which corresponds to an animated media object that accompanies the text content of message 5018. In response to detecting contact 5052 on media object icon 5022, media object icon 5022 is optionally highlighted (e.g., stroked) to indicate its changed state as the target object of input made by contact 5052. Other portions of the user interface 5014 of the messaging application do not change in response to detecting contact 5052. Detection of the contact is triggered when the intensity of contact 5052 exceeds a nominal contact detection intensity threshold IT0. In some embodiments, the device is capable of detecting a range of intensities above the nominal contact detection intensity threshold IT0, and optionally comparing the intensity of contact 5052 to one or more additional intensity thresholds (e.g., IT0) above the nominal contact detection intensity threshold IT0. L and IT D ) is compared. In some embodiments, the device is unable to distinguish contact intensities above the nominal contact detection intensity threshold IT0 and does not compare the intensity of contact 5052 to an additional intensity threshold above the nominal contact detection intensity threshold IT0. In some embodiments, detection of contact 5052 triggers monitoring of movement of contact 5052 after contact 5052 is detected and the duration for which contact 5052 remains substantially stationary (e.g., having less than a threshold amount of movement within a given time unit). In some embodiments, the device forgoes providing any non-visual output (e.g., tactile output or audio alert) at this time (e.g., for detecting contact or selection of a target for input) until the input made through contact 5052 satisfies the input threshold.

[0253] Figure 5A19 to Figure 5A20 An exemplary scenario is shown in which the input through contact 5052 is a tap input that includes less than a threshold amount of movement before contact 5052 is lifted and does not meet a predefined input threshold (e.g., an intensity-independent threshold, or an intensity threshold that is independent of duration, or an intensity-based duration threshold). In some embodiments, for devices that detect multiple contact intensity levels, the predefined input threshold is a duration threshold T(I) that has a first value when the intensity of the input is above the nominal contact detection intensity threshold IT0 and does not exceed a first intensity threshold (e.g., a light press intensity threshold IT0) that is above the nominal contact detection intensity threshold IT0. L)。When the intensity of the input exceeds the first intensity threshold, the duration threshold T(I) has a second value that is less than the first value. According to some embodiments, when the intensity of the contact exceeds the first intensity threshold for a first time after the contact is detected, a switch from the first value to the second value within the threshold duration is triggered. In some embodiments, for a device that does not distinguish contact intensities above the nominal contact detection intensity threshold IT0, the predefined input threshold is a first threshold duration (e.g., the first value) that does not change during the input. In Figure 5A20 In response to detecting the termination of the input (e.g., the lift-off of contact 5052) and based on determining that the input is substantially stationary and the input ends without meeting the predefined input threshold (lift-off detected when T < T3 + T(I)) (e.g., by lift-off of the contact or the contact intensity decreasing below the predefined intensity threshold (e.g., IT0)), the device launches the application corresponding to the media object icon 5022 (e.g., the emoji media recording application) and replaces the session user interface 5014 with the user interface of the emoji media recording application (e.g., user interface 5054) on the touchscreen 112. In this example, the user interface 5054 of the emoji media recording application includes media content corresponding to the media object icon 5022 (e.g., an animated emoji video object), and user interface objects 5058 for navigating within the emoji media recording application (e.g., for navigating to a list user interface of media objects within the emoji media recording application or for returning to a previous user interface outside the emoji media recording application (e.g., returning to the messaging application)) and for providing input to the emoji media recording application (e.g., a share button for sharing the animated emoji video object, a pause button for stopping the playback of the animated emoji video object, and controls for displaying other options).

[0254] Figure 5A19 Subsequent Figure 5A21 shows an alternative scenario from the Figure 5A19 to Figure 5A20 scenario shown. In Figure 5A19And in the exemplary scenario shown in 5A21 thereafter, before the end of the input is detected, the input has met a predefined input threshold (e.g., the contact remains substantially stationary for more than a threshold duration T(I)). As shown in Figure 5A21, when the contact has remained at the same position above the media object icon 5022 for at least a threshold amount of time T(I) after contact 5052 is detected, the device provides a visual indication that the input threshold has been met (e.g., the size of the media object icon 5022 is reduced and displayed on a gray disk, while other parts of the message user interface 5014 remain unchanged). In conjunction with the visual feedback, the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5060)) to indicate that the input has met the input threshold. In some embodiments, the tactile output 5060 has a first tactile output representation, including a tactile output having a first tactile output mode (e.g., a first amplitude, frequency, and waveform pattern), and optionally accompanied by a tactile audio output having a first tactile audio output mode (e.g., a first amplitude, frequency, waveform pattern, and an offset from the tactile output). In some embodiments, the appearance of the user interface is not responsive to continuous changes in the intensity of contact 5052 until the input threshold is met, and the user interface is not responsive to continuous changes in the intensity of contact 5052 for a user who is able to detect a first intensity threshold (e.g., IT0) that is higher than a nominal contact intensity detection threshold (e.g., IT0). L ) in intensity, adjusting the input threshold T(I) from a first value to a second value (e.g., a value less than the first value) based on determining that the intensity of the contact has exceeded the first intensity threshold.

[0255] Figure 5A21 after Figure 5A22 5A21 (e.g., the state entered when the input first meets the input threshold) before the input ends (e.g., when contact 5052 is continuously maintained on touch screen 112 with at least a threshold amount of intensity (e.g., contact detection intensity IT0) and before movement of contact 5052 exceeding the threshold amount is detected (e.g., when contact 5052 remains substantially stationary). This is a stable state in which the user has time and opportunity to decide how to proceed (e.g., by inputting a manner to continue and / or end) to achieve a desired result. In this state, media object icon 5022 is displayed in a reduced size overlaid on the gray disk, while other portions of message user interface 5014 remain unchanged and fully visible.

[0256] Figure 5A23 continues from 5A21 or 5A22 and shows an exemplary scenario where the input ends without movement after the input threshold has been met. Figure 5A21 or Figure 5A225066 of the animated emoji video object represented by the media object icon 5022) and, optionally, one or more user interface objects corresponding to different operations that can be performed within and / or outside the application with respect to the animated emoji video object represented by the media object icon 5022. The one or more user interface objects corresponding to the different operations include, for example, a copy button 5070 for making a copy of the animated emoji video object on a clipboard, a save button 5072 for saving a copy of the animated emoji video object in a video library, and a fast-forward button 5074 for sending a copy of the animated emoji video object via a communication application (e.g., an email or instant messaging application). In conjunction with displaying a system user interface (e.g., a user interface that occupies the entire touch screen or a system user interface element that occupies less than the entire touch screen and is overlaid on a background user interface 5064 (e.g., a blurred and dimmed version of the previous user interface 5014)), the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5062) to indicate that the end of the input has been detected, and in response to the end of the input, displays a preview 5066 of an animated emoji video object corresponding to the media object icon 5022. In some embodiments, the tactile output 5062 provided at the end of the input has a second tactile output representation that is different from the first tactile output representation of the tactile output 5060 provided when the input threshold is met (e.g., a tactile output having a different appearance for the tactile output 5060). There are different tactile output modes, different tactile output modes for tactile output and different tactile audio output modes for accompanying tactile audio output, or the same tactile output mode for tactile output but different tactile audio output modes for accompanying tactile audio output). The different tactile output performances for the end of input and for the tactile output for satisfying the input threshold alert the user to different changes that have occurred in the internal state of the device. In some embodiments, the tactile output performances for the end of input and for the tactile output for satisfying the input threshold are similar or identical, which contrasts with the tactile output provided when the input includes moving a user interface object (e.g., media object icon 5022) under the contact (e.g., contact 5052) away from its position.

[0257] In this example, preview 5066 is displayed above background 5064 (e.g., a blurred and darkened version of original message user interface 5014). User interface objects 5070, 5072, and 5074 corresponding to different actions that can be performed with respect to the animated emoji video object are displayed below preview 5066 (e.g., when the object is activated by a tap input (e.g., by a contact different from contact 5052), the corresponding action will be performed). Preview 5066 includes all or a portion of the content of the animated emoji video object beyond that shown in media object icon 5022. Figure 5A20 The user interface 5054 of the media object recording application shown differs in that preview 5066 does not include all of the controls present in user interface 5054 (e.g., control 5058) and does not include user interface objects for navigating within the media object recording application (e.g., a "Done" button).

[0258] Figure 5A24 5A23 shows that tap input on various parts of the system user interface or outside the system user interface causes the device to perform different operations. In some embodiments, a tap input detected on preview 5066 (e.g., by contact 5080) causes the device to dismiss preview 5066, launch a media object recording application, and display Figure 5A20 In some embodiments, a tap input (e.g., contact 5078) detected on a control 5068 (e.g., a "close" button) displayed concurrently with the preview 5066 causes the device to dismiss the currently displayed system user interface or system user interface element (e.g., including the preview 5066 and associated controls) and restore the session user interface 5014 (e.g., as shown in FIG. Figure 5A25 In some embodiments, input detected on background 5064 (e.g., by contact 5076) outside the area occupied by the system user interface (e.g., outside preview 5066 and associated controls) causes the device to dismiss the currently displayed system user interface or system user interface element (e.g., including preview 5066 and associated controls) and restore session user interface 5014 (e.g., as shown). Figure 5A25 In some embodiments, when the session user interface 5014 is restored, the device does not generate non-visual output (e.g., tactile output) to accompany the visual change in the user interface. In some embodiments, when the session user interface 5014 is restored, after the preview 5066 is collapsed into the media object icon 5022, the device generates non-visual output (e.g., tactile output (e.g., tactile output 5084)) in conjunction with the media object icon 5022 stabilizing to its original position in the session user interface 5014 (e.g., as shown in FIG. Figure 5A25In some embodiments, tactile output 5084 provided in conjunction with the stabilization of media object icon 5022 (e.g., as shown in 5A25) has a third tactile output representation that is different from the tactile output representation of tactile output 5062 provided in conjunction with presentation preview 5066 (e.g., as shown in FIG. 5A23). In some embodiments, tactile output 5084 provided in conjunction with the stabilization of media object icon 5022 (e.g., as shown in 5A25) has the same tactile output representation as or a similar tactile output representation as tactile output 5062 provided in conjunction with presentation preview 5066 (e.g., as shown in FIG. 5A23), as compared to the tactile output representation of the tactile output provided in conjunction with the movement of the user interface object in response to the drag input.

[0259] Figure 5A21 or Figure 5A22 Subsequent Figure 5A26 shows an exemplary scenario in which movement of contact 5052 exceeding a threshold amount is detected after the input threshold has been met. In some embodiments, as shown in Figure 5A26, in response to the movement of contact 5052, a representation of media object icon 5022 (e.g., copy 5090) is dragged from the original position of media object icon 5022 in session user interface 5014 in accordance with the movement of contact 5052. The device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5086)) having a fourth tactile output representation that is different from the first tactile output representation of the tactile output generated when the input threshold is met (e.g., tactile output 5060 in Figure 5A21) and / or the second tactile output representation of the tactile output generated when the input ends without movement after the input threshold has been met (e.g., tactile output 5062 in Figure 5A23). In some embodiments, as shown in Figure 5A26, when contact 5052 is maintained on the touch screen, maintaining the copy 5090 of the media object icon 5022, the media object icon 5022 displayed at its original location becomes grayed out.

[0260] Figure 5A26 and after Figures 5A27 to 5A29 As shown, when the copy 5090 of the media object icon 5022 is held by the contact 5052, the device detects one or more inputs that cause the device to navigate from the conversation user interface 5014 to the user interface 5100 of the memo application, including Figure 5A27 An upward edge swipe input detected on the user interface 5014 in the home screen by contact 5092 causing display of the home screen user interface 5096 is followed by Figure 5A285096 on the home screen user interface 5096 of the memo application. When the contact 5052 is maintained on the touch screen, the copy 5090 of the media object icon 5022 moves with the contact 5052, and the user interface changes on the touch screen in response to the one or more inputs. Figure 5A29 As shown, the user interface 5100 of the memo application includes a content area 5102 that includes an allowed drop location 5104 for an animated emoticon video object corresponding to the media object icon 5022 or its copy 5090, and a virtual keyboard 5020 for typing text input into the content area 5102.

[0261] exist Figure 5A30 , termination of input through contact 5052 is detected (e.g., lift-off of contact 5052), and in response to detecting the end of input through contact 5052, a copy of the animated emoji video object 5108 is displayed at an insertion location 5104 in the content area 5102 of the user interface 5100 of the memo application. In some embodiments, the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5106)) in conjunction with inserting and / or stabilizing the copy of the animated emoji video object 5108 into the insertion location 5104. In some embodiments, the tactile output representation of the tactile output (e.g., tactile output 5106) generated in conjunction with inserting and / or stabilizing a dragged object into an insertion location in a user interface (and detecting lift-off of the contact after movement of the contact, where the movement is after an input threshold has been met) is different from the tactile output representation of the tactile output provided in conjunction with satisfying the input threshold (e.g., tactile output 5060 in Figure 5A21) and / or the tactile output representation of the tactile output provided in conjunction with displaying a preview (e.g., tactile output 5062 in Figure 5A23) (and detecting lift-off of the contact without movement of the contact after the input threshold has been met).

[0262] Figures 5A31 to 5A34 Interaction with a home screen user interface and application icons displayed on the home screen user interface is shown. Figure 5A31Among them, on the home page of the main screen user interface (e.g., the main screen user interface 5096), multiple application launch icons are displayed, including the application launch icon 424 of the messaging application. A contact (e.g., contact 5110) is detected at the position on the touch screen 112 corresponding to the messaging application icon 424 at time T0. In some embodiments, in response to detecting the contact 5110, visual feedback is provided to indicate the target of the input on the main screen user interface 5096 corresponding to the position of the contact 5110 on the touch screen (e.g., highlighting, magnifying, and / or animating the application launch icon 424 while other application launch icons remain unchanged). In some embodiments, before the input made through the contact 5110 meets the input threshold (e.g., a duration threshold (e.g., a static duration threshold or a predefined intensity-based duration threshold T(I))), the device abandons providing any non-visual output at this time (e.g., haptic output or audio alert) (e.g., for detecting the contact or the selection of the target of the input).

[0263] Figures 5A31 to 5A32 An example is shown where the input made through the contact 5110 includes less movement than the threshold amount and does not meet the predefined input threshold (e.g., the input threshold as described herein) before the contact 5110 is lifted. Figure 5A6 to Figure 5A7 as described. Figure 5A32 Among them, in response to detecting the termination of the input (e.g., the lifting of the contact 5110) and based on the determination that the input is substantially stationary and the input ends without meeting the predefined input threshold (e.g., the lifting occurs at time T < T0 + T(I)), the device launches the application corresponding to the messaging application icon 424 (e.g., the messaging application), and replaces the main screen user interface 5096 with the user interface of the application on the touch screen 112 (e.g., the session user interface 5014 of the messaging application).

[0264] Figures 5A33 to 5A35An exemplary scenario is shown where an input made through contact 5110 includes more than a threshold amount of movement before meeting an input threshold (e.g., the input is a left swipe gesture). Based on determining that the input includes more than a threshold amount of movement of the contact before meeting the input threshold (e.g., detecting movement of contact 5110 at time T < T0+T(I)), the device scrolls the home screen user interface and replaces the primary page of the home screen user interface (e.g., home screen user interface 5096) with a secondary page of the home screen user interface (e.g., home screen user interface 5112). In some embodiments, in response to detecting movement of contact 5110 and lifting of contact 5110, and / or in conjunction with a page transition of the home screen user interface, the device abandons generating non-visual outputs (e.g., haptic outputs or audio alerts).

[0265] Figures 5A36 to 5A37 An exemplary scenario is shown where an input made through contact 5110 meets a predefined input threshold and has less than a threshold amount of movement before meeting the predefined input threshold (e.g., remains substantially stationary). In Figure 5A36 response to detecting that less than a threshold amount of movement of contact 5110 meets the input threshold (e.g., contact 5110 remains substantially stationary for at least a threshold amount of time T(I)), the device displays an information object 5116 (e.g., an educational prompt stating "Release to preview" and "Drag to rearrange") and generates a non-visual output (e.g., a haptic output (e.g., haptic output 5114)) to indicate that the input threshold has been met, and the subsequent portion of the input will be used to determine which of multiple possible responses will be provided. In some embodiments, haptic output 5114 has a first haptic output manifestation, including a haptic output having a first haptic output mode (e.g., a first amplitude, frequency, and waveform pattern), and optionally accompanied by a haptic audio output having a first haptic audio output mode (e.g., a first amplitude, frequency, waveform pattern, and offset from the haptic output). In some embodiments, prior to meeting the input threshold, the appearance of the user interface does not respond to a continuous change in the intensity of contact 5110; and for a device capable of detecting an intensity relative to a first intensity threshold above a nominal contact intensity detection threshold, the input threshold is adjusted from a first value to a second value (e.g., a value less than the first value) based on determining that the intensity of the contact has exceeded the first intensity threshold.

[0266] Figures 5A38 to 5A41 In Figure 5A37 subsequent thereto, an exemplary scenario is shown where lifting of contact 5110 is detected after meeting the input threshold.

[0267] In Figure 5A38, after the input threshold is met, the user interface remains in contact with the touch screen 112 until the input ends (e.g., when contact 5052 is continuously maintained on the touch screen 112 with at least a threshold amount of intensity (e.g., contact detection intensity IT0)) and until movement of contact 5052 exceeding a threshold amount is detected (e.g., when contact 5052 remains substantially stationary). Figure 5A37 96. The information object 5116 is in the same state as shown (e.g., the state entered when the input first satisfies the input threshold). This is a stable state in which the user has time and opportunity to decide how to proceed (e.g., by inputting a continuation and / or termination) to achieve the desired outcome. In this state, the application icons 424 are optionally displayed in a reduced size overlaid on the gray disk, while the rest of the home screen user interface 5096 remains unchanged and fully visible. In some embodiments, if no additional input is received from the user (e.g., lift-off of the contact or movement of the contact), the information object 5116 automatically fades out after a predefined period of time.

[0268] Figure 5A39 Continuing from 5A37 or 5A38, and showing an exemplary scenario where the input ends without movement after the input threshold has been met. Figure 5A39 As shown, in response to the input threshold being met (e.g., Figure 5A37 or Figure 5A38After detecting lift-off of contact 5110 (as shown), the device displays a system user interface or a system user interface element including a quick action menu 5120 (e.g., that, when activated, causes one or more selectable options to perform an operation associated with an application (e.g., compose a new message, compose a message to "Mom," compose a message to "S. Ramanujan," compose a message to "G. Hardy")), and, optionally, one or more mini-application objects (e.g., widgets or mini-applications, plug-ins, modules) associated with the target application icon (e.g., messaging application icon 424). In conjunction with displaying the system user interface (e.g., a user interface that occupies the entire touch screen or a system user interface element that occupies less than the entire touch screen and overlies background user interface 5118 (e.g., a blurred and dimmed version of previous user interface 5096)), the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5117)) to indicate that the end of the input has been detected, and displays a quick action menu for the application corresponding to application icon 424 in response to the end of the input. In some embodiments, the tactile output 5117 provided at the end of the input has a second tactile output manifestation that is different from the first tactile output manifestation of the tactile output 5114 provided when the input threshold is met (for example, having a different tactile output pattern for the tactile output, having a different tactile output pattern for the tactile output and having a different tactile audio output pattern for the accompanying tactile audio output, or having the same tactile output pattern for the tactile output but having a different tactile audio output pattern for the accompanying tactile audio output). The different tactile output manifestations for the tactile output for the end of the input and the tactile output for meeting the input threshold alert the user that different changes have occurred in the internal state of the device. In some embodiments, the tactile output manifestations for the end of the input and for the tactile output for meeting the input threshold are similar or identical, which contrasts with the tactile output provided when the input includes moving a user interface object (for example, application icon 424) under the contact (for example, contact 5110) away from its position.

[0269] In this example, quick action menu 5120 is displayed over background 5118 (e.g., a blurred and darkened version of home screen user interface 5096). Quick action menu 5120 includes selectable options corresponding to different actions associated with the messaging application that can be performed directly from the home screen user interface (when a selectable option is activated by a tap input (e.g., by a contact different from contact 5110), the corresponding action will be performed respectively), without first launching the application and then selecting the action from within the user interface of the application after launching the application. Figures 5A40 to 5A41 , a tap input by contact 5122 is detected on an option for composing a new message in quick action menu 5120 (e.g., as Figure 5A40 ), and in response to detecting the input, the device launches the messaging application and directly displays a new message composition user interface 5132 (e.g., including the message content area 5123 and the virtual keyboard 5020), without requiring the user to navigate from the default launch user interface to the new message composition user interface 5132 through conventional internal navigation within the messaging application.

[0270] Figure 5A42 An exemplary scenario is shown in which if the input made by contact 5110 is not on the message application icon 424, but on Figures 5A36 to 5A38 , then when lift-off of contact 5110 is detected with a movement less than a threshold amount after the input threshold has been met, the device displays a quick action menu 5130 and mini-app object 5128 corresponding to the photos application, and generates a non-visual output (e.g., a tactile output (e.g., tactile output 5124)) to indicate that the end of the input has been detected, and displays a quick action menu and mini-app object for the application corresponding to application icon 426 in response to the end of the input. In some embodiments, the quick action menu 5130 for the photos application includes selectable options corresponding to displaying folders for recently taken photos, displaying favorites, displaying photos taken a year ago, and displaying a search interface for searching the media library of the photos application. In some embodiments, the mini-app object corresponding to the photos application includes a list of folders corresponding to different date ranges, themes, and / or media types. Figure 5A39 and Figure 5A42 The quick action menus and mini-app objects shown in the figure are illustrative only. Different applications have different quick action menu items in their quick action menus and have different mini-app objects corresponding to different subsets (e.g., less than all) of the functionality and / or content from the full application. In some embodiments, a tap input to a mini-app object causes the device to dismiss the current user interface and launch the application corresponding to the mini-app object.

[0271] In some embodiments, a tap input (e.g., a tap input detected on the background outside the area occupied by the system user interface (e.g., outside of application icons, quick action menus (e.g., quick action menus 5120 or 5130), and mini-app objects (e.g., mini-app object 5128)) is detected. Figure 5A39 Background 5118 or Figure 5A42In some embodiments, when the home screen user interface is restored, the device does not generate non-visual output (e.g., tactile output) to accompany the visual change in the user interface. In some embodiments, when the home screen user interface is restored, after the quick action menu and mini-app objects are collapsed into the app icon, the device generates non-visual output (e.g., tactile output) in conjunction with stabilizing the app icon (e.g., 424 or 426) to its original position in the home screen user interface.

[0272] Figures 5A43 to 5A44 from Figure 5A37 or Figure 5A38 Continuing, an exemplary scenario is shown in which movement of contact 5110 exceeding a threshold amount (e.g., movement toward the right side of the display) is detected after the input threshold has been met. In some embodiments, as Figure 5A43 As shown, in response to movement of contact 5110, the application icon is dragged from application icon 424 at its original location in home screen user interface 5096 in accordance with the movement of contact 5110. The device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5134)) having a third tactile output representation that is different from the tactile output generated when the input threshold is met (e.g., Figure 5A37 5114) and / or a tactile output generated when the input ends without movement after the input threshold has been met (e.g., Figure 5A39 Tactile output in 5117 and Figure 5A42 The second tactile output performance of the tactile output 5124). Figure 5A44 In addition to generating tactile output 5134, the device also provides visual feedback (e.g., animating all application icons in the home screen user interface 5096) to indicate that the user interface reconfiguration mode is activated and that the user can drag the target application icon to reposition it in the home screen user interface.

[0273] exist Figure 5A45, the termination of input through contact 5110 is detected (e.g., lift-off of contact 5110), and in response to detecting the end of input through contact 5110, application icon 5424 is displayed at the most recently inserted location in home screen user interface 5096. In some embodiments, the device generates non-visual output (e.g., tactile output (e.g., tactile output 5136)) in conjunction with inserting or stabilizing the application icon to the inserted location (e.g., the original location of application icon 424 because the movement of contact 5110 is smaller). Figure 5A45 As shown, when movement of contact 5110 is detected, display of information object 5116 ceases. In some embodiments, the tactile output (e.g., tactile output 5136) generated in conjunction with inserting and / or stabilizing the dragged object into an insertion position in the user interface (and detecting lift-off of the contact after movement of the contact, where the movement is after the input threshold has been met) presents a different tactile output than the tactile output provided in conjunction with satisfying the input threshold (e.g., Figure 5A37 The tactile output 5114 in FIG. 5 is different from the tactile output provided in conjunction with displaying the quick action menu (e.g., Figure 5A39 5117) in the tactile output (and detecting lift-off of the contact without movement of the contact after the input threshold has been met) and / or is different from the tactile output provided in conjunction with moving the application icon out of its original position (e.g., Figure 5A44 The tactile output of the touch sensor 5134 is presented (and movement of the contact is detected after the input threshold has been met).

[0274] Figures 5A46 to 5A49 An exemplary process for triggering a user interface reconfiguration mode on a home screen user interface is shown. In some embodiments, the process may be performed in addition to or in lieu of Figures 5A36 to 5A37 and later Figures 5A43 to 5A44 The exemplary process shown is implemented. Figures 5A46 to 5A47 In some embodiments, contact 5110 maintains movement less than a threshold amount for at least a threshold amount of time T(I) since the contact was detected. Based on determining that input through contact 5110 has satisfied the input threshold and has undergone movement less than a threshold amount since the contact was detected (e.g., remained substantially stationary), the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5138)) to indicate that input through contact 5110 has satisfied the input threshold by moving less than a threshold amount. In some embodiments, such as Figure 5A47 As shown, when the input threshold is met, the device does not display information object 5116. Figures 5A48 to 5A49 , if contact 5110 is maintained with less than a threshold amount of movement (e.g., remains substantially stationary) for another threshold amount of time (e.g., Tth ), the device activates the user interface reconfiguration mode without moving contact 5110. Figure 5A48 As shown, after the input made through contact 5110 meets the extended duration threshold (e.g., T(I)+T th ) before, the user interface remains the same as Figure 5A47 The same state as shown. Figure 5A49 As shown, in response to detecting that the input satisfies the extended duration threshold, the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5140)) and animates the application icon to indicate that the extended duration threshold for entering the user interface reconfiguration mode has been met and the device has entered the user interface reconfiguration mode. In some embodiments, Figure 5A49 Tactile output in 5140 and Figure 5A44 In some embodiments, the tactile output 5134 has the same tactile output performance corresponding to the activation of the user interface reconfiguration mode. Figure 5A49 Tactile output in 5140 and Figure 5A44 The tactile output 5134 in has different tactile output appearances to indicate different ways (e.g., different input types) that the user interface reconfiguration mode is activated.

[0275] Figures 5A50 to 5A52 from Figure 5A44 or Figure 5A49 , and illustrates an exemplary scenario in which, after initiating reconfiguration mode, a larger movement of contact 5110 causes application icon 424 to be dragged and dropped into a different location in home screen user interface 5096. Figures 5A51 to 5A52 In , when application icon 524 is dragged to a different location in home screen user interface 5096, other application icons automatically shift and move into the location vacated by application icon 524. Figure 5A52 In some embodiments, when the termination of the input is detected (e.g., the lift-off of contact 5110 is detected), application icon 424 is inserted into the most recent insertion location in home screen user interface 5096, and home screen user interface 5096 is reconfigured. In some embodiments, in conjunction with the stabilization of application icon 424 (and the detection of the termination of the input made through contact 5110), the device generates a non-visual output (e.g., a tactile output (e.g., tactile output 5142)). In some embodiments, Figure 5A52 The tactile output 5142 has the same Figure 5A45 In some embodiments, the same tactile output as in tactile output 5136 is presented to indicate stabilization of the application icon into its inserted position in the home screen user interface. Figure 5A52 The tactile output 5142 has the same Figure 5A45 Tactile output 5136 in different tactile output performance to indicate the difference between a canceled reconfiguration operation and a successfully completed reconfiguration operation.

[0276] Figures 5A53 to 5A54 51. It is shown that after the end of input by contact 5110 is detected and application icon 424 is inserted into the new location, the user interface reconfiguration mode remains active until another input is detected. In some embodiments, the user interface reconfiguration mode is activated by another contact (e.g., Figure 5A53 An upward edge swipe gesture made by contact 5144 in the middle causes the device to deactivate the user interface reconfiguration mode and display the home screen user interface 5096 in the non-animated new configuration state (e.g., as shown in FIG. Figure 5A54 shown).

[0277] Figures 5A55 to 5A67 An exemplary interaction with a user interface object in a web browser application is shown. Figure 5A55 , a user interface of a web browser application (eg, user interface 5146) displays a web page including one or more images, text, and hyperlinks (eg, hyperlink 5148).

[0278] exist Figure 5A56 , a contact (e.g., contact 5150) is detected at a location on touch screen 112 corresponding to hyperlink 5148 at time T0. In some embodiments, in response to detecting contact 5150, visual feedback is provided to indicate the target of the input on user interface 5146 (e.g., hyperlink 5148 is highlighted, enlarged, and / or animated), while other parts of the user interface do not change. In some embodiments, the device forgoes providing any non-visual output (e.g., tactile output or audio alert) at this time (e.g., for detecting contact or selection of the target of the input) until the input made through contact 5150 satisfies an input threshold (e.g., a duration threshold (e.g., a static duration threshold or a predefined intensity-based duration threshold T(I))).

[0279] Figures 5A56 to 5A57 An exemplary scenario is shown in which input by contact 5150 includes less than a threshold amount of movement before contact 5150 is lifted off and does not satisfy a predefined input threshold (e.g., as described herein with reference to Figure 5A6 to Figure 5A7 The input threshold is described above). Figure 5A57In response to detecting the termination of an input (e.g., the lift-off of contact 5150) and based on determining that the input is substantially stationary and the input ends without meeting a predefined input threshold (e.g., lift-off occurs at time T < T0 + T(I)), the device displays another user interface 5151 within the web browser application, and this user interface displays a web page corresponding to the hyperlink 5148.

[0280] Figure 5A58 After Figure 5A56 and shows an exemplary scenario where the input made by contact 5150 meets a predefined input threshold and has less than a threshold amount of movement before meeting the predefined input threshold (e.g., remains substantially stationary). After Figure 5A58 In response to detecting that a movement of less than a threshold amount made by contact 5150 meets the input threshold (e.g., contact 5150 remains substantially stationary for at least a threshold amount of time T(I)), the device generates a non-visual output (e.g., a haptic output (e.g., haptic output 5152)) to indicate that the input threshold has been met, and the subsequent portion of the input will be used to determine which of a plurality of possible responses will be provided. In some embodiments, the haptic output 5152 has a first haptic output manifestation, including a haptic output having a first haptic output pattern (e.g., a first amplitude, frequency, and waveform pattern), and optionally accompanied by a haptic audio output having a first haptic audio output pattern (e.g., a first amplitude, frequency, waveform pattern, and offset from the haptic output). In some embodiments, before the input threshold is met, the appearance of the user interface does not respond to a continuous change in the intensity of contact 5150; and for a device capable of detecting an intensity relative to a first intensity threshold that is higher than a nominal contact intensity detection threshold, the input threshold is adjusted from a first value to a second value (e.g., a value less than the first value) based on determining that the intensity of the contact has exceeded the first intensity threshold. In Figure 5A58 In response to detecting that a movement of less than a threshold amount made by contact 5150 meets the input threshold, the device changes the appearance of the hyperlink 5148 (e.g., the hyperlink 5148 is highlighted and / or detached from the web page in the user interface 5146) in combination with generating a non-visual output (e.g., a haptic output) to indicate that the input threshold has been met.

[0281] Figure 5A60 After Figure 5A59 and shows an exemplary scenario where the lift-off of contact 5150 is detected after the input threshold has been met. As Figure 5A60 shown, in response to detecting the lift-off of contact 5150 after the input threshold has been met (e.g., as Figure 5A595148 ), the device displays a system user interface or a system user interface element including a web page preview 5156 corresponding to a hyperlink 5148, and optionally, one or more control user interface objects for actions that can be performed with respect to the web page (e.g., a control 5160 for opening the web page in a new tab, a control 5162 for adding the web page to a reading list, and a control 5164 for copying the web page or hyperlink). In conjunction with displaying the system user interface (e.g., a user interface that occupies the entire touch screen or a system user interface element that occupies less than the entire touch screen and is overlaid on a background user interface 5158 (e.g., a blurred and dimmed version of the previous user interface 5146)), the device generates non-visual output (e.g., tactile output (e.g., tactile output 5154)) to indicate that the end of the input has been detected, and displays the web page preview 5156 corresponding to the hyperlink 5148 in response to the end of the input. In some embodiments, the tactile output 5154 provided at the end of the input has a different tactile output manifestation than the tactile output 5152 provided when the input threshold is met (e.g., a different tactile output pattern for the tactile output, a different tactile output pattern for the tactile output and a different tactile audio output pattern for the accompanying tactile audio output, or the same tactile output pattern for the tactile output but a different tactile audio output pattern for the accompanying tactile audio output). The different tactile output manifestations for the tactile output for the end of the input and the tactile output for meeting the input threshold alert the user that different changes have occurred in the internal state of the device. In some embodiments, the tactile output manifestations for the end of the input and for the tactile output for meeting the input threshold are similar or identical, in contrast to the tactile output provided when the input includes moving a user interface object (e.g., hyperlink 5148) under the contact (e.g., contact 5150) away from its position.

[0282] In this example, preview 5156 is displayed over background 5154 (e.g., a blurred and darkened version of previous user interface 5146) with one or more controls for triggering actions that are to be performed relative to the web page corresponding to hyperlink 5148 (e.g., the corresponding actions will be performed when the controls are activated by a tap input (e.g., by a contact different from contact 5150), respectively), without navigating to the web page from the currently displayed web page within the browser application (e.g., as shown in FIG. Figures 5A58 to 5A59In some embodiments, a tap input detected at a location outside the area occupied by preview 5156 and controls 5160, 5162, and 5164 causes the device to dismiss the system user interface or a system user interface element and return to the previously displayed user interface 5146. In some embodiments, a tap input detected on preview 5156 causes the device to dismiss the system user interface or a system user interface element and navigate to user interface 5151 that displays a new web page corresponding to hyperlink 5148 (e.g., as shown in FIG. 5B ). Figure 5A57 shown).

[0283] Figures 5A61 to 5A62 from Figure 5A58 Continuing, an exemplary scenario is shown in which movement of contact 5150 exceeding a threshold amount (e.g., movement toward the right side of the display) is detected after the input threshold has been met. In some embodiments, as Figure 5A62 As shown, in response to movement of contact 5150, copy 5166 of hyperlink 5148 is dragged from its original location in web browser user interface 5146 in accordance with the movement of contact 5150. The device generates a non-visual output (e.g., tactile output (e.g., tactile output 5168)) having a third tactile output representation that is different from the tactile output generated when the input threshold is met (e.g., Figure 5A58 5152) and / or a tactile output generated when the input ends without movement after the input threshold has been met (e.g., Figure 5A60 The second tactile output performance of the tactile output 5154).

[0284] Figure 5A63 exist Figure 5A62 Thereafter, an exemplary scenario is shown in which lift-off of contact 5150 is detected when no permitted drop location is available on the user interface for copy 5166 of hyperlink 5148. In response to detecting termination of input through contact 5150, copy 5166 of hyperlink 5148 is dropped back to its original location (and merged with hyperlink 5148), and the device optionally generates non-visual output (e.g., tactile output (e.g., tactile output 5170)) to indicate that copy 5166 of hyperlink 5148 is stabilized at its original location in user interface 5146. In some embodiments, when no permitted drop location is available for the dragged object and the drag-and-drop operation is canceled, the device forgoes generating non-visual output (e.g., tactile output).

[0285] Figures 5A64 to 5A67 exist Figure 5A62 Thereafter, and an exemplary scenario is shown, wherein a copy 5166 of the hyperlink 5148 is placed into an allowed drop location after the end of the input is detected. Figure 5A64 As shown, while contact 5150 remains on the touch screen holding copy 5166 of hyperlink 5148, an upward edge swipe input by contact 5167 is detected near the bottom edge of touch screen 112. Figure 5A65 , while contact 5150 remains on the touch screen holding copy 5166 of hyperlink 5148, in response to an upward edge swipe input by contact 5167, the device navigates from web browser user interface 5146 to home screen user interface 5096. While home screen user interface 5096 is displayed and while contact 5150 remains on the touch screen and, optionally, copy 5166 of hyperlink 5148 is dragged from one location to another on the display, a tap input by contact 5172 is detected on application icon 424 for the messaging application. Figure 5A66 , in response to a tap input through contact 5172, the device launches a messaging application and displays a conversation user interface 5014. Conversation user interface 5014 includes a conversation presentation area 5016 that displays messages exchanged in the conversation (e.g., message 5018). Conversation user interface 5014 also includes a message input box and a virtual keyboard 5020. When contact 5150 moves to the message input box, a copy 5166 of hyperlink 5148 is dragged to a position above the message input box. Figure 5A67 , lift-off of contact 5150 is detected when contact 5150 and copy 5166 of hyperlink 5148 is placed in the message input box (e.g., converted to hyperlink text 5176). In some embodiments, the device generates non-visual output (e.g., tactile output (e.g., tactile output 5178)) to indicate that a hyperlink has been placed in the message input box.

[0286] Figures 5B1 to 5B6 Figures 1 and 2 illustrate exemplary user interfaces for interacting with user interface objects according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including Figures 8A to 8C According to some embodiments, 6A to 6D 、 7A to 7B 、 Figures 9A to 9G and Figures 10A to 10E The process shown and Figures 5E1 to 5E4 The various aspects of the interaction model shown are also Figures 5B1 to 5B6 For ease of explanation, some of these embodiments will be discussed with reference to operations performed on a device (e.g., see Figures 5B1 to 5B3 ), the electronic device having a touch-sensitive display 112, an input element (e.g., a touch-sensitive surface on the display 112), and a plurality of audio output elements 111 (e.g., audio output elements or speakers 111a, 111b, and 111c, Figures 5B1 to 5B3 ).exist Figures 5B1 to 5B3 In the example shown, device 100 (e.g., a mobile phone, sometimes referred to as a cellular phone or portable electronic device) has three audio output elements 111, including one audio output element (111c) near the top edge of the device (e.g., suitable for placement near a user's ear when making a phone call), one audio output element (111a) at a position to the left of the bottom edge of the device (e.g., near the lower left corner), and one audio output element (111b) at the right of the bottom edge of the device (e.g., near the lower right corner). In other embodiments, the number of audio output elements may be fewer (e.g., two) or more (e.g., four or more than four), and / or the placement of the audio output elements may be different (e.g., in a position corresponding to the lower right corner). Figures 5B1 to 5B3 The positions shown are at different locations along the outer edge of the device).

[0287] Figures 5B1 to 5B3 Interaction with a home screen user interface and application launch icons displayed on the home screen user interface is shown. Figure 5B1 , a plurality of application launch icons are displayed on a home page of a home screen user interface (e.g., home screen user interface 5182), including application launch icon 424 for a message application. At time T0, a first input, such as (or including) a contact (e.g., contact 5310, Figure 5B2 ). In some embodiments, in response to detecting contact 5310, visual feedback is provided to indicate the target of the first input on the home screen user interface 5182 corresponding to the location of contact 5310 on the touch screen 112 (e.g., the application launch icon 424 is highlighted, enlarged, and / or animated while other application launch icons remain unchanged). In some embodiments, the device forgoes providing any tactile output or audio alert (e.g., for detecting contact or selection of the target of the first input) at this time until the first input made through contact 5310 satisfies an input threshold (e.g., a duration threshold (e.g., a static duration threshold or a predefined intensity-based duration threshold T(I))).

[0288] Figure 5B2 and Figure 5B3An exemplary scenario is shown in which a first input, such as contact 5310, satisfies predefined criteria. In some embodiments, the predefined criteria include a time threshold criterion (e.g., a criterion that the contact of the first input (e.g., contact 5310) remains on the touch-sensitive display for at least a threshold amount of time T(I)) and a movement criterion (e.g., a criterion that the contact of the input has less than a threshold amount of movement (e.g., remains substantially stationary)). In some embodiments, the predefined criteria also include an input termination criterion (e.g., the first input ends (e.g., by lifting off contact 5310)) where the first input movement is less than a threshold amount of movement (e.g., remains substantially stationary relative to a lateral direction parallel to the touch-sensitive surface of the touch-sensitive display). In some embodiments, the appearance of the user interface does not respond to continuous changes in the intensity of contact 5110 until the predefined criteria are met, and for devices capable of detecting an intensity relative to a first intensity threshold that is higher than a nominal contact intensity detection threshold, the time threshold (e.g., the time criterion) included in the predefined criteria is optionally adjusted from a first value to a second value (e.g., a value less than the first value) based on determining that the intensity of the contact has exceeded the first intensity threshold.

[0289] Figure 5B3 from Figure 5B2 Continue. Figure 5B3 As shown, in response to determining (e.g., determined by electronic device 100) that the first input meets predefined criteria, including the device detecting a lift-off of contact 5310 after a time threshold criterion has been met (e.g., as shown in FIG. Figure 5B3 ), the device displays a system user interface or a system user interface element including a quick action menu 5320 (e.g., one or more selectable options that, when activated, cause an operation associated with an application to be performed (e.g., compose new message, compose message to "Mom," compose message to "S. Ramanujan," compose message to "G. Hardy")), and, optionally, one or more mini-application objects (e.g., widgets or mini-applications, plug-ins, modules) associated with a target application launch icon (e.g., messaging application icon 424). In conjunction with displaying the system user interface (e.g., a user interface that occupies the entire touch screen or a system user interface element that occupies less than the entire touch screen and overlays background user interface 5182 (e.g., a blurred and dimmed version of the previous user interface 5182)), the device generates an audio output (e.g., audio output 5330) to indicate that an end of the input has been detected, and displays a quick action menu for the application corresponding to application icon 424 in response to the end of the input.

[0290] In some embodiments, the audio output 5330 provided at the end of the first input is provided to one or more of the audio output elements 111, as described below. Based on the device 100 determining that the user interface object 424 to which the input (e.g., contact 5310) is directed is positioned closer to a first audio output element (e.g., speaker 111c) of the plurality of audio output elements than to a second audio output element (e.g., speaker 111a or 111b) of the plurality of audio output elements, the device generates a first audio output (e.g., audio output 5330) corresponding to the first input, wherein the first audio output has a greater amplitude at the first audio output element than at the second audio output element.

[0291] On the other hand, if the first input is directed to another user interface object (e.g., user interface object 416), for example, in the lower left portion of user interface 5182, and the first input meets predefined criteria, a second audio output is generated. Based on device 100 determining that the user interface object (e.g., user interface object 416) to which the first input (e.g., contact 5310) is directed is positioned closer to a second audio output element (e.g., speaker 111a) among the multiple audio output elements than to a first audio output element (e.g., speaker 111c) among the multiple audio output elements, the device generates a second audio output (not shown) corresponding to the input, wherein the second audio output has a greater amplitude at the second audio output element than at the first audio output element.

[0292] In some embodiments, the first audio output and the second audio output share one or more audio output properties (e.g., volume, frequency, waveform, etc.). In some embodiments, the first audio output and the second audio output are identical.

[0293] In some embodiments, the first audio output is output on the first audio output element and not on the second audio output element. Similarly, in some embodiments, the second audio output is output on the second audio output element and not on the first audio output element. On the other hand, in some embodiments, generating a first audio output corresponding to the first input includes outputting the first audio output at a first amplitude on the first audio output element and outputting the first audio output at a second amplitude on the second audio output element, and the amount of the difference between the first amplitude and the second amplitude is based on the position of the user interface object. For example, if the distance between the user interface object and the position associated with the first audio output element is one-fourth the distance between the user interface object and the position associated with the second audio output element, the amount of the difference between the first amplitude and the second amplitude will be based on a four-to-one ratio of these distances, or based on the size of the difference measured relative to a predefined unit of distance (e.g., pixels or millimeters).

[0294] In some embodiments, device 100 identifies a subset of the plurality of audio output elements based on the location of the user interface object to which the first input (e.g., contact 5310) is directed, including at least two but less than all of the plurality of audio output elements, and generating the first audio output includes outputting the first audio output at the identified subset of the plurality of audio output elements. Figure 5B3 , first audio output 5330 is output at two audio output elements ( 111a and 111c ) of the three audio output elements identified by the device based on the position of user interface object 424 .

[0295] Figures 5B4 to 5B6 Similar to Figures 5B1 to 5B3 , except that the device 100 is a tablet computer, which in this example has four audio output elements 111d, 111e, 111f and 111g, which in this example are positioned near the four corners of the device 100 (along the periphery of the device). Figures 5B4 to 5B6 Interaction with a home screen user interface and application launch icons (including application launch icon 5424 for a messaging application) displayed on the home screen user interface 5400 is shown. At time T0, a first input, such as (or including) a contact (e.g., contact 5450, Figure 5B5 ). In some embodiments, in response to detecting contact 5450, visual feedback is provided to indicate the target of the first input on the home screen user interface 5400 corresponding to the location of contact 5450 on the touch screen 112 (e.g., the application launch icon 5424 is highlighted, enlarged, and / or animated while other application launch icons remain unchanged). In some embodiments, the device forgoes providing any tactile output or audio alert (e.g., for detecting contact or selection of the target of the first input) at this time until the first input made through contact 5450 satisfies an input threshold (e.g., a duration threshold (e.g., a static duration threshold or a predefined intensity-based duration threshold T(I))).

[0296] Figure 5B6 An exemplary scenario is shown where a first input (e.g., contact 5450) satisfies predefined criteria, as described herein with reference to Figure 5B3 As described. Figure 5B6As shown, in response to determining (e.g., determined by electronic device 100) that input made through contact 5450 meets predefined criteria, including the device detecting lift-off of contact 5450 after a time threshold criterion is met (e.g., contact 5450 remains on touch screen 112 for at least a threshold amount of time T(I)), the device displays a system user interface including a quick action menu 5460 (e.g., one or more selectable options that, when activated, enable performance of an operation associated with an application (e.g., compose a new message, compose a message to "Mom," compose a message to "S. Ramanujan," compose a message to "G. Hardy")), and, optionally, one or more mini-application objects (e.g., widgets or mini-applications, plug-ins, modules) associated with a target application launch icon (e.g., messaging application icon 424). In conjunction with displaying the system user interface, the device generates an audio output (e.g., audio output 5370) to indicate that the end of the input has been detected, and displays a quick action menu for the application corresponding to application icon 5424 in response to the end of the input.

[0297] In some embodiments, audio output 5370 is provided to one or more of audio output elements 111 based on the location of user interface object 5424 to which input (e.g., contact 5450) is directed. Figure 5B6 As shown, user interface object 5424 is positioned closer to speaker 111f than to speaker 111d, and is positioned closer to speaker 111d than to speakers 111e and 111g. Figure 5B6 In the example shown, the device generates an audio output (e.g., audio output 5370) that has a greater amplitude at speaker 111f than at speaker 111d, and that is not generated at speakers 111e and 111g. On the other hand, if the input is directed to another user interface object, e.g., closer to speaker 111d than any speaker 111e and closer to speaker 111e than to speakers 111f or 111g, and the input meets predefined criteria, a different audio output will be generated (e.g., audio output having a given amplitude at speaker 111d, a smaller amplitude at speaker 111e, and even smaller amplitudes (if any) at speakers 111f and 111g, respectively).

[0298] In some embodiments, reference is made herein to Figures 5B1 to 5B6 The audio output is or has the reference herein Figures 5A1 to 5A67 、 Figures 5C1 to 5C19 、 Figures 5D1 to 5D30 and Figures 5E1 to 5E4 as well as 6A to 6D 、 7A to 7B 、 Figures 8A to 8C 、 Figures 9A to 9G and Figures 10A to 10E The properties of the non-visual output.

[0299] Figures 5C1 to 5C19 An exemplary user interface for interacting with user interface objects is shown according to some embodiments.

[0300] exist Figures 5C1 to 5C2 , a tap input by contact 5402 is detected on the touch screen at a location corresponding to the application launch icon 424 of the messaging application shown on the home screen user interface 5096. In response to the tap input by contact 5402, the device launches the messaging application and displays the messaging application's conversation user interface 5014. In some embodiments, to detect the tap input, the device detects lift-off of contact 5402 within a first threshold amount of time (e.g., the amount of time required to detect a touch hold input) of the touch press of contact 5402, without detecting substantial movement of the contact 5402 since the touch press (e.g., the contact is substantially stationary).

[0301] Figure 5C3 to Figure 5C5 exist Figure 5C1 After that. In Figure 5C3 to Figure 5C5 , a swipe input by contact 5402 is detected after contact 5402 is touched down at a location corresponding to the application launch icon 424 of the messaging application. In some embodiments, the device detects a swipe input by contact 5402 when the device detects substantial movement (e.g., more than a nominal amount of movement within a nominal threshold amount of time) after the touch press of the contact but before the contact has remained substantially stationary for a first threshold amount of time T(I). Figure 5C4 , the home screen user interface 5096 (including the application launch icon 424 and other concurrently displayed application launch icons) moves in accordance with the movement of the contact 5402 to the left, and a previously undisplayed page of the home screen is dragged onto the display. Figure 5C5 , based on determining that the movement of contact 5402 has exceeded a predefined threshold amount of movement (e.g., half the width of the display) to complete the page navigation operation, after detecting the end of the swipe input through contact 5402, a second page of the home screen user interface (e.g., home screen user interface 5112) is displayed, thereby replacing the previously displayed home screen user interface 5095.

[0302] exist Figures 5C1 to 5C5In the exemplary scenario shown, input via contact 5402 does not satisfy the requirement that the contact remain substantially stationary (e.g., have less than a nominal amount of movement during a nominal amount of time) for at least a first threshold amount of time T(I) after being touched down on the user interface object. Upon detecting a lift-off of the contact without substantial movement before T(I), an operation corresponding to activation of the user interface object is performed (e.g., an application launch operation is performed, such as Figures 5C1 to 5C2 In the event that substantial movement of the contact is detected before T(I), an operation corresponding to the entire user interface (e.g., as opposed to an operation specific to a user interface object) is performed. For example, a page navigation operation is performed on the entire user interface without specifically selecting a user interface object as the subject of the operation, as shown in FIG5C3 to FIG5C4. Figure 5C5 As shown. The relative timing between the movement and / or end of the input and various input thresholds (e.g., time thresholds T(I), T(II), and T(III)) is indicated on the input timeline shown on the right side of the user interface. A solid triangle on the timeline indicates that contact is detected or maintained on the touch screen, and a hollow triangle on the timeline indicates the lift-off of the contact. The arrows close to the solid triangles indicate the movement of the contact on the touch screen. The position of the triangles on the timeline indicates the timing of the touch press, hold, movement, and / or lift-off of the contact relative to the various input thresholds (e.g., time thresholds T(I), T(II), and T(III), etc.). In some embodiments, the input intensity requirements are optionally used in conjunction with the time thresholds in the standard to trigger various user interface responses.

[0303] Figure 5C1 Figures 5C6 to 6 Figure 5C8 An exemplary scenario is shown in which, following a touch press of contact 5402, contact 5402 has remained substantially stationary for at least a first threshold amount of time T(I) without substantial movement (e.g., the contact remains substantially stationary). In Figure 5C6, in response to detecting that contact 5402 has remained substantially stationary for a first threshold amount of time T(I), the device generates a tactile output 5404 to indicate that the first time threshold has been met and that contact 5402 has remained substantially stationary since the touch press of the contact. In some embodiments, the appearance of application launch icon 424 is optionally changed to indicate that selection of application launch icon 424 is the subject of a subsequent operation performed in response to input made through contact 5402. Figure 5C7 , no additional visual or tactile output (e.g., visual and tactile feedback) is provided while contact 5402 remains in its touch-down position on application launch icon 424 until movement or lift-off of contact 5402 is detected after first input threshold T(I) has been satisfied. Figure 5C8, a lift-off of contact 5402 is detected after the first input threshold T(I) is satisfied but before the second input threshold T(II) is satisfied. In response to detecting the lift-off of contact 5402 (without detecting any substantial movement of contact 5402), the device displays quick action menu 5120, overlaying background user interface 5118 (e.g., a blurred and dimmed version of previous user interface 5096). In some embodiments, in conjunction with displaying quick action menu 5120 associated with the messaging application, the device also generates tactile output 5406.

[0304] Figure 5C9 Continue from 5C7. Figure 5C6, Figure 5C7 and Figure 5C9 The sequence shows an exemplary scenario in which, after a touch press of contact 5402, the contact remains substantially stationary for at least a second threshold amount of time T(II) greater than T(I). In response to detecting that the contact has remained substantially stationary for a second threshold amount of time T(II) since the touch press of the contact, the device automatically displays the quick action menu 5120 overlaying the background user interface 5118 without any additional input from the user (e.g., without lifting the contact and without increasing the strength of the contact to above a light press strength threshold that is higher than the nominal contact detection strength threshold). In some embodiments, the quick action menu 5120 is displayed more quickly while the contact 5402 remains on the touch screen than it was before. Figure 5C8 The final stable state shown in FIG is slightly exaggerated. In some embodiments, the quick action menu 5120 has Figure 5C8 . In some embodiments, when the second time threshold T(II) is met, the device generates a tactile output in conjunction with automatically displaying the quick action menu. In some embodiments, when the quick action menu is displayed in response to meeting the second time threshold T(II) without lifting off, the device abandons generating a tactile output. In some embodiments, the display of the quick action menu is maintained as long as the contact remains at the touch-down position of the contact without substantial movement (e.g., as long as the contact remains substantially stationary). In some embodiments, the display of the quick action menu is maintained until a third time threshold T(III) as long as the contact remains substantially stationary at the touch-down position of the contact; then, the device automatically performs another operation (e.g., stops displaying the quick action menu and starts a reconfiguration mode of the user interface) without requiring additional input from the user.

[0305] exist Figure 5C10, after contact 5402 has remained substantially stationary for at least a second threshold amount of time T(II) but less than a third threshold amount of time T(III) and while a slightly enlarged quick action menu 5120 is displayed overlaying background user interface 5116, a lift-off of the contact is detected. In response to detecting the lift-off of contact 5402, the device displays Figure 5C8 , the device generates tactile output 5408 in conjunction with the display of the quick action menu 5120 in its final stable state overlaying the background user interface 5118.

[0306] Figure 5C11 to Figure 5C12 from Figure 5C9 Continuing, an exemplary scenario is shown in which, after a second time threshold T(II) (e.g., as shown in FIG. 1 ) has been satisfied (e.g., satisfied by a substantially stationary contact), Figure 5C9 After contact 5402 moves to one of the selectable options in quick action menu 5120, a lift-off of contact 5420 is detected while contact 5420 is over the selectable option for creating a new message. In response to detecting the lift-off of contact 5420, the device stops displaying quick action menu 5120 and background user interface 5118, and launches the messaging application. Figure 5C12 , the device displays a new message composition user interface 5132 (e.g., including message content area 5123 and virtual keyboard 5020) without requiring the user to navigate from the default launch user interface to the new message composition user interface 5132 through conventional internal navigation in the message application. In some embodiments, if a lift-off of contact 5402 is detected without substantial movement of contact 5402, quick action menu 5120 is displayed in a stable state, such as Figure 5C10 In some embodiments, a subsequent input (e.g., a tap input) made by another contact on a selectable option in the steady-state quick action menu causes the device to perform the operation corresponding to the selectable option (e.g., launching the messaging application to show a new message composition user interface 5132, as shown in FIG. Figure 5C12 shown).

[0307] Figure 5C9 Figure 5C13 to Figure 5C14 An exemplary scenario is shown, in which after the second time threshold T(II) has been met (e.g., Figure 5C95C13 , contact 5402 moves to an area outside the area occupied by quick action menu 5120 (e.g., moves horizontally to the right). In response to detecting that contact 5402 has substantially moved to an area outside the quick action menu 5120, the device stops displaying the quick action menu 5120 and starts the reconfiguration mode of the user interface 5096. In some embodiments, the application icons shown in the user interface 5096 are displayed in an animated manner to indicate that the reconfiguration mode of the user interface has been started. In some embodiments, the application icon 5402 is dragged from its original position on the user interface 5096 to a new position according to the movement of contact 5402. In some embodiments, the device generates a tactile output 5500 to indicate that the user interface reconfiguration mode has been started. This feature allows the user to start the user interface reconfiguration mode and move the application icons at the same time using the same input, rather than starting the reconfiguration mode with one input and then selecting and moving the application icons. This improves interaction efficiency if the user knows which icon needs to be repositioned in the user interface and starts inputting directly to that icon without waiting for the third threshold amount of time T(III) to expire.

[0308] Figure 5C9 After Figure 5C15 An exemplary scenario is shown in which contact 5402 has remained substantially stationary for a third threshold amount of time T(III) since the touch depression of contact 5402. In response to detecting that contact 5402 has remained substantially stationary on application icon 424 for at least the third threshold amount of time T(III), the device automatically stops displaying quick action menu 5120 and initiates a user interface reconfiguration mode of the home screen user interface, as shown. Figure 5C15 As shown. In this case, the application icons are maintained in their original positions after the user interface reconfiguration mode is initiated, and subsequent moves can be initiated on application icon 424 or any of the other application icons shown on the user interface. This feature allows the user to initiate the user interface reconfiguration mode by interacting with any of these application icons, and then decide which application icon to move after initiating the reconfiguration mode. This allows the user to enter the reconfiguration mode without making a decision about which icon to reposition, and avoids accidentally moving icons and changing the user interface configuration. In addition, providing two alternative paths to trigger the user interface reconfiguration mode allows the user to select the path that is most effective for his / her particular needs, and improves the efficiency of user interaction with the user interface and reduces user errors when interacting with the device.

[0309] Figure 5C16 to Figure 5C17 from Figure 5C14 or Figure 5C15 5096 continues, and contact 5402 is shown dragging application icon 424 from its original location in user interface 5096 to another location, with the other application icons in user interface 5096 being repositioned accordingly. Figure 5C18 and Figure 5C19 from Figure 5C17 Continuing, it is shown that during the user interface reconfiguration mode, a tap input by another contact 5506 in the user interface 5596 at a location not occupied by an application icon causes the device to exit the user interface reconfiguration mode. Figure 5C19 , user interface 5096 is shown in a final reconfigured state (e.g., the position of the first row of application icons is the same as Figure 5C1 The positions shown in the figure are different).

[0310] Figures 5C1 to 5C19 The exemplary interaction scenario shown is used to illustrate the Figure 5E1 According to some embodiments, in Figure 5E1 In the interaction model shown, there are three time thresholds (eg, T(I), T(II), and T(III)) for which movement and / or lift-off of contact is evaluated.

[0311] Figures 5D1 to 5D30 A sharing user interface is shown according to some embodiments for interacting with previews corresponding to content (e.g., hyperlinks, file icons for documents with long file names, media files with metadata (e.g., lyrics, album, author, composer and performer information, location and time of capture information), etc.) and displaying a sharing user interface for sharing content using various sharing protocols. Figures 5D1 to 5D30 shows corresponding to some embodiments Figure 5E3 , which illustrates exemplary interaction scenarios for different branches of the interaction model shown in FIG. Figure 5E3 The interaction model shown includes three time thresholds (e.g., T(i), T(I), and T(II)) for which movement and / or liftoff of contacts are evaluated. In some embodiments, user interface responses corresponding to time threshold T(i) are optional for some user interface objects and are not optional for some user interface objects. Figure 5E2, and a corresponding time threshold for each of the two or more types of user interface objects.

[0312] Figures 5D1 to 5D2 An exemplary scenario is shown in which a tap input by contact 5508 is detected at a location corresponding to a hyperlink 5148 in a user interface 5146 of a web browser application. Based on determining that the input by contact 5508 is a tap input that includes less than a nominal amount of movement before liftoff of contact 5508 and does not satisfy a first time threshold T(i) (e.g., a quick look time threshold) that is less than a second time threshold T(i), the device displays another user interface 5151 within the web browser application that displays a web page corresponding to hyperlink 5148.

[0313] Figure 5D1 5D3 shows an exemplary scenario in which substantial movement of contact 5508 is detected after the touch depression of contact 5508 on hyperlink 5148. In response to detecting the movement of contact 5508 before the quick view time threshold T(i) is satisfied, the device performs an operation that applies to the entire user interface (e.g., the device scrolls user interface 5146) without specifically selecting hyperlink 5148 as the subject of the operation.

[0314] Figure 5D1 After Figure 5D4 An exemplary scenario is shown in which contact 5508 has remained substantially stationary for a first threshold amount of time, T(i), since the touch depression of contact 5508. In response to detecting that contact 5508 has remained substantially stationary for the first threshold amount of time, T(i), the device displays a visual indication to indicate that hyperlink 5148 has been selected. In some embodiments, the device generates a tactile output 5510 in conjunction with the selection of hyperlink 5148 to indicate that the first time threshold, T(i), has been met and that a subsequent action will be performed with respect to the selected hyperlink 5148. In some embodiments, the device forgoes generating a tactile output at this point.

[0315] exist Figure 5D5 in Figure 5D4Thereafter, after contact 5508 has remained substantially stationary at the location of hyperlink 5148 for at least a first threshold amount of time T(i), lift-off of contact 5508 is detected. In response to detecting lift-off of contact 5508, the device displays a first version of the preview corresponding to hyperlink 5148 (e.g., preview 5514), overlaying background user interface 5158 (e.g., a blurred and dimmed version of previous user interface 5146). Figure 5D5 As shown, in some embodiments, a first version of the preview (e.g., preview 5514) displays metadata corresponding to the user interface object that is the subject of the input (e.g., a universal resource locator or web address corresponding to hyperlink 5148). The first version of the preview is different from the second version of the preview, which displays some of the content of the user interface object (e.g., an image or excerpt from the web page corresponding to hyperlink 5148). This feature allows a user to easily view metadata associated with the user interface object before the user interface object is activated to perform a corresponding action (e.g., initiate a navigation action or execute a script associated with the user interface object). This improves the security of the device and prevents unintentional actions from being performed without sufficient user supervision. Other types of user interface objects that implement and benefit from this type of preview include user interface objects corresponding to metadata that a user may often want to view before opening the content corresponding to the user interface object (e.g., file icons for files with long file names, images with location and timestamps, media files with lyrics, synopses, author, composer, album, and performer information, file attachments in emails and messages with originator and security authentication information, document download links with download source identification information, etc.). In some embodiments, controls associated with additional operations that can be performed with respect to hyperlink 5148 or the web page corresponding to the hyperlink (e.g., control 5160) are displayed concurrently with the preview of the first version (e.g., preview 5514). In some embodiments, the preview of the first version is capable of scrolling in response to a subsequent swipe input performed on the preview of the first version (e.g., for a user to view additional information that was not initially fully displayed in the preview of the first version).

[0316] Figure 5D6Multiple possible inputs are shown on the first version 5514 of the preview according to some embodiments, including a swipe input through contact 5516, a tap input through contact 5518, and a light press input through contact 5520. In some embodiments, these inputs are detected at the same location on the preview 5514 of the first version, and the device determines which operation to perform in response to the inputs based on an evaluation of the inputs against different criteria (e.g., criteria based on a comparison of the location, movement, duration, and intensity of the contact with predefined movement, duration, and / or intensity thresholds). In some embodiments, according to some embodiments, a downward swipe input through contact 5516 causes the device to stop displaying the preview 5514 of the first version and initiates a drag operation on the hyperlink 5516 or a copy thereof based on the movement of contact 5516. In some embodiments, after initiating the drag operation on the hyperlink, the user optionally provides additional navigation input to navigate to a different user interface and place the hyperlink in an allowed insertion location on the other user interface (e.g., after moving and then lifting contact 5516) (e.g., similar to Figure 5D9 to Figure 5D15 5D8 ). In some embodiments, in response to detecting a light press input (or in some embodiments, a touch hold input) through contact 5520, the device displays a preview 5156 of the second version, which shows a preview of the web page corresponding to the hyperlink 5148. In some embodiments, controls associated with additional operations that can be performed with respect to the hyperlink 5148 or the web page corresponding to the hyperlink (e.g., controls 5160, 5164) are displayed simultaneously with the preview of the second version (e.g., preview 5156 in Figure 5D8 ). In some embodiments, the device generates tactile output in conjunction with the display of the preview 5156 of the second version. In some embodiments, after the input through contact 5520 meets the intensity threshold (or duration threshold) for displaying the preview of the second version, the preview of the second version is displayed in response to the lifting of contact 5520. In some embodiments, the preview of the second version (e.g., preview 5534 in Figure 5D16 ) is displayed without the need to lift contact 5502. For example, in some embodiments, after a preview of the second version is displayed while contact 5520 remains on the touch screen (e.g., preview 5534 in Figure 5D16), an upward swipe input causes a sharing user interface corresponding to hyperlink 5148 to be displayed (e.g., as shown in Figure 5D16). Figures 5D23 to 5D24 In some embodiments, in response to detecting a tap input via contact 5518, the device stops displaying the preview of the first version and displays a web page 5151 corresponding to hyperlink 5148 (e.g., as shown in FIG. Figure 5D2 shown) (e.g., performing a navigation operation corresponding to hyperlink 5148).

[0317] Figure 5D4 After Figure 5D7 An exemplary scenario is shown in which, after a touch press of contact 5516, contact 5516 remains substantially stationary for at least a second threshold amount of time T(I) greater than T(i). In response to detecting that contact 5516 has satisfied the second time threshold T(I), the device generates a tactile output 5536 to indicate that the second time threshold T(I) has been satisfied. In some embodiments, the device displays a visual indication showing that hyperlink 5148 has been selected and that the hyperlink is now the subject of a subsequent operation performed in response to a subsequent input made through contact 5516. In some embodiments, if substantial movement of contact 5508 is detected after the contact has remained substantially stationary for more than the second threshold amount of time T(I), a drag operation is initiated on the hyperlink 5148 or a copy thereof based on the movement of contact 5508, for example, as Figure 5D9 to Figure 5D15 shown.

[0318] Figure 5D8 Figure 5D6 or Figure 5D7 After any of . Figure 5D6 and the sequence of Figures 5D8 show that according to some embodiments, a light press input or touch-and-hold input performed on the first version of preview 5514 through contact 5520 causes the device to switch from displaying the preview 5514 of the first version to displaying the preview 5156 of the second version. Figure 5D7 5D8 shows displaying preview 5156 of the second version in response to lift-off of contact 5516 after second time threshold T(I) has been satisfied by substantially stationary contact 5516. In some embodiments, the device generates tactile output 5522 in conjunction with display of preview 5156 of the second version. In some embodiments, controls associated with additional operations that can be performed with respect to hyperlink 5148 or the web page corresponding to the hyperlink (e.g., controls 5160, 5164) are displayed simultaneously with the preview of the second version (e.g., preview 5156 in FIG. 5D8).

[0319] Figure 5D4 After Figure 5D9 to Figure 5D11 An exemplary scenario is shown in which substantial movement of contact 5508 is detected after a first time threshold T(i) is satisfied and before a second time threshold T(I) is satisfied. In response to detecting movement of contact 5508, a drag operation is performed on hyperlink 5148 or a copy thereof in accordance with the movement of contact 5508. In some embodiments, as Figure 5D9 to Figure 5D11As shown, an acceptable insertion location for hyperlink 5148 on web page 5146 is unavailable, and when the end of input is detected (e.g., lift-off of contact 5508), the drag operation is canceled and hyperlink 5148 is not moved or copied to a different location. In some embodiments, the device generates tactile output 5524 to indicate that hyperlink 5146 is repositioned to its original location in web page 5146.

[0320] Figure 5D9 Figures 5D12 to 5D13 Figure 5D15 An exemplary scenario is shown in which substantial movement of contact 5508 is detected after a first time threshold T(i) has been met and before a second time threshold T(I) is met. In response to detecting movement of contact 5508, a drag operation is performed on hyperlink 5148 or a copy thereof in accordance with the movement of contact 5508. In some embodiments, as shown in FIG. Figure 5D13 As shown, while a copy of hyperlink 5148 is dragged by contact 5508, an upward edge swipe input by contact 5526 is detected at the bottom of user interface 5146, resulting in navigation to home screen user interface 5096. Figures 5D13 to 5D14 , a tap input by contact 5528 on application icon 424 causes the device to launch a messaging application and display user interface 5014 with an input field for a message to a message recipient. When contact 5508 drags hyperlink 5148 over insertion point 5174 in the input field and lifts off over insertion point 5147, content and / or metadata (e.g., URL text 5530) corresponding to hyperlink 5148 is inserted into the input field at insertion point 5147, as shown in FIG. Figures 5D14 to 5D15 In some embodiments, the device generates tactile output 5532 in conjunction with inserting content and / or metadata corresponding to hyperlink 5148 into the input field.

[0321] In some embodiments, Figures 5D5 to 5D6 The interactions shown in are optional, and the time threshold T(i) is merged with the time threshold T(I) (e.g., T(i) is increased to T(I)), which means that the above interactions with respect to the time window ending in T(i) (e.g., in Figure 5D1 to Figure 5D4 and Figure 5D7 to Figure 5D15 ) is valid for the time window ending in T(I), but only a single version of the preview (e.g., content preview or metadata preview) is shown.

[0322] Figure 5D7FIG5D16 shows an exemplary scenario in which contact 5508 has remained substantially stationary for a third threshold amount of time, T(II), which is greater than the first threshold amount of time and the second threshold amount of time (e.g., T(i) and T(I)). In response to detecting that contact 5508 has remained substantially stationary for the third threshold amount of time, T(II), the device displays a slightly expanded second version 5534 of the preview, including Figure 5D2 51. The contents of web page 5151 are shown. When contact 5508 remains at its original touch-down location on the touch screen, a preview of a slightly expanded version of the second version is displayed.

[0323] Figure 5D16 and after Figure 5D17 Detection of lift-off of contact 5508 is shown after the contact has remained substantially stationary for at least a third threshold amount of time, T(II), without substantial movement prior to the lift-off of the contact. In response to detecting lift-off of contact 5508 after the contact has remained substantially stationary for at least a third threshold amount of time, T(II), the device displays a preview 5156 of the steady-state second version overlaying the background user interface 5158. In some embodiments, the device generates tactile output 5528 in conjunction with the display of preview 5156 of the steady-state second version. In some embodiments, controls associated with operations that can be performed with respect to the web page shown in the preview (e.g., controls 5160) are displayed simultaneously with the preview.

[0324] Figure 5D18 continues from Figure 5D16 and shows that after a slightly enlarged preview 5534 of the second version is displayed, movement of contact 5508 is detected. Figure 5D17 Continuing, a subsequent swipe input is detected on the preview 5156 of the steady-state second version via contact 5540. Figures 5D19 to 5D21 Shown is a diagram of a method of initiating a drag operation that drags hyperlink 5148 or a copy 5540 thereof in response to detecting movement of contact 5540 in a first predefined direction (e.g., downward) based on subsequent movement of contact 5508 or contact 5540. Figure 5D19 In response to determining that a drag operation has been initiated, the device displays a transition showing the preview 5156 or 5534 (e.g., with intermediate states 5534' and 5542) transforming into a copy 5166 of the hyperlink 5148 on the restored user interface 5146. Figure 5D21 In some embodiments, when an allowed insertion position for the dragged hyperlink is not available, liftoff of contact 5540 is detected, and after the drag input by contact 5540 ends, the drag operation is canceled and hyperlink 5148 is restored to its original position. Figure 5D15If an allowed insertion location is available (e.g., after additional input to navigate to another user interface) as shown, a copy of the hyperlink is inserted at the allowed insertion location after the drag input through contact 5508 or contact 5540 ends.

[0325] Figure 5D22, which follows Figure 5D16, shows that movement of contact 5508 is detected after a slightly enlarged preview 5534 of the second version is displayed. Alternatively, Figure 5D22 optionally follows Figure 5D17 Continuing, detection of a subsequent swipe input by another contact on preview 5156 of the steady state second version is shown. Figures 5D23 to 5D24 Continuing from Figure 5D22, movement of a contact (e.g., contact 5508 or a subsequently detected contact on preview 5156) is detected after a preview of the second version (e.g., preview 5534 or preview 5156) is displayed. Based on determining that the direction of movement is a second predefined direction (e.g., upward), the device displays an indicator (e.g., indicator 5544) near the bottom edge of the display to indicate that additional upward movement of the contact (e.g., contact 5508 or a subsequently detected contact on preview 5156) may trigger an additional action. Figure 5D23 5156) exceeds a predefined movement threshold, another indicator (e.g., indicator 5546) is displayed to indicate that an operation for displaying a sharing user interface has been initiated. In some embodiments, indicator 5546 is displayed simultaneously with the preview of the second version. In some embodiments, based on upward movement of a contact (e.g., contact 5508 or a contact subsequently detected on preview 5156), the preview of the second version is converted to a reduced-scale version of preview 5554 (e.g., including a reduced set of content items from preview 5156 of the second version).

[0326] Figure 5D24 exist Figure 5D23 Thereafter, and after detecting movement of the contact in the second direction exceeding the threshold amount, a liftoff of the contact (e.g., contact 5508 or a contact subsequently detected on preview 5156) is detected. In response to detecting the end of input by the contact, the device displays sharing user interface 5548 corresponding to hyperlink 5148. Figure 5D24 As shown, the sharing user interface 5548 includes a reduced scale version 5554 of the preview of the second version. According to some embodiments, the reduced scale version 5554 includes the preview from the web page 5151 (e.g., Figure 5D2 In some embodiments, the sharing user interface 5548 includes a blurred and / or dimmed user interface 5146 (e.g., as shown in FIG. 5D8 ). Figure 5D1In some embodiments, the sharing user interface 5548 includes a plurality of selectable options 5550 corresponding to a plurality of recipients for sharing the hyperlink 5146 or a web page corresponding to the hyperlink, and a plurality of selectable options 5552 corresponding to a plurality of transfer protocols for sharing the hyperlink 5146 or a web page corresponding to the hyperlink. For example, the plurality of selectable options 5550 represent a plurality of users selected from one or more applications with various sharing protocols (such as a contact application, an instant messaging application, an email application, a social networking application, a file sharing application, a cloud storage application, a teleconferencing application, a peer-to-peer file transfer application, etc.), and selection of a corresponding selectable option 5550 causes the device to initiate a sharing operation with the user corresponding to the selected option using the corresponding sharing protocol (e.g., a default sharing protocol corresponding to the selected user or user interface object). In some embodiments, the plurality of selectable options 5550 represent a plurality of users (e.g., users who are physically near the device and have allowed content sharing requests) that can be used to receive shared content using a preset sharing protocol (e.g., a direct peer-to-peer connection). In some embodiments, the plurality of selectable options 5552 represents a plurality of sharing protocols, including sharing using different applications (e.g., an instant messaging application, an email application, a social networking application, a file sharing application, a cloud storage application, a teleconferencing application, a peer-to-peer file transfer application) or other communication protocols provided by the device's operating system. In some embodiments, during movement of the contact, before the sharing user interface 5548 is fully displayed, the reduced preview 5554 and the first and second plurality of selectable options (e.g., including selectable options 5550 and 5552) move upward together in accordance with the movement of the contact and, optionally, after liftoff of the contact (e.g., after the contact has moved upward a threshold amount).

[0327] Figure 5D25 Shown in Figure 5D24Various inputs detected on the sharing user interface 5548 after the sharing user interface is displayed in the . These various inputs include a downward swipe input performed by contact 5562 on the reduced-scale version of the preview 5554, and a tap input performed by contact 5551 on a selectable option corresponding to a first recipient user (e.g., Ursula) among multiple possible recipients. In some embodiments, the multiple recipients shown in the sharing user interface are dynamically updated based on various criteria (e.g., the physical proximity of users who can receive files using a preset sharing protocol (e.g., an ad hoc peer-to-peer direct transfer protocol based on a preset wireless communication protocol (e.g., Bluetooth, WiFi, etc.)), mutual permissions of users communicating with the user of the device within a preset time window for sending files to and receiving files from each other, etc.). These various inputs also include a tap input via contact 5558 on a second selectable option from the plurality of selectable options 5552, the second selectable option corresponding to the first sharing protocol (e.g., sharing via direct peer-to-peer transmission); a tap input via contact 5556 on a third selectable option from the plurality of selectable options 5552, the third selectable option corresponding to the second sharing protocol (e.g., sharing via instant messaging); and a tap input via contact 5560 on a fourth selectable option from the plurality of selectable options 5552, the fourth selectable option corresponding to the third sharing protocol (e.g., sharing via email messaging). In some embodiments, a tap input outside of the area occupied by the preview and selectable options on the sharing user interface 5548 causes the device to dismiss the sharing user interface and restore display of the preview 5156 overlaying the background 5158, or to restore display of the browser user interface 5146 including the hyperlink 5148.

[0328] Figure 5D26 from Figure 5D25 Continuing, and in accordance with some embodiments, it is shown that in response to a downward swipe input made through contact 5562 on the reduced scale version of preview 5554 (or, optionally, a downward swipe input made through contact anywhere on sharing user interface 5548), the device stops displaying the sharing user interface and resumes display of preview 5156 of the second version.

[0329] Figure 5D27 from Figure 5D25Continuing, it is shown that in response to a tap input made via contact 5551 on a selectable option corresponding to the recipient "Ursula" in the plurality of selectable options 5552, transfer of the web page or URL corresponding to the hyperlink 5148 is automatically initiated using a default sharing...

Claims

1. A method comprising: In electronic devices having a display and input elements: displaying on the display a first user interface including a user interface object; While displaying the user interface object, detecting, by the input element, a first input directed to the user interface object; as well as In response to detecting the first input directed to the user interface object: replacing display of the first user interface with a second user interface corresponding to the user interface object after detecting an end of the first input based on determining that the first input does not satisfy a first input threshold and includes movement less than a threshold amount; displaying, upon detecting said end of the first input, a first corresponding user interface area including information corresponding to the user interface object, based on determining that the first input satisfies the first input threshold and includes movement less than the threshold amount and the first input does not satisfy a second input threshold greater than the first input threshold, wherein the first corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of at least a portion of the first user interface at a location outside of the first corresponding user interface area; as well as Based on determining that the first input satisfies the second input threshold and includes movement less than the threshold amount, before detecting the end of the first input, displaying the first corresponding user interface area including information corresponding to the user interface object, wherein the first corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of at least a portion of the first user interface at a position outside of the first corresponding user interface area.

2. The method according to claim 1, comprising: maintaining display of the first corresponding user interface area and the representation of at least a portion of the first user interface at the location outside of the first corresponding user interface area after detecting the end of the first input; while maintaining display of the first corresponding user interface area and the representation of at least a portion of the first user interface at the location outside the first corresponding user interface area, detecting a second input directed to a location outside the first corresponding user interface area; as well as In response to detecting the second input directed to a location outside of the first corresponding user interface area: Stop displaying the first corresponding user interface area; as well as The first user interface is restored to display.

3. The method according to any one of claims 1 to 2, wherein after detecting the end of the first input, displaying a first corresponding user interface area including information corresponding to the user interface object comprises: In response to detecting the end of the first input, displaying the first corresponding user interface area at a first size; as well as The display size of the first corresponding user interface area is changed from the first size to a second size different from the first size.

4. The method of claim 3, wherein before detecting the end of the first input, displaying the first corresponding user interface area including information corresponding to the user interface object comprises: In response to detecting that the first input satisfies the second input threshold before detecting the end of the first input, displaying the first corresponding user interface area at the second size.

5. The method according to any one of claims 1 to 2, comprising: In response to detecting the first input directed to the user interface object: Based on determining that the first input includes movement exceeding the threshold amount before the first input satisfies the first input threshold, at least a portion of the first user interface is moved based on the movement of the first input.

6. The method according to any one of claims 1 or 2, comprising: In response to detecting the first input directed to the user interface object: Based on determining that the first input includes movement exceeding the threshold amount after the first input threshold is satisfied and before the second input threshold is satisfied, moving the user interface object relative to the first user interface based on the movement of the first input.

7. The method of any one of claims 1 or 2, comprising, in response to detecting the first input: Based on determining that the first input includes movement exceeding the threshold amount after the second input threshold is met, initiating a drag operation to move the user interface object.

8. The method of claim 7, wherein initiating the drag operation to move the user interface object comprises: replacing display of the first corresponding user interface area with a representation of the user interface object; as well as The representation of the user interface object is moved according to the movement of the first input.

9. The method of claim 7, comprising, in response to detecting the first input: Initiating a user interface reconfiguration mode of the first user interface based on determining that the first input includes movement exceeding the threshold amount after the second input threshold is satisfied, wherein: In a user interface reconfiguration mode of the first user interface, multiple user interface objects in the first user interface can be moved relative to the first user interface by corresponding inputs including movements exceeding the threshold amount without requiring the corresponding inputs to satisfy the first input threshold.

10. The method of claim 7, comprising, in response to detecting the first input: Based on determining that the first input includes movement exceeding the threshold amount after the second input threshold is satisfied: Based on determining that the first input includes movement in a first direction relative to the first corresponding user interface area, performing a first operation corresponding to a position within the first corresponding user interface area; and Based on determining that the first input includes movement relative to the first corresponding user interface area in a second direction different from the first direction, a drag operation to move the user interface object is initiated.

11. The method of claim 10, wherein initiating the drag operation to move the user interface object comprises: Stop displaying the first corresponding user interface area; as well as The user interface object is moved according to the movement of the first input.

12. The method according to any one of claims 1 or 2, comprising: after detecting the end of the first input and while maintaining display of the first corresponding user interface area on the display, detecting a second input directed to a first portion of the first corresponding user interface area, wherein the first portion of the first corresponding user interface area corresponds to the first operation; as well as In response to detecting the second input directed to the first portion of the first corresponding user interface area, the first operation is performed based on determining that the second input satisfies a first criterion.

13. The method according to any one of claims 1 or 2, comprising: After detecting the end of the first input and while maintaining display of the first corresponding user interface area on the display, detecting a third input directed to the first corresponding user interface area; and In response to detecting the third input directed to the first corresponding user interface area, a drag operation to move the user interface object is initiated based on determining that the third input includes movement exceeding the threshold amount.

14. The method according to any one of claims 1 or 2, comprising: In response to detecting the first input directed to the user interface object: Based on determining that the first input satisfies a third input threshold greater than the second input threshold and includes movement less than the threshold amount, before detecting the end of the first input, a user interface reconfiguration mode of the first user interface is initiated, wherein, in the user interface reconfiguration mode of the first user interface, multiple user interface objects in the first user interface can be moved relative to the first user interface by corresponding inputs including movement exceeding the threshold amount, without requiring the corresponding inputs to satisfy the first input threshold.

15. The method according to claim 14, comprising: during the user interface reconfiguration mode of the first user interface, detecting a fourth input directed to a first location in the first user interface; as well as In response to detecting the fourth input directed to the first location in the first user interface: terminating the user interface reconfiguration mode of the first user interface based on determining that the first position is not occupied by a user interface object; as well as Based on determining that the first position is occupied by a user interface object, maintaining the user interface reconfiguration mode of the first user interface.

16. The method according to any one of claims 1 or 2, comprising: In response to detecting the first input directed to the user interface object: Based on determining that the first input satisfies the first input threshold and includes movement less than the threshold amount, and the first input does not satisfy the second input threshold greater than the first input threshold, providing a visual indication that the first input threshold has been satisfied with movement less than the threshold amount.

17. The method according to any one of claims 1 or 2, comprising: In response to detecting the first input directed to the user interface object: Based on determining that the first input satisfies the first input threshold and includes movement less than the threshold amount, and the first input does not satisfy the second input threshold greater than the first input threshold, providing a non-visual indication indicating that the first input threshold has been satisfied with movement less than the threshold amount.

18. The method according to any one of claims 1 or 2, comprising: In response to detecting the first input directed to the user interface object: Based on determining that the first input satisfies a third input threshold and includes movement less than the threshold amount, and the first input does not satisfy the first input threshold or the second input threshold that is greater than the third input threshold, after detecting the end of the first input, display a second corresponding user interface area including information corresponding to the user interface object, wherein the second corresponding user interface area is different from the first corresponding user interface area, and wherein the second corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of at least a portion of the first user interface at a location outside of the second corresponding user interface area.

19. The method according to claim 18, wherein: The second user interface corresponding to the user interface object is caused to be displayed when the first corresponding user interface area and the second corresponding user interface area are activated by corresponding inputs that do not meet the first input threshold and do not meet the third input threshold.

20. The method according to claim 18, wherein: The second respective user interface area, when activated by a respective input that satisfies the first input threshold and includes movement less than the threshold amount, causes display of the first respective user interface area.

21. The method according to claim 18, wherein: The user interface object is a hyperlink, the second user interface displays a document identified by the hyperlink, the first corresponding user interface area displays a preview of the document identified by the hyperlink, and the second corresponding user interface area displays a document address of the document.

22. An electronic device, comprising: monitor; Input components; one or more processors; as well as A memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs comprising instructions for: displaying on the display a first user interface including a user interface object; While displaying the user interface object, detecting, through the input element, a first input directed to the user interface object; and In response to detecting the first input directed to the user interface object: replacing display of the first user interface with a second user interface corresponding to the user interface object after detecting an end of the first input based on determining that the first input does not satisfy a first input threshold and includes movement less than a threshold amount; in response to determining that the first input satisfies the first input threshold and includes movement less than the threshold amount and the first input does not satisfy a second input threshold greater than the first input threshold, after detecting the end of the first input, displaying a first corresponding user interface area including information corresponding to the user interface object, wherein the first corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of at least a portion of the first user interface at a location outside of the first corresponding user interface area; Based on determining that the first input satisfies the second input threshold and includes movement less than the threshold amount, before detecting the end of the first input, displaying the first corresponding user interface area including information corresponding to the user interface object, wherein the first corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of at least a portion of the first user interface at a position outside of the first corresponding user interface area.

23. The electronic device of claim 22, wherein the one or more programs include instructions for executing the method of any one of claims 2 to 21.

24. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by an electronic device having a display and an input element, cause the electronic device to: displaying on the display a first user interface including a user interface object; While displaying the user interface object, detecting, by the input element, a first input directed to the user interface object; as well as In response to detecting the first input directed to the user interface object: replacing display of the first user interface with a second user interface corresponding to the user interface object after detecting an end of the first input based on determining that the first input does not satisfy a first input threshold and includes movement less than a threshold amount; displaying, upon detecting said end of the first input, a first corresponding user interface area including information corresponding to the user interface object, based on determining that the first input satisfies the first input threshold and includes movement less than the threshold amount and the first input does not satisfy a second input threshold greater than the first input threshold, wherein the first corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of at least a portion of the first user interface at a location outside of the first corresponding user interface area; as well as Based on determining that the first input satisfies the second input threshold and includes movement less than the threshold amount, before detecting the end of the first input, displaying the first corresponding user interface area including information corresponding to the user interface object, wherein the first corresponding user interface area is displayed to cover at least a portion of the first user interface while maintaining a representation of at least a portion of the first user interface at a position outside of the first corresponding user interface area.

25. The computer-readable storage medium of claim 24, wherein the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method of any one of claims 2 to 21.

Images